DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 25, 2025 has been entered.
Response to Amendment
New grounds of rejection under 35 U.S.C. 101 to claims 1-9 are made as necessitated by the filed claim amendments.
New grounds of rejection under 35 U.S.C. 103 to claims 1-9 are made as necessitated by the filed claim amendments.
Response to Arguments
Applicant’s arguments with respect to the rejections under 35 U.S.C. 112(b) have been fully considered and are partially persuasive. The previously identified antecedent basis issues in claims 2, 3, 6, and 7 appear to be obviated by the claim amendments. Therefore, the respective rejections under 35 U.S.C. 112(b) have been withdrawn. However, the rejections under 35 U.S.C. 112(b) regarding a relative term to claims 1, 7, and 9 and subsequently claims 2-6 and 8 for their dependency on claims 1 and 7, remain as discussed below.
Applicant's arguments with respect to the rejections under 35 U.S.C. 101 have been fully considered but they are not persuasive. Applicant argues “These amendments cause the claims to be even more clearly directed to a practical application. As discussed in paragraphs [0076]-[0078] of the specification as filed, the claimed embodiment provides the advantage of reducing a size of information needed to construct a neural network. The claim 1 further includes the limitation ‘transmitting the encoded data’. This causes the claims to be directed to a practical application of transmitting a reduced sized encoded data for constructing a neural network”. Examiner respectfully disagrees.
The new limitation in amended independent claims 1, 7, and 9 of “quantizing weight information for each edge weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information” has been analyzed under 35 U.S.C. 101 and have been found to recite mental processes. The portion of the claim that allegedly provides a practical application is part of the judicial exception. Specifically, “quantizing weight information for each edge weight, the quantization being common to a plurality of node indies or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information” is mentally performable since “quantizing …” and “encoding …” are claimed at a high level of generality such that they can encompass mental processes. See MPEP 2106.05(I), “An inventive concept ‘cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself.’” Genetic Techs. Ltd. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016). Therefore, the limitations do not integrate the judicial exception into a practical application because they are part of the judicial exception itself. For more, please see the 35 U.S.C. 101 analysis below.
Applicant’s arguments with respect to the U.S.C. 103 rejections of claims of 1-9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments with respect to the Nonstatutory Double Patenting rejection of claims 1-9 have been considered and are persuasive since a terminal disclaimer has been filed. Therefore, the Nonstatutory Double Patenting rejection is withdrawn.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “reducing a size of edge weight information” in claims 1, 7, and 9 is a relative term which renders the claim indefinite. The term “reducing a size” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It is unclear to what degree and from what baseline, relatively, the “size of edge weight information” is reduced. Claims 2-6 and 8 are rejected for their dependency on claims 1 and 7 respectively.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1,
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
generating encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each edge weight, the quantization being common to a plurality of node indies or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
generating encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each edge weight, the quantization being common to a plurality of node indies or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network (corresponds to evaluation and judgment; “quantizing” and “encoding” are claimed at a high level of generality such that they can encompass mental processes).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “processing circuitry” and “train a neural network, the neural network including at least two layers”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Moreover, the claim recites additional element(s) that amount to insignificant extra-solution activities, which do not integrate a judicial exception into a practical application. For example, the additional element of “transmit the encoded data” amounts to mere data gathering, which is an insignificant extra-solution activity that does not integrate a judicial exception into a practical application. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Moreover, the additional element of “transmit the encoded data” amounts to an insignificant extra-solution activity that is well-understood, routine, and conventional. Similarly, this has been re-evaluated under step 2B and do not amount to significantly more. See MPEP 2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity…i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”; Intellectual Ventures v. Symantec, 838 F.3d 1307, 1321; 120 USPQ2d 1353, 1362 (Fed. Cir. 2016)). Therefore, the claim is not patent eligible.
Regarding Claim 2,
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
encode weight information of edges belonging to each layer, on a bit-plane-by-bit-plane basis from higher bits
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
encode weight information of edges belonging to each layer, on a bit-plane-by-bit-plane basis from higher bits (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “processing circuitry”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 3,
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
encode weight information of edges belonging to each layer identified by the layer header information
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
encode weight information of edges belonging to each layer identified by the layer header information (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “processing circuitry”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 4,
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
encode a difference between an edge weight value and a specific value
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
encode a difference between an edge weight value and a specific value (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “processing circuitry”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 5,
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
encode the edge weight information as base encoded data and enhancement encoded data separately
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
encode the edge weight information as base encoded data and enhancement encoded data separately (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “processing circuitry”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 6,
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
decode the encoded data generated by processing circuitry to generate decoded information
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
decode the encoded data generated by processing circuitry to generate decoded information (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional element of “processing circuitry” and “train the neural network using the decoded information”, as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 7,
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a system, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
generate encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each edge weight, the quantization being common to a plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network
decode the encoded data generated by the first processing circuitry
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
generate encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each edge weight, the quantization being common to a plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network (corresponds to evaluation and judgment; “quantizing” and “encoding” are claimed at a high level of generality such that they can encompass mental processes);
decode the encoded data generated by the first processing circuitry (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “a first data processing device”, “first processing circuitry to train a neural network, the neural network including at least two layers”, “a second data processing device”, “second processing circuitry”, and “create the neural network using information decoded by the decoding unit, and performing data processing using the neural network” as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Moreover, the claim recites additional element(s) that amount to insignificant extra-solution activities, which do not integrate a judicial exception into a practical application. For example, the additional element of “transmit the encoded data” amounts to mere data gathering, which is an insignificant extra-solution activity that does not integrate a judicial exception into a practical application. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Moreover, the additional element of “transmit the encoded data” amounts to an insignificant extra-solution activity that is well-understood, routine, and conventional. Similarly, this has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity…i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”; Intellectual Ventures v. Symantec, 838 F.3d 1307, 1321; 120 USPQ2d 1353, 1362 (Fed. Cir. 2016)). Therefore, the claim is not patent eligible.
Regarding Claim 8,
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a device, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
encode information about a portion of the neural network up to an intermediate layer
perform data processing using, as a feature, data outputted from the intermediate layer of the neural network
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
encode information about a portion of the neural network up to an intermediate layer (corresponds to evaluation and judgment);
perform data processing using, as a feature, data outputted from the intermediate layer of the neural network (corresponds to evaluation and judgment).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “the first processing circuitry” and “the second processing circuitry” as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Regarding Claim 9,
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The following limitation:
generating, …, encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network
as drafted, under the broadest reasonable interpretation, covers mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) but for the recitation of mere instructions to apply an exception language and insignificant extra-solution activity language. In particular, the above limitation in the context of this claim encompasses:
generating, …, encoded data in which model header information that identifies a model of the neural network, layer header information that identifies each layer of the neural network, and layer-by-layer edge weight information are encoded, the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information by quantizing weight information for each weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized weight information, the edge weight information indicating a configuration of the neural network (corresponds to evaluation and judgment; “quantizing” and “encoding” are claimed at a high level of generality such that they can encompass mental processes).
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f). For example, the additional elements of “training a neural network by processing circuitry, the neural network including at least two layers” and “by the processing circuitry” as drafted, amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Moreover, the claim recites additional element(s) that amount to insignificant extra-solution activities, which do not integrate a judicial exception into a practical application. For example, the additional element of “transmitting the encoded data” amounts to mere data gathering, which is an insignificant extra-solution activity that does not integrate a judicial exception into a practical application. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B Analysis: As discussed above with respect to integration of the abstract idea into a practical application, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. This has been re-evaluated under step 2B and does not amount to significantly more. See MPEP 2106.05(f). Mere instructions to apply an exception cannot provide an inventive concept. Moreover, the additional element of “transmitting the encoded data” amounts to an insignificant extra-solution activity that is well-understood, routine, and conventional. Similarly, this has been re-evaluated under step 2B and do not amount to significantly more. See MPEP 2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity…i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”; Intellectual Ventures v. Symantec, 838 F.3d 1307, 1321; 120 USPQ2d 1353, 1362 (Fed. Cir. 2016)). Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 3-9 are rejected under 35 U.S.C. 103 as being unpatentable over Cricri et al. (US 20190311259 A1 (Filed 2018)) in view of Gupta et al. (US 10755172 B2 (Filed 2017); hereinafter Gupta) and further in view of Naumov et al. (US 11468313 B1 (Filed 2018); hereinafter Naumov).
Regarding Claim 1,
Cricri teaches a data processing device comprising:
processing circuitry (Cricri Specification [0079] “The main processing unit 100 may comprise or be implemented as one or more processors or processor circuitry”; discloses processing circuitry) configured to:
Train a neural network, the neural network including at least two layers (Cricri Specification [0006] “at least one processor, to cause the apparatus to receive media content for streaming to a user device; to train a neural network”; discloses using a processor to train a neural network (corresponds to a data processing unit for training a neural network; Cricri Specification [0084] “An example of a feature extractor in deep-learning techniques is included in the Convolutional Neural Network (CNN), shown in FIG. 2. A CNN is composed of one or more convolutional layers, fully connected layers, and a classification layer on top CNNs are easier to train than other deep neural networks and have fewer parameters to be estimated”; discloses performing deep-learning with a Convolutional Neural Network composed of one or more convolutional layers (corresponds to the neural network including at least two layers));
generating encoded data in which model header information that identifies a model of the neural network (Cricri Specification [0081] “Cricri Specification [0094] “there may be different trained neural network instances and/or topologies based on the inpainting operation (face, building, natural content, synthetic content, etc.). The server may communicate with the client which neural network topology type is to be used for inpainting”; discloses communicating to a client what type of neural network topology is being used. Sending information from a server to a client over a network requires data encoding (corresponds to generating encoded data in which model header information that identifies a model of the neural network)), …, and layer-by-layer edge weight information are encoded (Cricri Specification [0102] “The server sends the parameters (weights) of the overfitted neural network to the user device at block 408”; discloses sending weights of a neural network to a user device which implicitly requires encoding (corresponds to layer-by-layer edge weight information are encoded)), the generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information (Cricri Specification [0081] “Data compression, such as image and video compression, comprises reducing the amount of data used to represent certain information. The output of such an operation is a reduced set of data, which occupies less memory space or can be transmitted using less bitrate or bandwidth”; Cricri Specification [0095] “the server may stream the network topology in-band or out-band of/from the video bitstream and have the new topology ready in the client before it is used for inpainting. Furthermore, instead of sending the whole topology and parameters at every update time, the system may send only the difference between the currently used topology and parameters and their updated or latest version, in order to further reduce the bitrate”; discloses only sending a difference of topology and parameters of a neural network to reduce the size of encoded, transmitted data depicting parameters (weights) of a neural network (corresponds to generating the encoded data including reducing a size of edge weight information to generate the layer-by-layer edge weight information))…, the edge weight information indicating a configuration of the neural network (Cricri Specification [0102] “The server sends the parameters (weights) of the overfitted neural network to the user device at block 408”; discloses sending weights of a neural network to a user device. Weights of a neural network indicate the configuration of a neural network (corresponds to the edge weight information indicating a configuration of the neural network)); and
transmit the encoded data (Cricri Specification [0102] “The server sends the parameters (weights) of the overfitted neural network to the user device at block 408”; discloses sending weights of a neural network to a user device (corresponds to transmit). Data must be encoded to be transmitted from a server to a user device (corresponds to the encoded data).
Cricri appears to not explicitly disclose generating encoded data in which … layer header information that identifies each layer of the neural network [are encoded] and quantizing weight information for each edge weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized information.
However, Gupta teaches generating encoded data in which … layer header information that identifies each layer of the neural network [are encoded] (Gupta Specification Col.3, Lines 20-22 “gradients for Bob's input layer are encrypted by Bob, sent to Alice and then decrypted by Alice”; describes a scenario where two networks, Alice and Bob, are sent to each other. Output activations of a particular layer are encrypted and sent from one network to another (corresponds to layer header information that identifies each layer of the neural network)).
Cricri and Gupta are considered to be analogous to the claimed invention because they are in the same field of training neural network models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cricri to incorporate the teachings of Gupta. Doing so would could help securely transfer models from Cricri between computers in a more efficient manner, as suggested by Gupta (Gupta Specification Col.3, Lines 33-35 “In illustrative implementations, the present invention is much more computationally efficient than conventional methods of securely training a multi-party DNN”).
Cricri in view of Gupta appears to not explicitly disclose quantizing weight information for each edge weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices, and encoding the quantized information.
However, Naumov teaches quantizing weight information for each edge weight, the quantization being common to a plurality of node indices or a plurality of edge indices, or the plurality of node indices and the plurality of edge indices (Naumov Col.2, Lines 1-5 “a computer-implemented method for quantizing neural networks via periodic regularization functions may include identifying an artificial neural network that may include a set of nodes interconnected via a set of connections, and training the artificial neural network by, for each connection in the set of connections, determining a quantized weight value associated with the connection”; Col.4, Lines 66-67 “determining the quantized weight value associated with the connection may include associating a loss function with the connection, and the loss function may include a periodic regularization function that describes a relationship between an input value and a weight value of the connection”; discloses determining quantized weights (corresponds to quantizing weight information) associated with each connection of a set of connections between nodes (corresponds to each edge weight) where a loss function including a periodic regularization function is used to determine a quantized weight value for each connection between each node (corresponds to the quantization being common to the plurality of node indices and the plurality of edge indices)), and encoding the quantized information (Naumov Col.5, Lines 29-34 “scale the determined minimum of the loss function and/or round the determined minimum of the loss function to a nearest integer value … then designate the scaled and/or rounded determined minimum of the loss function as the quantized weight value associated with the connection”; discloses determining a quantized weight using a scaled and/or rounded minimum of a loss function (corresponds to encoding the quantized information)).
Cricri, Gupta, and Naumov are considered to be analogous to the claimed invention because they are in the same field of training neural network models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cricri and Gupta to incorporate the teachings of Naumov. Doing so could assist in making compact and more efficient neural networks in Cricri and Gupta as discussed by Naumov (Naumov Col.1, Lines 39-42 “quantization processes may apply techniques to store numbers and/or perform calculations associated with an artificial neural network in more compact and/or more efficient formats”).
Regarding Claim 3,
Cricri in view of Gupta in view of Naumov teaches the data processing device according to claim 1 and the processing circuitry.
Cricri further teaches [encoding] weight information of edges belonging to each layer identified by the layer header information (Cricri Specification [0102] “The server sends the parameters (weights) of the overfitted neural network to the user device at block 408”; discloses sending weights of a neural network to a user device which implicitly requires encoding. Weights indicate the strength of an edge between neurons in layers of a neural network (corresponds to [encoding] weight information of edges belonging to each layer identified by the layer header information)).
Regarding Claim 4,
Cricri in view of Gupta in view of Naumov teaches the data processing device according to claim 1 and the processing circuitry.
Cricri further teaches [encoding] a difference between an edge weight value and a specific value (Cricri Specification [0095] “instead of sending the whole topology and parameters at every update time, the system may send only the difference between the currently used topology and parameters and their updated or latest version”; discloses sending a difference between the current parameters of a neural network (corresponds to an edge weight value) and their updated version (corresponds to a specific value). The difference is sent to a client which would require encoding (corresponds to [encoding] a difference; for more please see Cricri Specification [0095])).
Regarding Claim 5,
Cricri in