Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments filed 04/25/2025 on pages 8-11 of Remarks regarding the rejection under 35 U.S.C. 101 with respect to claims 1-30 have been fully considered but they are not persuasive.
Beginning on page 8, Applicant asserts that under 101 Step 1 and 2 the claims are not directed to a mental process because the claims are directed to a specific, technological process for generating a “third AI model” that comprises shared features of two models, and for producing “inference data” based on that “third AI model”. However, Examiner respectfully disagrees. MPEP 2106.04(a)(2)(III)(c) talks about mental processes on a generic computer which includes “parallel processing units”. Also, see MPEP 2106.04(d) and 2106.05(f). Also see, 2106.5(g)
A claim reciting a judicial exception is not directed to the judicial exception if it also recites additional elements demonstrating that the claim as a whole integrates the exception into a practical application. 2106.04(d)(1); Generating a third model is an additional element.
It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. 2106.05(a); and
If it is asserted that the invention improves upon conventional functioning of a computer, or upon conventional technology or technological processes, a technical explanation as to how to implement the invention should be present in the specification. That is, the disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. See updated rejection below.
Applicant’s arguments filed 04/25/2025 on pages 8-11 of Remarks regarding the rejection under 35 U.S.C. 102 with respect to claims 1-30 have been fully considered but they are not persuasive.
Beginning on page 11 regarding the rejection under 35 U.S.C. 102, Applicant asserts that with respect to claim 19, Zhou does not teach “identifying a plurality of neural network layers shared by a first trained model and a second trained model”, however Examiner respectfully disagrees. Zhou shows on page 7, Section 4.1 “Figure 2: Test error on MNIST by continually sharing neurons in (a) the first and (b) the second fully connected layers of two dense LeNet-300-100 networks till the merged layers are fully shared.” The first and second layers are the plurality of layers and a fully shared layer is a shared layer.
Beginning on page 12 regarding the rejection under 35 U.S.C. 102, Applicant asserts that with respect to claim 19, Zhou does not teach “converting the first trained model to a first data structure and the second trained model to a second data structure, wherein generating the derived model is based on the first data structure and the second data structure.” However, Examiner respectfully disagrees. The neural network’s weight matrices are data structures under broadest reasonable interpretation and the conversion to a data structure reads on reorganizing the model’s parameters into a structured form as shown in Algorithm 1, line 8 and Section 3.2. Therefore, “generating a derived model” is taught because MC is built directly from those reorganized representations.
Applicant’s arguments on pages 13-16 of Remarks regarding the rejection under 35 U.S.C. 103 with respect to claims 1-30 have been fully considered but they are not persuasive.
Beginning on page 13 regarding the rejection under 35 U.S.C. 103, Applicant asserts that with respect to claim 1, Zhou does not teach converting trained models into data structures. However, Examiner respectfully disagrees. The neural network’s weight matrices are data structures under broadest reasonable interpretation and the conversion to a data structure reads on reorganizing the model’s parameters into a structured form as shown in Algorithm 1, line 8 and Section 3.2. In addition, Zhou shows on page 7, Section 4.1 “Figure 2: Test error on MNIST by continually sharing neurons in (a) the first and (b) the second fully connected layers of two dense LeNet-300-100 networks till the merged layers are fully shared.” The first and second layers are the plurality of layers and a fully shared layer is a shared layer. Therefore, generating a third AI model that comprises “at least one common feature of the first AI model and the second AI model” is taught because of the layers being shared.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-30 are further rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Claim 1
Step 1: The claim recites a method; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: The claim recites, inter alia:
identifying at least one common feature of a first artificial intelligence (Al) model and a second Al model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying a common feature, such as a neuron or layer of a first and second AI model, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally identify the presence of a layer or neuron that is shared between two models.
converting the first Al model to a first data structure and the second Al model to a second data structure, wherein generating the third Al model is based on the first data structure and the second data structure: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of converting models to data structures, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generate an output comprising inference data associated with the first Al model and the second Al model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating an output comprising inference data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally generate an inference associated with a model.
Step 2A Prong 2: Claim 1 does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “generating, using one or more parallel processing units, a third Al model based on the first data structure and the second data structure, the third Al model generated to comprise the at least one common feature of the first Al model and the second Al model; and processing the third Al model to”.
The additional elements of “using one or more parallel processing units” amount to generic computer components used as a tool to perform an existing process. The additional elements of “generating, using one or more parallel processing units, a third Al model based on the first Al model and the second Al model, the third Al model generated to comprise the at least one common feature of the first Al model and the second Al model; and processing the third Al model to” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the the third AI model is broadly generated from the first and second AI models and how the third AI model is broadly processed to generate output data. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Thus, even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claim is thus directed to the abstract idea
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “using one or more parallel processing units” amount to generic computer components used as a tool to perform an existing process. The additional elements of “generating, using one or more parallel processing units, a third Al model based on the first Al model and the second Al model, the third Al model generated to comprise the at least one common feature of the first Al model and the second Al model; and processing the third Al model to” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the the third AI model is broadly generated from the first and second AI models and how the third AI model is broadly processed to generate output data. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 2
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein processing the third Al model is performed by a first processor and generating the third Al model is performed by a second processor that is different than the first processor” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 3
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU)” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 4
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
determining a data structure type for the third Al model based on a complexity of the first and second Al models: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of determining a data structure type for the third model based on a complexity, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is subject-matter ineligible.
Claim 5
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the at least one common feature of the first Al model and the second Al model is at least a portion of a neural network architecture used by the first Al model and the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 7
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
wherein generating the third Al model comprises including the plurality of the nodes in the third Al model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deciding to include nodes in a model, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “wherein each of the first data structure and the second data structure comprises nodes and at least a plurality of the nodes are common between the first and second data structures” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 8
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 9
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
generating a data structure corresponding to an Al model architecture that is shared by the first Al model and the second Al model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a data structure that is shared between models, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is subject-matter ineligible.
Claim 10
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the data structure is at least one of: a graph data structure, a tree data structure, or a data structure comprising a hash map” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 11
Step 1: The claim recites a system; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: The claim recites, inter alia:
generate at least one data structure representing the first and second AI models: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generate data: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2: Claim 1 does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “a first processor to: receive a first artificial intelligence (AI) model and a second Al model, the first Al model including one or more neural network layers also comprised in the second Al model; and generate a third Al model comprising the one or more neural network layers comprised in the first Al model and the second Al model; a second processor to: receive the third Al model;” and “using the third AI model”.
The additional elements of “a first processor to”, “a second processor to” amount to generic computer components used as a tool to perform an existing process. The additional elements of “reformat the first and second AI models” and “generate a third Al model comprising the one or more neural network layers comprised in the first Al model and the second Al model” and “using the third AI model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the third AI model is broadly generated or how the third AI model is used to generate data. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional elements “receive a first artificial intelligence (AI) model and a second Al model” and “receive the third Al model” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)).
The additional element of “the first Al model including one or more neural network layers also comprised in the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h).
Thus, even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claim is thus directed to the abstract idea
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “a first processor to”, “a second processor to” amount to generic computer components used as a tool to perform an existing process. The additional elements of “generate a third Al model comprising the one or more neural network layers comprised in the first Al model and the second Al model” and “using the third AI model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the third AI model is broadly generated or how the third AI model is used to generate data. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional elements “receive a first artificial intelligence (AI) model and a second Al model” and “receive the third Al model” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and is are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”).
The additional element of “the first Al model including one or more neural network layers also comprised in the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 12
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the data is based on input data associated with the first Al model and the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 13
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the data is inference data based on the input data associated with the first Al model and the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 14
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first and second Al models share a common neural network architecture” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 15
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first processor is a central processing unit (CPU) and the second processor is a graphics processing unit (GPU)” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 16
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
process the first input data and the second input data using the third Al model to generate the data: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of processing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional elements of “using the third AI model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the third AI model is broadly used to process data. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)). The additional elements “receive first input data to be processed using the first Al model and second input data to be processed by the second Al model;” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and is are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 17
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
process the first parameter data and the second parameter data to configure the third Al model before using the third Al model to generate the data: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of processing data to configure a model to generate data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional elements of “the second processor” amount to generic computer components used as a tool to perform an existing process. The additional elements of “configure the first Al model and second parameter data to at least configure the second Al model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the first and second AI models are broadly configured. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)). The additional elements “receive first parameter data to at least configure the first Al model and second parameter data to at least configure the second Al model” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and is are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 18
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the data generated by the third Al model is inference data comprising first inference data associated with the first Al model and second inference data associated with the second Al model” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 19
Step 1: The claim recites a non-transitory machine readable medium; therefore, it is directed to the statutory category of a manufacture.
Step 2A Prong 1: The claim recites, inter alia:
identify a plurality of neural network layers shared by a first trained model and a second trained mode: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying neural network layers that are shared, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
convert the first trained model to a first data structure and the second trained model to a second data structure, wherein generating the derived model is based on the first data structure and the second data structure; and: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of converting models to data structures, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2: Claim 1 does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “based at least in part on the first and second data structures, generate a derived model based on the first trained model and the second trained model, the derived model generated to comprise at least the plurality of neural network layers shared by the first trained model and the second trained model”.
The additional elements of “generate a derived model based on the first trained model and the second trained model, the derived model generated to comprise at least the plurality of neural network layers shared by the first trained model and the second trained model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the derived model is generated based on the first and second trained models. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Thus, even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claim is thus directed to the abstract idea
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “generate a derived model based on the first trained model and the second trained model, the derived model generated to comprise at least the plurality of neural network layers shared by the first trained model and the second trained model” amount to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the derived model is generated based on the first and second trained models. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 20
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first trained model and the second trained model share the same neural network model architecture” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 21
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the same neural network model architecture is based on at least one of: a U-net neural network architecture, a V-net neural network architecture, or a SegNet neural network architecture” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 22
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
process the derived model to generate an output comprising inference data associated with the first trained model and the second trained model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of processing a model to generate output, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is subject-matter ineligible.
Claim 23
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein processing the derived model to generate the output comprising the inference data is to be performed by first instructions in the set of instructions executed on a first processor of the one or more processors and wherein generating the derived model based on the first trained model and the second trained model is to be performed by second instructions in the set of instructions executed on a second processor of the one or more processors” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 24
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first processor is a graphics processing unit (GPU), and the second processor is a central processing unit (CPU)” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 25
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
determine a data structure type for the derived model based on a complexity of the first and second trained models: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of determining a data structure type for a model based on a complexity, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is subject-matter ineligible.
Claim 27
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
including the nodes that are common between the first and second data structures in the derived model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deciding to include common nodes in a model, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “wherein each of the first data structure and the second data structure comprises nodes and the nodes are common between the first and second data structures” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 28
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 29
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
generating a data structure corresponding to an artificial intelligence (AI) model architecture that is shared by the first trained model and the second trained model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating data structures, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is subject-matter ineligible.
Claim 30
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the data structure is a graph data structure, a tree data structure, or a data structure comprising a hash map” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 19-22, 25 and 27-30 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhou et al. (Zhou et al, "Multi-Task Zipping via Layer-wise Neuron Sharing", Mar. 13, 2019, arXiv:1805.09791v2, pp. 1-11, hereinafter "Zhou").
Regarding claim 19, Zhou discloses [a] non-transitory machine-readable medium including stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least: (Page 6, Experiments; a computer readable medium including processor-executable instructions is inherently used to implement the Experiments section of Zhou)
identify a plurality of neural network layers shared by a first trained model and a second trained model; and (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, identify a plurality of neural network layers (shaded in gray) shared by a first trained model and a second trained model. The figure shows the first AI model on left side of Figure 1A and the second AI model on the right side of Figure 1A. l; and see generally §3.2)
convert the first trained model to a first data structure and the second trained model to a second data structure, wherein generating the derived model is based on the first data structure and the second data structure; and (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, converting (which has been performed to produce the image) the first Al model to a first data structure (such as a graph data structure) and the second Al model to a second data structure (another graphical data structure), wherein generating the third Al model is based on the first data structure and the second data structure, and this is disclosed in Figure 1B as a zipped neural network M^C).
based at least in part on the first and second data structures, generate a derived model comprising at least the plurality of neural network layers shared by the first trained model and the second trained model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, generating a derived model (M^C) that is a zipped combination of the first and second AI models (M^A and M^B) and it comprises one or more NN layers (shaded in gray in Figure 1B) shared by the first trained model and the second trained model).
Regarding claim 20, the rejection of claim 19 is incorporated and Zhou further discloses wherein the first trained model and the second trained model share the same neural network model architecture ((Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein the first trained model and the second trained model share the same neural network model architecture because they each comprise three layers with the same number of neurons in each layer).
Regarding claim 21, the rejection of claim 19 is incorporated and Zhou further discloses wherein the same neural network model architecture is based on at least one of: a U-net neural network architecture, a V-net neural network architecture, or a SegNet neural network architecture (Page 8, §4.2; the section discloses using a neural network model architecture called VGG-16 which is a type of V-net NN architecture that is used for image segmentation).
Regarding claim 22, the rejection of claim 19 is incorporated and Zhou further discloses process the derived model to generate an output comprising inference data associated with the first trained model and the second trained model (Page 7; “Table 1 summarizes the errors of each LeNet pair before zipping (errA and errB), after fully merged with retraining (re-errC) and the number of retraining iterations involved (# re-iter)”, which discloses wherein the data generated by the third Al model is inference data (re-errC) comprising first inference data associated with the first Al model (errA( and second inference data associated with the second Al model (errB); and Table 1).
Regarding claim 25, the rejection of claim 19 is incorporated and Zhou further discloses determining a data structure type for the derived model based on a complexity of the first and second Al models (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, determining a data structure type such as a graph data structure for the derived model (zipped model), and the figure discloses the neural network zipping method in the form of zipping graph data structural representations of neural networks. The determining is further based on a complexity such as a number of neurons in a neural network layer for each of the first and second AI models (M^A and M^B)).
Regarding claim 27, the rejection of claims 19 and 26 are incorporated and Zhou further discloses wherein each of the first data structure and the second data structure comprises nodes and the nodes are common between the first and second data structures, and wherein generating the derived model comprises including the nodes that are common between the first and second data structures in the derived model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein each of the first data structure and the second data structure comprises nodes (the nodes are interpreted as neurons which are depicted in the figure as circles) and at least a plurality of the nodes are common between the first and second data structures (the common nodes are the neurons highlighted in gray in the figure), and wherein generating the third Al model comprises including the plurality of the nodes in the third Al model (the figure discloses this in figure 1B with the third AI model M^C including the gray nodes/neurons in the third AI model)).
Regarding claim 28, the rejection of claims 19 and 26 are incorporated and Zhou further discloses wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure, as the neural networks are depicted as graphical data structures in the figure).
Regarding claim 29, the rejection of claim 19 is incorporated and Zhou further discloses wherein generating the derived model comprises generating a data structure corresponding to an artificial intelligence (AI) model architecture that is shared by the first trained model and the second trained model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, generating the derived model comprises generating a data structure corresponding to an artificial intelligence (AI) model architecture that is shared by the first trained model and the second trained model and this is shown as the zipped model depicted as a graph data structure in Figure 1B, and the AI model architecture is shared by the first AI model (M^A) and the second AI model (M^B)).
Regarding claim 30, the rejection of claims 19 and 29 are incorporated and Zhou further discloses wherein the data structure is a graph data structure, a tree data structure, or a data structure comprising a hash map (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein the data structure is at least one of: a graph data structure).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-18, 23, and 24 are rejected under 35 U.S.C. 103 as being obvious over Zhou et al. (Zhou et al, "Multi-Task Zipping via Layer-wise Neuron Sharing", Mar. 13, 2019, arXiv:1805.09791v2, pp. 1-11, hereinafter "Zhou") in view of Englert et al. (US 20200218969 A1, hereinafter "Englert") .
Regarding claim 1, Zhou discloses [a] computer-implemented method, comprising: (Algorithm 1: “Algorithm 1: Multi-task Zipping via Layer-wise Neuron Sharing”, which discloses a computer-implemented method of multi-task zipping)
identifying at least one common feature of a first artificial intelligence (Al) model and a second Al model; (Figure 1; “An illustration of layer zipping via neuron sharing: the number of neurons and the corresponding weight matrices (a) before and (b) after zipping the l-th layers of MA and MB.”, the image, (a) before zipping, clearly shows the common features between two models highlighted in grey; and Page 3, “Neuron Sharing”; the neuron sharing is interpreted as identifying the common features or neurons between two AI models or neural networks MA and MB)
converting the first Al model to a first data structure and the second Al model to a second data structure, wherein generating the third Al model is based on the first data structure and the second data structure (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, converting (which has been performed to produce the image) the first Al model to a first data structure (such as a graph data structure) and the second Al model to a second data structure (another graphical data structure), wherein generating the third Al model is based on the first data structure and the second data structure, and this is disclosed in Figure 1B as a zipped neural network M^C).
generating, … a third Al model based on the first Al model and the second Al model based on the first data structure and the second data structure, the third Al model generated to comprise the at least one common feature of the first Al model and the second Al model; (Page 2-3, Section 3.1, Problem Statement; ”Our goal is to construct a combined model M^c by sharing as many neurons between layers in M^A and M^B as possible such that (i)M^c has minimal loss in inference accuracy for the two tasks and (ii) the construction of MC involves minimal retraining.” Which discloses generated a combined model "M^c” by “sharing as many neurons” or common features between M^A and M^B)
processing the third Al model to generate an output comprising inference data associated with the first Al model and the second Al model; (Page 2-3, section 3.1, Problem Statement; “Our goal is to construct a combined model M^c by sharing as many neurons between layers in M^A and M^B as possible such that (i)M^c has minimal loss in inference accuracy for the two tasks and (ii) the construction of MC involves minimal retraining.”, which discloses, under a broadest reasonable interpretation of the claim language, processing the third AI model (interpreted as a combined model) to generate an output comprising inference data associated with the first AI model and the second AI model (M^A and M^B); and Tables 1-4; the tables disclose, under a broadest reasonable interpretation of the claim language, processing the third AI model (the zipped model Mc) to generate inference output that is reflected as test errors)
Zhou fails to explicitly disclose but Englert discloses using one or more parallel processing units; (Claim 14; The system of claim 10, wherein the system includes multiple processors, the multiple processors comprising at least a CPU, a GPU, and a neural processor.” which discloses parallel processing units or hardware components; and [0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train an AI model such as a neural network).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel processing units of Englert with the method of Zhou to yield the predictable result of generating, using one or more parallel processing units, a third Al model based on the first Al model and the second Al model, the third Al model generated to comprise the at least one common feature of the first Al model and the second Al model. The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 2, the rejection of claim 1 is incorporated and Zhou fails to explicitly disclose but Englert discloses wherein processing the third Al model is performed by a first processor and generating the third Al model is performed by a second processor that is different than the first processor ([0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train an AI model such as a neural network, and wherein processing or executing a third AI model (zipped model M^C) is performed by a first processor (such as a CPU) and generating the third AI model (zipped model M^C) is performed by a second processor (such as a GPU) that is different from the first processor).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel or heterogeneous processing units of Englert with the method of Zhou to yield the predictable result of wherein processing the third Al model is performed by a first processor and generating the third Al model is performed by a second processor that is different than the first processor. The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 3, the rejection of claims 1 and 2 are incorporated and Zhou fails to explicitly disclose but Englert discloses wherein the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU) ([0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train an AI model such as a neural network, and wherein processing or executing a third AI model (zipped model M^C) is performed by a first processor (such as a CPU) and generating the third AI model (zipped model M^C) is performed by a second processor (such as a GPU) that is different from the first processor).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel or heterogeneous processing units of Englert with the method of Zhou to yield the predictable result of wherein the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU). The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 4, the rejection of claim 1 is incorporated and Zhou further discloses determining a data structure type for the third Al model based on a complexity of the first and second Al models (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, determining a data structure type such as a graph data structure, and the figure discloses the neural network zipping method in the form of zipping graph data structural representations of neural networks. The determining is further based on a complexity such as a number of neurons in a neural network layer for each of the first and second AI models (M^A and M^B)).
Regarding claim 5, the rejection of claim 1 is incorporated and Zhou further discloses wherein the at least one common feature of the first Al model and the second Al model is at least a portion of a neural network architecture used by the first Al model and the second Al model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, a common feature such as shared neurons (depicted in gray) that are shared between or used by the first AI model (M^A) and the second AI model (M^B)).
Regarding claim 6, the rejection of claim 1 is incorporated and Zhou further discloses converting the first Al model to a first data structure and the second Al model to a second data structure, wherein generating the third Al model is based on the first data structure and the second data structure (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, converting (which has been performed to produce the image) the first Al model to a first data structure (such as a graph data structure) and the second Al model to a second data structure (another graphical data structure), wherein generating the third Al model is based on the first data structure and the second data structure, and this is disclosed in Figure 1B as a zipped neural network M^C).
Regarding claim 7, the rejection of claims 1 and 6 are incorporated and Zhou further discloses wherein each of the first data structure and the second data structure comprises nodes and at least a plurality of the nodes are common between the first and second data structures, and wherein generating the third Al model comprises including the plurality of the nodes in the third Al model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein each of the first data structure and the second data structure comprises nodes (the nodes are interpreted as neurons which are depicted in the figure as circles) and at least a plurality of the nodes are common between the first and second data structures (the common nodes are the neurons highlighted in gray in the figure), and wherein generating the third Al model comprises including the plurality of the nodes in the third Al model (the figure discloses this in figure 1B with the third AI model M^C including the gray nodes/neurons in the third AI model)).
Regarding claim 8, the rejection of claims 1 and 6 and 7 are incorporated and Zhou further discloses wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein the first data structure is a first graph data structure and the second data structure is a second graph data structure, as the neural networks are depicted as graphical data structures in the figure).
Regarding claim 9, the rejection of claim 1 is incorporated and Zhou further discloses wherein generating the third Al model comprises generating a data structure corresponding to an Al model architecture that is shared by the first Al model and the second Al model (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, generating the third Al model comprises generating a data structure corresponding to an Al model architecture that is shared by the first Al model and the second Al model and this is shown as the zipped model depicted as a graph data structure in Figure 1B, and the AI model architecture is shared by the first AI model (M^A) and the second AI model (M^B)).
Regarding claim 10, the rejection of claims 1 and 9 are incorporated and Zhou further discloses wherein the data structure is at least one of: a graph data structure, a tree data structure, or a data structure comprising a hash map (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, wherein the data structure is at least one of: a graph data structure).
Regarding claim 11, Zhou discloses [a] system, comprising: (Abstract; “We propose Multi-Task Zipping (MTZ), a framework to automatically merge correlated, pre-trained deep neural networks for cross-model compression”, which discloses a system for merging neural networks for compression)
receive a first artificial intelligence (AI) model and a second Al model, the first Al model including one or more neural network layers also comprised in the second Al model; (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, receiving a first AI model (M^A) and a second AI model (M^B). The figure shows the first AI model on left side of Figure 1A and the second AI model on the right side of Figure 1A. The figure further discloses the first AI model including one more NN layers (shaded in gray) that are comprised in the second AI model; and see generally §3.2)
reformat the first and second AI models to generate at least one data structure representing the first and second AI models; (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, converting (which has been performed to produce the image) the first Al model to a first data structure (such as a graph data structure) and the second Al model to a second data structure (another graphical data structure), wherein generating the third Al model is based on the first data structure and the second data structure, and this is disclosed in Figure 1B as a zipped neural network M^C).
based on the at least one data structure, generate a third Al model comprising the one or more neural network layers comprised in the first Al model and the second Al model; (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, generating a third AI model (M^C) that is a zipped combination of the first and second AI models (M^A and M^B) and it comprises one or more NN layers (shaded in gray in Figure 1B) comprised in the first and second AI models)
generate data using the third Al model (Page 2-3, section 3.1, Problem Statement; “Our goal is to construct a combined model M^c by sharing as many neurons between layers in M^A and M^B as possible such that (i)M^c has minimal loss in inference accuracy for the two tasks and (ii) the construction of MC involves minimal retraining.”, which discloses, under a broadest reasonable interpretation of the claim language, processing the third AI model (interpreted as a combined model) to generate a data output comprising inference data associated with the first AI model and the second AI model (M^A and M^B); and Tables 1-4; the tables disclose, under a broadest reasonable interpretation of the claim language, processing the third AI model (the zipped model Mc) to generate inference output that is reflected as test errors)
Zhou fails to explicitly disclose but Englert discloses a first processor to: (Claim 14; The system of claim 10, wherein the system includes multiple processors, the multiple processors comprising at least a CPU, a GPU, and a neural processor.” which discloses parallel processing units or hardware components that are used to generate and execute/test/validate neural network models; and [0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train and execute or test or validate an AI model such as a neural network)
a second processor to: receive the third Al model; and (Claim 14; The system of claim 10, wherein the system includes multiple processors, the multiple processors comprising at least a CPU, a GPU, and a neural processor.” which discloses parallel processing units or hardware components that are used to generate and execute/test/validate neural network models; and [0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to first receive and then generate or train and execute or test or validate an AI model such as a neural network; and [0023]; “a target device for receiving the software package with the compiled code and for executing the compiled code in a runtime environment”).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the first and second processors of Englert with the method of Zhou to yield the predictable result of a first processor to: receive a first artificial intelligence (AI) model and a second Al model and a second processor to: receive the third Al model; and generate data using the third Al model. The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 12, the rejection of claim 11 is incorporated and Zhou further discloses wherein the data is based on input data associated with the first Al model and the second Al model (Page 6, Algorithm 1; the algorithm discloses, under a broadest reasonable interpretation of the claim language, wherein the data is based on input data associated with the first Al model and the second Al model (the “input” are weight matrices that are associated with a first AI model (M^A) and a second AI model (M^B)).
Regarding claim 13, the rejection of claims 11 and 12 are incorporated and Zhou further discloses wherein the data is inference data based on the input data associated with the first Al model and the second Al model (Tables 1-4; the tables disclose, under a broadest reasonable interpretation of the claim language, wherein the data is inference data (test errors) based on the input data associated with the first Al model and the second Al model).
Regarding claim 14, the rejection of claim 1 is incorporated and Zhou further discloses wherein the first and second Al models share a common neural network architecture (Page 3, Figure 1; the figure discloses, under a broadest reasonable interpretation of the claim language, a common neural network architecture such as shared neurons (depicted in gray) that are shared between or used by the first AI model (M^A) and the second AI model (M^B)).
Regarding claim 15, the rejection of claim 11 is incorporated and Zhou fails to explicitly disclose but Englert discloses wherein the first processor is a central processing unit (CPU) and the second processor is a graphics processing unit (GPU) ([0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train an AI model such as a neural network, and wherein processing or executing a third AI model (zipped model M^C) is performed by a first processor (such as a CPU) and generating the third AI model (zipped model M^C) is performed by a second processor (such as a GPU) that is different from the first processor).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel or heterogeneous processing units of Englert with the method of Zhou to yield the predictable result of wherein the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU). The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 16, the rejection of claim 11 is incorporated and Zhou further discloses receive first input data to be processed using the first Al model and second input data to be processed by the second Al model; and process the first input data and the second input data using the third Al model to generate the data (Page 7; “We experiment on MNIST dataset with the LeNet-300-100 and LeNet-5 networks [14] to recognize handwritten digits from zero to nine”, which discloses a first input data that is received to be processed by the first AI model and second input data to be processed by the second AI model; and Page 7; “Table 1 summarizes the errors of each LeNet pair before zipping (errA and errB), after fully merged with retraining (re-errC) and the number of retraining iterations involved (# re-iter)”, which discloses processing the first and second input data using the third AI model that is zipped; and Table 1).
Regarding claim 17, the rejection of claim 11 is incorporated and Zhou further discloses receive first parameter data to at least configure the first Al model and second parameter data to at least configure the second Al model; and process the first parameter data and the second parameter data to configure the third Al model before using the third Al model to generate the data (Page 6, Algorithm 1; the algorithm discloses receiving first and second parameter data to configure the first and second AI models (weight matrices of M^A and M^B), and processing the parameter data or weight matrices to configure or determine weights of the zipped third AI model before using the third model to generate data or make inferences).
Regarding claim 18, the rejection of claim 11 is incorporated and Zhou further discloses wherein the data generated by the third Al model is inference data comprising first inference data associated with the first Al model and second inference data associated with the second Al model (Page 7; “Table 1 summarizes the errors of each LeNet pair before zipping (errA and errB), after fully merged with retraining (re-errC) and the number of retraining iterations involved (# re-iter)”, which discloses wherein the data generated by the third Al model is inference data (re-errC) comprising first inference data associated with the first Al model (errA( and second inference data associated with the second Al model (errB); and Table 1).
Regarding claim 23, the rejection of claims 19 and 22 are incorporated and Zhou fails to explicitly disclose but Englert discloses wherein processing the derived model to generate the output comprising the inference data is to be performed by first instructions in the set of instructions executed on a first processor of the one or more processors and wherein generating the derived model based on the first trained model and the second trained model is to be performed by second instructions in the set of instructions executed on a second processor of the one or more processors ([0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses wherein processing the derived model to generate the output comprising the inference data is to be performed by first instructions in the set of instructions executed on a first processor of the one or more processors (CPU or GPU to execute the neural network) and wherein generating the derived model based on the first trained model and the second trained model is to be performed by second instructions in the set of instructions executed on a second processor of the one or more processors (CPU or GPU to train the neural network)).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel or heterogeneous processing units of Englert with the method of Zhou to yield the predictable result of wherein processing the derived model to generate the output comprising the inference data is to be performed by first instructions in the set of instructions executed on a first processor of the one or more processors and wherein generating the derived model based on the first trained model and the second trained model is to be performed by second instructions in the set of instructions executed on a second processor of the one or more processors. The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Regarding claim 24, the rejection of claims 19, 22, and 23 are incorporated and Zhou fails to explicitly disclose but Englert discloses wherein the first processor is a graphics processing unit (GPU), and the second processor is a central processing unit (CPU) ([0013]; “utilizing a graphical processing unit (GPU) for training a neural network (NN) model, and also for executing the neural network model on new input data post-training. … A central processing unit (CPU) and memory can also be utilized to instantiate and execute neural network models of various configurations”, which discloses using one or more parallel processing units to generate or train an AI model such as a neural network, and wherein processing or executing a third AI model (zipped model M^C) is performed by a first processor (such as a GPU) and generating the third AI model (zipped model M^C) is performed by a second processor (such as a CPU) that is different from the first processor).
Zhou and Englert are analogous art because both are concerned with optimization of model performance. Before the effective filling date of the claimed invention, it would have been obvious for a person having ordinary skill in the art of neural network model optimizations to combine the parallel or heterogeneous processing units of Englert with the method of Zhou to yield the predictable result of wherein the first processor is a graphics processing unit (GPU), and the second processor is a central processing unit (CPU). The motivation for doing so would be to instantiate and execute neural network models of various configurations (Englert; [0013]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TRAN whose telephone number is (703)756-1525. The examiner can normally be reached M-F 9:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H TRAN/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights