DETAILED ACTION
This action is in response to amendments and remarks filed 04/27/2026. Claims 1-20 are pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/27/2026 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to non-statutory subject matter without significantly more.
Claim 1
Step 1: The claim recites “A neural architecture search method”, and is therefore directed to the statutory category of process
Step 2A Prong 1: The claim recites the following judicial exception(s)
generating, based on a search space, a plurality of neural network architectures: This can be performed as a mental process. One can merely identify a search space (e.g., feedforward networks with 2-10 layers) and think of a plurality of architectures within that space.
obtaining, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This can be performed as a mental process. One can merely score each architecture.
determining, based on the trained second neural network, a first target neural network architecture meeting a first preset condition: This can be performed as a mental process. One can merely identify a target architecture meeting some condition.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
A neural architecture search method implemented by a processor of a search system: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
obtaining, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
training, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
training, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
A neural architecture search method implemented by a processor of a search system: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
obtaining, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
training, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
training, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Claim 2
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
Claim 3
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the evaluation indicator values are based on performing inference on the first child models: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 4
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein each of the evaluation indicator values comprises a hardware-related performance value: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model. As stated in paragraph [0017] of the instant specification, a hardware-related performance value can include inference time.
obtaining performance values of a plurality of second child models on second hardware, wherein the performance values are based on performing inference on the second child models: This can be performed as a mental process. One can merely assign a performance value to each model of the second child models, proportional to its inference time.
determining based on the neural network architectures and the performance values based on performing inference on the second child models, a second target neural network architecture meeting a second preset condition: This can be performed mentally. One can merely select the model of the second child models with the highest performance value.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 5
Step 1: The claim recites a process, as in claim 4
Step 2A Prong 1: The claim recites the following further judicial exception(s)
the hardware-related performance value comprises any one or more of model inference time, a quantity of activations, throughput, power consumption, or video random-access memory (RAM) usage: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 6
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the search space comprises an attribute value space of each attribute of a neuron: Generating a plurality of neural network architectures based on a search space can still be performed as a mental process. One can merely identify a plurality of architectures with different neuron attributes.
wherein the neural architecture search method further comprises randomly selecting an attribute value for each attribute from the attribute value space to obtain the neural network architectures: One can mentally derive network architectures with randomly assigned neuron attribute values.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 7
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites the following further judicial exception(s)
further generating, for the user through the application programming interface, the neural network architectures: One can mentally generate a set of neural network architectures.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
providing, to a user, an application programming interface: This amounts to mere transmission of data and is insignificant extra-solution activity (MPEP 2106.05(g)).
further generating, for the user through the application programming interface, the neural network architectures: This is mere instruction to apply a judicial exception to a user through an API in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
providing, by the generator, an application programming interface to a user: This is an instance of transmitting data over a network, a limitation known by the courts to be well-understood, routine, and conventional (MPEP 2106.05(d) II. i.).
further generating, for the user through the application programming interface, the neural network architectures: This is mere instruction to apply a judicial exception to a user through an API in a generic manner (MPEP 2106.05(f)).
Claim 8
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
performing federated learning on each of N initial child models using M datasets to obtain N child models of the plurality of first child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
performing federated learning on each of N initial child models using M datasets to obtain N child models of the plurality of first child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Claim 9
Step 1: The claim recites “A computer cluster”, and is therefore directed to the statutory category of machine
Step 2A Prong 1: The claim recites the following judicial exception(s)
generate, based on a search space, a plurality of neural network architectures: This can be performed as a mental process. One can merely identify a search space (e.g., feedforward networks with 2-10 layers) and think of a plurality of architectures within that space.
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This can be performed as a mental process. One can merely score each architecture.
determine, based on the trained second neural network, a first target neural network architecture meeting a first preset condition: This can be performed as a mental process. One can merely identify a target architecture meeting some condition.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
at least one computer, comprising: a memory configured to store instructions; and a processor coupled to the memory and configured to execute the instructions: This is mere instruction to execute the judicial exceptions with generic computing hardware (MPEP 2106.05(f)).
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
train, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
at least one computer, comprising: a memory configured to store instructions; and a processor coupled to the memory and configured to execute the instructions: This is mere instruction to execute the judicial exceptions with generic computing hardware (MPEP 2106.05(f)).
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
train, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Claim 10
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
wherein the first neural network is configured to receive a representation of a neural network architecture as input and to output a predicted evaluation indicator value: This amounts to mere data gathering and is insignificant extra-solution activity (MPEP 2106.05(g)).
wherein the second neural network is configured to provide neural network architectures according to the search space: This is mere instruction to provide neural network architectures in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
wherein the first neural network is configured to receive a representation of a neural network architecture as input and to output a predicted evaluation indicator value: This is an instance of storing information in and retrieving information from memory, a limitation known to be well-understood, routine, and conventional (MPEP 2106.05(d) II. iv.)
wherein the second neural network is configured to provide neural network architectures according to the search space: This is mere instruction to provide neural network architectures in a generic manner (MPEP 2106.05(f)).
Claim 11
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the evaluation indicator values are based on performing inference on the first child models: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 12
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein each of the evaluation indicator values comprises a hardware-related performance value: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model. As stated in paragraph [0017] of the instant specification, a hardware-related performance value can include inference time.
obtain performance values of a plurality of second child models on second hardware, wherein the performance values are based on performing inference on the second child models: This can be performed as a mental process. One can merely assign a performance value to each model of the second child models, proportional to its inference time.
determine, based on the neural network architectures and the performance values based on performing inference on the second child models, a second target neural network architecture meeting a second preset condition: This can be performed mentally. One can merely select the model of the second child models with the highest performance value.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions on generic computer hardware (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions on generic computer hardware (MPEP 2106.05(f)).
Claim 13
Step 1: The claim recites a machine, as in claim 12
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the hardware-related performance value comprises any one or more of model inference time, a quantity of activations, throughput, power consumption, or video random-access memory (RAM) usage: Obtaining evaluation indicator values can still be performed as a mental process. One can merely assign evaluation indicator values proportional to the speed of inference on each model.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 14
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the search space comprises an attribute value space of each attribute of a neuron: Generating a plurality of neural network architectures based on a search space can still be performed as a mental process. One can merely identify a plurality of architectures with different neuron attributes.
wherein the processor is further configured to execute the instructions to cause the at least one computer to randomly select an attribute value for each attribute from the attribute value space to obtain the neural network architectures: One can mentally derive network architectures with randomly assigned neuron attribute values.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the processor is further configured to execute the instructions to cause the at least one computer to randomly select an attribute value for each attribute from the attribute value space to obtain the neural network architectures: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the processor is further configured to execute the instructions to cause the at least one computer to randomly select an attribute value for each attribute from the attribute value space to obtain the neural network architectures: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
Claim 15
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites the following further judicial exception(s)
further generate the neural network architectures for the user through the application programming interface: One can mentally generate a set of neural network architectures.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions on generic computer hardware (MPEP 2106.05(f)).
provide an application programming interface to a user: This amounts to mere transmission of data and is insignificant extra-solution activity (MPEP 2106.05(g)).
further generate the neural network architectures for the user through the application programming interface: This is mere instruction to apply a judicial exception to a user through an API in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions on generic computer hardware (MPEP 2106.05(f)).
provide an application programming interface to a user: This amounts to mere transmission of data and is insignificant extra-solution activity (MPEP 2106.05(g)).
further generate the neural network architectures for the user through the application programming interface: This is mere instruction to apply a judicial exception to a user through an API in a generic manner (MPEP 2106.05(f)).
Claim 16
Step 1: The claim recites a machine, as in claim 9
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
perform federated learning on each of N initial child models of the plurality of first child models using M datasets to obtain N child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
train each of the N initial child models using the M datasets to obtain N*M child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
divide the N initial child models into M groups of initial child models, wherein the M groups of initial child models are in a one-to-one correspondence with the M datasets, and training the M groups of initial child models using corresponding datasets to obtain M groups of child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
the processor is further configured to execute the instructions: This is mere instruction to execute the judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
perform federated learning on each of N initial child models of the plurality of first child models using M datasets to obtain N child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
train each of the N initial child models using the M datasets to obtain N*M child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
divide the N initial child models into M groups of initial child models, wherein the M groups of initial child models are in a one-to-one correspondence with the M datasets, and training the M groups of initial child models using corresponding datasets to obtain M groups of child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Claim 17
Step 1: The claim recites “A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium”, and is therefore directed to the statutory category of article of manufacture
Step 2A Prong 1: The claim recites the following judicial exception(s)
generate, based on a search space, a plurality of neural network architectures: This can be performed as a mental process. One can merely identify a search space (e.g., feedforward networks with 2-10 layers) and think of a plurality of architectures within that space.
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on hardware: This can be performed as a mental process. One can merely score each architecture.
determine, based on the trained second neural network, a first target neural network architecture meeting a first preset condition: This can be performed as a mental process. One can merely identify a target architecture meeting some condition.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium, and that, when executed by a processor, cause an apparatus to: This is mere instruction to execute the judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
train, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium, and that, when executed by a processor, cause an apparatus to: This is mere instruction to execute the judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on hardware: This is mere instruction to apply a judicial exception with generic computer hardware (MPEP 2106.05(f)).
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2105.05(f)).
train, using the trained first neural network, a second neural network to obtain a trained second neural network: This is mere instruction to train a neural network based on judicial exceptions in a generic manner (MPEP 2106.05(f)).
wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: This is mere instruction to train the plurality of first child models in a generic manner, and to execute a judicial exception (determining the first target network architecture) on generic parallelized computer hardware (MPEP 2106.05(f)).
Claim 18
Step 1: The claim recites an article of manufacture, as in claim 17
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network: This is mere instruction to use a judicial exception as a reward in a reinforcement learning system in a generic manner (MPEP 2106.05(f)).
Claim 19
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
training each of N initial child models of the plurality of first child models using M datasets to obtain N*M child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
training each of N initial child models of the plurality of first child models using M datasets to obtain N*M child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Claim 20
Step 1: The claim recites a process, as in claim 1
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
dividing N initial child models of the plurality of first child models into M groups of initial child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein the M groups of initial child models are in a one-to-one correspondence with M datasets, wherein N is greater than 1, and wherein M is greater than 1; and training the M groups of initial child models using corresponding datasets to obtain M groups of child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
dividing N initial child models of the plurality of first child models into M groups of initial child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein the M groups of initial child models are in a one-to-one correspondence with M datasets, wherein N is greater than 1, and wherein M is greater than 1; and training the M groups of initial child models using corresponding datasets to obtain M groups of child models: This is mere instruction to train models in a generic manner (MPEP 2106.05(f)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-6, 9-14, and 17-28 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable over Zhou et al. (RESOURCE-EFFICIENT NEURAL ARCHITECTS, filed 3/8/2019, US 2019/0354837 A1), hereafter referred to as Zhou, in view of Wen et al. (Neural Predictor for Neural Architecture Search, published 12/2/2019, arXiv:1912.00848v1), hereafter referred to as Wen, and further in view of Singh et al. (LEARNING TO SEARCH DEEP NETWORK ARCHITECTURES, filed 9/5/2018, US 2020/0074296 A1), hereafter referred to as Singh.
Regarding claim 1, Zhou teaches [a] neural architecture search method, implemented by a processor of a search system: “In this patent document, embodiments of a resource-constrained NAS framework, which may be generally referred to as Resource-Efficient Neural Architect (RENA) (search system), are presented. A goal is to automate the process of finding high-performance neural network architectures under different resource constraints with a reasonable amount of search” (Zhou, [0034]); “Aspects of the present disclosure may be encoded upon one or more non-transitory computer-readable media with instructions for one or more processors or processing units to cause steps to be performed.” (Zhou, [0124])
wherein the neural architecture search method comprises:
generating, based on a search space, a plurality of neural network architectures:
“In one or more embodiments, actions of scale and insert are mapped to a search space to define the neural network architectures” (Zhou, [0066]); “In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (plurality of neural network architectures), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
obtaining, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware
“As shown in FIG. 1, in one or more embodiments, a RENA embodiment may comprise two principal networks: a policy network 110 and a value network (or a performance simulation network) 140” (Zhao, [0045]).
“Components may be implemented in software, hardware (first hardware), or a combination thereof” (Zhou, [0023]).
“In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network (child model) is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (evaluation indicator values)” (Zhou, [0048])
“In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (first child models), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
training, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“In one or more embodiments, a performance simulation network (first neural network) takes a target network embedding and a training dataset in terms of size, distribution, and regularity to generate approximated accuracy and training time (evaluation indicator values). Leveraging the embedding network, layer representation may be unified and the information from individual layers may be integrated. Given a set of sample networks, performance curves for each network may be obtained. For each network x, a validation accuracy a, and training time t, may be obtained, for example” (Zhou, [0087])
“If trained jointly, the performance simulation network becomes a value network V (trained first neural network)” (Zhou, [0088])
“The parameters
θ
v
[o]f the value network (first neural network) is updated via gradient descent using
PNG
media_image1.png
55
519
media_image1.png
Greyscale
”(Zhou, [0089]). The output of the network, V, which corresponds to the approximated accuracy and training time (evaluation indicator values) is used to train the network.
training, using the trained first neural network, a second neural network to obtain a trained second neural network: “The parameters
θ
of the policy network (second neural network) may be optimized via gradient descent as follows:
PNG
media_image2.png
146
544
media_image2.png
Greyscale
” (Zhou, [0088]). As can be seen in the formulas above, the value network V (first neural network) is used to train the policy network (second neural network).
determining, based on the trained second neural network, a first target neural network architecture meeting a first preset condition:
“In one or more embodiments, a policy network (second neural network) converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration” (Zhou, [0048])
“a) Training Details” (Zhou, [0102]); “For KWS architecture search … The policy network (second neural network) was trained with the Adam optimization algorithm with a learning rate of 0.0006. An episode size of 5 and a batch size of 10 was used for all experiments, i.e. 10 child models (neural network[s]) are trained concurrently … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network. At the end of each episode, the policy was updated and the best 10 child models were used as the baseline for the new episode.” (Zhou, [0103]).
“b) Results” (Zhou, [0104]); ”TABLE 2 presents the search results for KWS, as well as the optimal architectures. Without any resource constraints, the state-of-the-art accuracy, 95.81 %, can be obtained using an architecture comprising depth-separable convolutions (that apply significant downsampling), followed by gated recurrent units (GRUs) and multiple 2-D convolutions. When aggressive resource constraints are imposed, it was observed that the RENA embodiment can find architectures (first target neural network architecture[s]) that outperform hand-optimized architectures in the literature. A tight model size constraint (first preset condition) results in an optimal architecture composed of GRUs with small hidden units. Similarly, tight constraints (first preset condition[s]) on computational complexity also favor for GRUs with small hidden units. When compute intensity (first preset condition) is considered, an efficient architecture is achieved by enabling most of the computation on 2-D convolutions with large channel size.” (Zhou, [0106])
PNG
media_image3.png
440
1283
media_image3.png
Greyscale
(Zhou, Table 2). RENA finds an optimal architecture with 95.81% accuracy for KWS architecture search, which is based on the trained policy network (second neural network).
Zhou relates to evolutionary child neural architecture searches and is analogous to the claimed invention
While Zhou fails to disclose the further limitations of the claim, Wen discloses a method of training, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“We tried many options for the architecture of the predictor (first neural network). Due to space constraints we will limit our discussion to Graph Convolutional Networks” (Wen, page 2, right column, paragraph 6)
“Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N random architectures (neural network architectures) to generate N (architecture, validation accuracy) pairs and use them to train (supervised training) a regression model (first neural network) that predicts accuracy (evaluation indicator values) based on the architecture. Next, we use this regression model (trained first neural network) to predict the validation accuracies of a large number of random architectures” (Wen, page 1, left column, Abstract)
Wen relates to supervised evaluation of candidate models in a neural architecture search and is analogous to the claimed invention. Zhou teaches a method of evaluating candidate neural network architectures with a score-generating neural network. The claimed invention differs from this method by training the first neural network using supervised learning. Wen teaches a method of training a score-generating neural network using supervised learning. Because both Zhou and Wen teach the use of evaluating candidate network architecture using an evaluator neural network, it would have been obvious to a person of ordinary skill in the art to substitute Zhou’s reinforcement learning-trained value network for Wen’s supervised learning-trained neural predictor to achieve the predictable result of learning to predict model quality based on observed quality measurements (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Wen fails to disclose the further limitations of the claim, Singh discloses a method, wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: “The recurrent neural network may systematically search a space of deep network architectures by emulating the natural evolution of how the architectures may be designed and evaluated for a given task by researchers working in the field of deep learning. For example, the ImageNet dataset has millions of images from 1000 classes and is a benchmark for image classification tasks. The recurrent neural network may model researchers in the field of machine learning as ‘agents’ in a reinforcement learning framework training deep networks (training process) and conducting the architecture search (determining the first target neural network architecture), where multiple researchers may be training and searching in parallel. Over time, various network designs (plurality of first child models) are proposed and validated by the agents, and design elements from the highest performing architectures may be combined and incorporated to augment previous architectures or create a new architecture (first target neural network architecture) to perform well at the classification task.” (Singh, [0015]); “Multiple deep network architectures may allow for training and performance evaluations of the deep network architectures (plurality of first child models) in parallel” (Singh, [0057])
Singh relates to parallelized neural architecture searches and is analogous to the claimed invention. The existing combination teaches a method of performing a neural architecture search by evaluating and modifying a plurality of evolving child networks. The claimed invention improves upon this method by parallelizing candidate architecture search and training. Singh teaches a method of parallelizing candidate architecture search and training, applicable to The existing combination. A person of ordinary skill in the art would have recognized that parallelizing the search and candidate model training of The existing combination would lead to the predictable result of processing, modifying, and searching through candidate networks independently, and would improve the known device by decreasing computation time needed to perform candidate network operations (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 2, the rejection of claim 1 is incorporated. Zhou further discloses a method, wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network:
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
“In this section, embodiments of the overall reinforcement learning (RL) framework of RENA and the corresponding search space are explained. In one or more embodiments, the framework comprises a policy network to generate an action or actions that define the neural network architecture. In one or more embodiments, the environment outputs the performance of the trained neural network, as well as its resource use. In one or more embodiments, a policy gradient with accumulated rewards was used to train the policy network” (Zhou, [0056])
Regarding claim 3, the rejection of claim 1 is incorporated. Zhou further discloses a method, wherein the evaluation indicator values are based on performing inference on the first child models:
“the trained adapted neural network architecture (trained child model) may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (evaluation indicator values)” (Zhou, [0048]).
Regarding claim 4, the rejection of claim 1 is incorporated. Zhou further discloses a method, wherein
each of the evaluation indicator values comprises a hardware-related performance value: “the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency (inference time), etc.) (evaluation indicator values)” (Zhou, [0048]). As noted in paragraph [0017] of the instant specification, model inference time is a form of hardware-related performance values.
the neural architecture search method further comprises:
obtaining performance values of a plurality of second child models on second hardware, wherein the performance values are based on performing inference on the second child models:
“In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network (child model) is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (performance values)” (Zhou, [0048])
“Keyword spotting (KWS) systems (second hardware) aim to detect a particular keyword from a continuous stream of audio. They are commonly used in conversational human-machine interfaces, such as in smart home systems or virtual assistants. A high detection accuracy and a low latency is critical to enable satisfactory user experience. In addition, KWS systems are typically deployed on a wide range of devices with different resource constraints” (Zhou, [0100])
“For KWS architecture search, layer-by-layer search was considered … An episode size of 5 and a batch size of 10 was used for all experiments, i.e. 10 child models (second child models) are trained concurrently … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network” (Zhou, [0103]). Computation of performance values for each child model are inherent in each model being evaluated in this system.
determining, based on the neural network architectures and the performance values based on performing inference on the second child models, a second target neural network architecture meeting a second preset condition:
“For KWS architecture search … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network. At the end of each episode, the policy was updated and the best 10 child models (neural network architectures) were used as the baseline for the new episode” (Zhou, [0103]). Computation of performance values are inherently used to evaluate child models and find the best set.
“the RENA embodiment learns to generate models (second target neural network architecture[s]) that meet both constraints after about 120 searched models” (Zhou, [0105])
“When aggressive resource constraints are imposed, it was observed that the RENA embodiment can find architectures that outperform hand-optimized architectures in the literature. A tight model size constraint results in an optimal architecture composed of GRUs with small hidden units. Similarly, tight constraints on computational complexity (second preset condition) also favor for GRUs with small hidden units” (Zhou, [0106])
Regarding claim 5, the rejection of claim 4 is incorporated. Zhou further discloses a method, wherein the hardware-related performance value comprises any one or more of model inference time, a quantity of activations, throughput, power consumption, or video random-access memory (RAM) usage: “In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency (inference time), etc.) (performance values)” (Zhou, [0048])
Regarding claim 6, the rejection of claim 1 is incorporated. Zhou further discloses a method, wherein
the search space comprises an attribute value space of each attribute of a neuron:
“In one or more embodiments, actions of scale and insert are mapped to a search space to define the neural network architectures. Two example approaches to defining search spaces are presented next” (Zhou, [0066]).
“a) Layer-by-Layer Search Embodiments” (Zhou, [0067]); “layer-by-layer search aims to find the optimal architecture with a search granularity of predefined layers … In one or more embodiments, the neural network architecture is defined (705) by stacking these layers, potentially with skip connections between them. For each feature, an LSTM in the policy network chooses (710) the layer type and the corresponding hyperparameters (e.g., filter width) (each attribute of a neuron). In one or more embodiments, the location of the inserted layer is denoted by an input source identifier (e.g., "Src1"), where the new layer gets its input data from. To support skip connection, the Insert Controller generates operation "add" that connects layer outputs of source identifiers (e.g., "Src1" and "Src2") with either an addition or a concatenation operation” (Zhou, [0068]).
the neural architecture search method further comprises randomly selecting an attribute value for each attribute from the attribute value space to obtain the neural network architectures:
“In one or more embodiments, the policy network (generator) generates (1205) a batch of actions a, m which produce a series of child networks, which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11) … in one or more embodiments, the starting network architecture for a branch may be: varied (e.g., randomly varied) from an initial architecture input 1110 (particularly, if this is the first episode)” (Zhou, [0076]).
“The Random Search generates random actions (i.e., insert, remove, and scale) and also selects the hyperparameters (each attribute) of each layer randomly with uniform probability” (Zhou, [0105]).
Regarding claim 9, Zhou teaches a computer cluster comprising: at least one computer, comprising:
a memory configured to store instructions: “The computing system 1500 may also include a storage controller 1507 for interfacing with one or more storage devices 1508 each of which includes a storage medium (memory) such as magnetic tape or disk, or an optical medium that might be used to record programs of instructions for operating systems, utilities, and applications, which may include embodiments of programs that implement various aspects of the present disclosure” (Zhou, [0122]).
a processor coupled to the memory and configured to execute the instructions to cause the at least one computer to: “As illustrated in FIG. 15, the computing system 1500 includes one or more central processing units (CPU) 1501 that provides computing resources and controls the computer“ (Zhou, [0121]); “Embodiments of the present disclosure may be implemented in whole or in part as machine-executable instructions that may be in program modules that are executed by a processing device” (Zhou, [0125]).
Zhou’s instructions to cause the at least one computer to:
generate, based on a search space, a plurality of neural network architectures:
“In one or more embodiments, actions of scale and insert are mapped to a search space to define the neural network architectures” (Zhou, [0066]); “In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (plurality of neural network architectures), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on first hardware
“Components may be implemented in software, hardware (first hardware), or a combination thereof” (Zhou, [0023]).
“In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network (child model) is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (evaluation indicator values)” (Zhou, [0048])
“In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (first child models), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“In one or more embodiments, a performance simulation network (first neural network) takes a target network embedding and a training dataset in terms of size, distribution, and regularity to generate approximated accuracy and training time (evaluation indicator values). Leveraging the embedding network, layer representation may be unified and the information from individual layers may be integrated. Given a set of sample networks, performance curves for each network may be obtained. For each network x, a validation accuracy a, and training time t, may be obtained, for example” (Zhou, [0087])
“If trained jointly, the performance simulation network becomes a value network V (trained first neural network)” (Zhou, [0088])
“The parameters
θ
v
[o]f the value network (first neural network) is updated via gradient descent using
PNG
media_image1.png
55
519
media_image1.png
Greyscale
”(Zhou, [0089]). The output of the network, V, which corresponds to the approximated accuracy and training time (evaluation indicator values) is used to train the network.
train, using the trained first neural network, a second neural network to obtain a trained second neural network: “The parameters
θ
of the policy network (second neural network) may be optimized via gradient descent as follows:
PNG
media_image2.png
146
544
media_image2.png
Greyscale
” (Zhou, [0088]). As can be seen in the formulas above, the value network V (first neural network) is used to train the policy network (second neural network).
determine, based on the trained second neural network, a first target neural network architecture meeting a first preset condition:
“In one or more embodiments, a policy network (second neural network) converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration” (Zhou, [0048])
“a) Training Details” (Zhou, [0102]); “For KWS architecture search … The policy network (second neural network) was trained with the Adam optimization algorithm with a learning rate of 0.0006. An episode size of 5 and a batch size of 10 was used for all experiments, i.e. 10 child models (neural network[s]) are trained concurrently … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network. At the end of each episode, the policy was updated and the best 10 child models were used as the baseline for the new episode.” (Zhou, [0103]).
“b) Results” (Zhou, [0104]); ”TABLE 2 presents the search results for KWS, as well as the optimal architectures. Without any resource constraints, the state-of-the-art accuracy, 95.81 %, can be obtained using an architecture comprising depth-separable convolutions (that apply significant downsampling), followed by gated recurrent units (GRUs) and multiple 2-D convolutions. When aggressive resource constraints are imposed, it was observed that the RENA embodiment can find architectures (first target neural network architecture[s]) that outperform hand-optimized architectures in the literature. A tight model size constraint (first preset condition) results in an optimal architecture composed of GRUs with small hidden units. Similarly, tight constraints (first preset condition[s]) on computational complexity also favor for GRUs with small hidden units. When compute intensity (first preset condition) is considered, an efficient architecture is achieved by enabling most of the computation on 2-D convolutions with large channel size.” (Zhou, [0106])
PNG
media_image3.png
440
1283
media_image3.png
Greyscale
(Zhou, Table 2). RENA finds an optimal architecture with 95.81% accuracy for KWS architecture search, which is based on the trained policy network (second neural network).
Zhou relates to evolutionary child neural architecture searches and is analogous to the claimed invention
While Zhou fails to disclose the further limitations of the claim, Wen discloses instructions to train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“We tried many options for the architecture of the predictor (first neural network). Due to space constraints we will limit our discussion to Graph Convolutional Networks” (Wen, page 2, right column, paragraph 6)
“Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N random architectures (neural network architectures) to generate N (architecture, validation accuracy) pairs and use them to train (supervised training) a regression model (first neural network) that predicts accuracy (evaluation indicator values) based on the architecture. Next, we use this regression model (trained first neural network) to predict the validation accuracies of a large number of random architectures” (Wen, page 1, left column, Abstract)
Wen relates to supervised evaluation of candidate models in a neural architecture search and is analogous to the claimed invention. Zhou teaches a method of evaluating candidate neural network architectures with a score-generating neural network. The claimed invention differs from this method by training the first neural network using supervised learning. Wen teaches a method of training a score-generating neural network using supervised learning. Because both Zhou and Wen teach the use of evaluating candidate network architecture using an evaluator neural network, it would have been obvious to a person of ordinary skill in the art to substitute Zhou’s reinforcement learning-trained value network for Wen’s supervised learning-trained neural predictor to achieve the predictable result of learning to predict model quality based on observed quality measurements (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Wen fails to disclose the further limitations of the claim, Singh discloses instructions, wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: “The recurrent neural network may systematically search a space of deep network architectures by emulating the natural evolution of how the architectures may be designed and evaluated for a given task by researchers working in the field of deep learning. For example, the ImageNet dataset has millions of images from 1000 classes and is a benchmark for image classification tasks. The recurrent neural network may model researchers in the field of machine learning as ‘agents’ in a reinforcement learning framework training deep networks (training process) and conducting the architecture search (determining the first target neural network architecture), where multiple researchers may be training and searching in parallel. Over time, various network designs (plurality of first child models) are proposed and validated by the agents, and design elements from the highest performing architectures may be combined and incorporated to augment previous architectures or create a new architecture (first target neural network architecture) to perform well at the classification task.” (Singh, [0015]); “Multiple deep network architectures may allow for training and performance evaluations of the deep network architectures (plurality of first child models) in parallel” (Singh, [0057])
Singh relates to parallelized neural architecture searches and is analogous to the claimed invention. The existing combination teaches a method of performing a neural architecture search by evaluating and modifying a plurality of evolving child networks. The claimed invention improves upon this method by parallelizing candidate architecture search and training. Singh teaches a method of parallelizing candidate architecture search and training, applicable to The existing combination. A person of ordinary skill in the art would have recognized that parallelizing the search and candidate model training of The existing combination would lead to the predictable result of processing, modifying, and searching through candidate networks independently, and would improve the known device by decreasing computation time needed to perform candidate network operations (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 10, the rejection of claim 9 is incorporated. Zhou further discloses a system, wherein
the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network:
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
“In this section, embodiments of the overall reinforcement learning (RL) framework of RENA and the corresponding search space are explained. In one or more embodiments, the framework comprises a policy network to generate an action or actions that define the neural network architecture. In one or more embodiments, the environment outputs the performance of the trained neural network, as well as its resource use. In one or more embodiments, a policy gradient with accumulated rewards was used to train the policy network” (Zhou, [0056])
wherein the first neural network is configured to receive a representation of a neural network architecture as input and to output a predicted evaluation indicator value: “In one or more embodiments, a value network 140 (first neural network) takes in network embedding of the generated target network 145 (representation of a neural network architecture) and data distributions to approximate the reward by ascertain metrics (predicted evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined. In one or more embodiments, the value network may predict target network accuracy and training time without actually running the target network till convergence” (Zhou, [0046])
the second neural network is configured to provide neural network architectures according to the search space: “FIG. 3 depicts a policy network 300 (second neural network) with network embedding, in which a long short-term memory (LSTM)-based network transforms an existing neural network configuration into a trainable representation and the trainable representation is fed to a LSTM-based policy network to generate actions, according to embodiments of the present disclosure. An embodiment of a policy network 300, shown in FIG. 3, adapts an existing network configuration by modifying its parameters (which may be referred to as the scale action), or by inserting a new layer (which may be referred to as the insert action), or by removing an existing layer (which may be referred to as the remove action). Rather than building the target network from scratch, modifications via these operations allow more sample-efficient search with a simpler architecture. In one or more embodiments, a search can start with any baseline models, a well-designed or even a rudimentary one” (Zhou, [0058])
Regarding claim 11, the rejection of claim 9 is incorporated. Zhou further discloses a system, wherein the evaluation indicator values are based on performing inference on the first child models: “the trained adapted neural network architecture (trained child model) may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (evaluation indicator values)” (Zhou, [0048]).
Regarding claim 12, the rejection of claim 9 is incorporated. Zhou further discloses a system, wherein
each of the evaluation indicator values comprises a hardware-related performance value: “the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency (inference time), etc.) (evaluation indicator values)” (Zhou, [0048]). As noted in paragraph [0017] of the instant specification, model inference time is a form of hardware-related performance values.
the processor is further configured to execute the instructions to cause the at least one computer to:
obtain, performance values of a plurality of second child models on second hardware, wherein the performance values are based on performing inference on the second child models:
“In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network (child model) is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (performance values)” (Zhou, [0048])
“Keyword spotting (KWS) systems (second hardware) aim to detect a particular keyword from a continuous stream of audio. They are commonly used in conversational human-machine interfaces, such as in smart home systems or virtual assistants. A high detection accuracy and a low latency is critical to enable satisfactory user experience. In addition, KWS systems are typically deployed on a wide range of devices with different resource constraints” (Zhou, [0100])
“For KWS architecture search, layer-by-layer search was considered … An episode size of 5 and a batch size of 10 was used for all experiments, i.e. 10 child models (second child models) are trained concurrently … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network” (Zhou, [0103]). Computation of performance values for each child model are inherent in each model being evaluated in this system.
determine, based on the neural network architectures and the performance values based on performing inference on the second child models, a second target neural network architecture meeting a second preset condition:
“For KWS architecture search … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network. At the end of each episode, the policy was updated and the best 10 child models (neural network architectures) were used as the baseline for the new episode” (Zhou, [0103]). Computation of performance values are inherently used to evaluate child models and find the best set.
“the RENA embodiment learns to generate models (second target neural network architecture[s]) that meet both constraints after about 120 searched models” (Zhou, [0105])
“When aggressive resource constraints are imposed, it was observed that the RENA embodiment can find architectures that outperform hand-optimized architectures in the literature. A tight model size constraint results in an optimal architecture composed of GRUs with small hidden units. Similarly, tight constraints on computational complexity (second preset condition) also favor for GRUs with small hidden units” (Zhou, [0106])
Regarding claim 13, the rejection of claim 12 is incorporated. Zhou further discloses a system, wherein the hardware-related performance value comprises any one or more of model inference time, a quantity of activations, throughput, power consumption, or video random-access memory (RAM) usage: “In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency (inference time), etc.) (performance values)” (Zhou, [0048])
Regarding claim 14, the rejection of claim 9 is incorporated. Zhou further discloses a system, wherein
the search space comprises an attribute value space of each attribute of a neuron:
“In one or more embodiments, actions of scale and insert are mapped to a search space to define the neural network architectures. Two example approaches to defining search spaces are presented next” (Zhou, [0066]).
“a) Layer-by-Layer Search Embodiments” (Zhou, [0067]); “layer-by-layer search aims to find the optimal architecture with a search granularity of predefined layers … In one or more embodiments, the neural network architecture is defined (705) by stacking these layers, potentially with skip connections between them. For each feature, an LSTM in the policy network chooses (710) the layer type and the corresponding hyperparameters (e.g., filter width) (each attribute of a neuron). In one or more embodiments, the location of the inserted layer is denoted by an input source identifier (e.g., "Src1"), where the new layer gets its input data from. To support skip connection, the Insert Controller generates operation "add" that connects layer outputs of source identifiers (e.g., "Src1" and "Src2") with either an addition or a concatenation operation” (Zhou, [0068]).
the processor is further configured to execute the instructions to cause the at least one computer to randomly select an attribute value for each attribute from the attribute value space to obtain the neural network architectures:
“In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks, which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11) … in one or more embodiments, the starting network architecture for a branch may be: varied (e.g., randomly varied) from an initial architecture input 1110 (particularly, if this is the first episode)” (Zhou, [0076]).
“The Random Search generates random actions (i.e., insert, remove, and scale) and also selects the hyperparameters (each attribute) of each layer randomly with uniform probability” (Zhou, [0105]).
Regarding claim 17, Zhou teaches [a] computer program product comprising: at least one computer, comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium, and that, when executed by a processor, cause an apparatus to: “Aspects of the present disclosure may be encoded upon one or more non-transitory computer-readable media with instructions for one or more processors or processing units to cause steps to be performed” (Zhou, [0124]).
Zhou’s instructions to cause an apparatus to:
generate, based on a search space, a plurality of neural network architectures:
“In one or more embodiments, actions of scale and insert are mapped to a search space to define the neural network architectures” (Zhou, [0066]); “In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (plurality of neural network architectures), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
obtain, based on the neural network architectures, evaluation indicator values of a plurality of first child models on hardware
“Components may be implemented in software, hardware (hardware), or a combination thereof” (Zhou, [0023]).
“In one or more embodiments, a policy network converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration. In one or more embodiments, the adapted neural network (child model) is trained (220) to convergence, and the trained adapted neural network architecture may be evaluated (225) based upon one or more metrics (e.g., accuracy, memory footprint, power consumption, inference latency, etc.) (evaluation indicator values)” (Zhou, [0048])
“In one or more embodiments, the policy network generates (1205) a batch of actions a, m which produce a series of child networks (first child models), which may be considered in evolutionary branches (e.g., branch 1125 in FIG. 11)” (Zhou, [0076])
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“In one or more embodiments, a performance simulation network (first neural network) takes a target network embedding and a training dataset in terms of size, distribution, and regularity to generate approximated accuracy and training time (evaluation indicator values). Leveraging the embedding network, layer representation may be unified and the information from individual layers may be integrated. Given a set of sample networks, performance curves for each network may be obtained. For each network x, a validation accuracy a, and training time t, may be obtained, for example” (Zhou, [0087])
“If trained jointly, the performance simulation network becomes a value network V (trained first neural network)” (Zhou, [0088])
“The parameters
θ
v
[o]f the value network (first neural network) is updated via gradient descent using
PNG
media_image1.png
55
519
media_image1.png
Greyscale
”(Zhou, [0089]). The output of the network, V, which corresponds to the approximated accuracy and training time (evaluation indicator values) is used to train the network.
train, using the trained first neural network, a second neural network to obtain a trained second neural network: “The parameters
θ
of the policy network (second neural network) may be optimized via gradient descent as follows:
PNG
media_image2.png
146
544
media_image2.png
Greyscale
” (Zhou, [0088]). As can be seen in the formulas above, the value network V (first neural network) is used to train the policy network (second neural network).
determine, based on the trained second neural network, a first target neural network architecture meeting a first preset condition:
“In one or more embodiments, a policy network (second neural network) converts (210) that initial neural network architecture configuration representation into a network embedding. Then, in embodiments, the policy network uses (215) that network embedding to automatically generate adaptations to the neural network architecture configuration” (Zhou, [0048])
“a) Training Details” (Zhou, [0102]); “For KWS architecture search … The policy network (second neural network) was trained with the Adam optimization algorithm with a learning rate of 0.0006. An episode size of 5 and a batch size of 10 was used for all experiments, i.e. 10 child models (neural network[s]) are trained concurrently … Each model was evaluated after training and an action is selected according to the current policy in order to transform the network. At the end of each episode, the policy was updated and the best 10 child models were used as the baseline for the new episode.” (Zhou, [0103]).
“b) Results” (Zhou, [0104]); ”TABLE 2 presents the search results for KWS, as well as the optimal architectures. Without any resource constraints, the state-of-the-art accuracy, 95.81 %, can be obtained using an architecture comprising depth-separable convolutions (that apply significant downsampling), followed by gated recurrent units (GRUs) and multiple 2-D convolutions. When aggressive resource constraints are imposed, it was observed that the RENA embodiment can find architectures (first target neural network architecture[s]) that outperform hand-optimized architectures in the literature. A tight model size constraint (first preset condition) results in an optimal architecture composed of GRUs with small hidden units. Similarly, tight constraints (first preset condition[s]) on computational complexity also favor for GRUs with small hidden units. When compute intensity (first preset condition) is considered, an efficient architecture is achieved by enabling most of the computation on 2-D convolutions with large channel size.” (Zhou, [0106])
PNG
media_image3.png
440
1283
media_image3.png
Greyscale
(Zhou, Table 2). RENA finds an optimal architecture with 95.81% accuracy for KWS architecture search, which is based on the trained policy network (second neural network).
Zhou relates to evolutionary child neural architecture searches and is analogous to the claimed invention
While Zhou fails to disclose the further limitations of the claim, Wen discloses instructions to train, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network:
“We tried many options for the architecture of the predictor (first neural network). Due to space constraints we will limit our discussion to Graph Convolutional Networks” (Wen, page 2, right column, paragraph 6)
“Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N random architectures (neural network architectures) to generate N (architecture, validation accuracy) pairs and use them to train (supervised training) a regression model (first neural network) that predicts accuracy (evaluation indicator values) based on the architecture. Next, we use this regression model (trained first neural network) to predict the validation accuracies of a large number of random architectures” (Wen, page 1, left column, Abstract)
Wen relates to supervised evaluation of candidate models in a neural architecture search and is analogous to the claimed invention. Zhou teaches a method of evaluating candidate neural network architectures with a score-generating neural network. The claimed invention differs from this method by training the first neural network using supervised learning. Wen teaches a method of training a score-generating neural network using supervised learning. Because both Zhou and Wen teach the use of evaluating candidate network architecture using an evaluator neural network, it would have been obvious to a person of ordinary skill in the art to substitute Zhou’s reinforcement learning-trained value network for Wen’s supervised learning-trained neural predictor to achieve the predictable result of learning to predict model quality based on observed quality measurements (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Wen fails to disclose the further limitations of the claim, Singh discloses instructions, wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel: “The recurrent neural network may systematically search a space of deep network architectures by emulating the natural evolution of how the architectures may be designed and evaluated for a given task by researchers working in the field of deep learning. For example, the ImageNet dataset has millions of images from 1000 classes and is a benchmark for image classification tasks. The recurrent neural network may model researchers in the field of machine learning as ‘agents’ in a reinforcement learning framework training deep networks (training process) and conducting the architecture search (determining the first target neural network architecture), where multiple researchers may be training and searching in parallel. Over time, various network designs (plurality of first child models) are proposed and validated by the agents, and design elements from the highest performing architectures may be combined and incorporated to augment previous architectures or create a new architecture (first target neural network architecture) to perform well at the classification task.” (Singh, [0015]); “Multiple deep network architectures may allow for training and performance evaluations of the deep network architectures (plurality of first child models) in parallel” (Singh, [0057])
Singh relates to parallelized neural architecture searches and is analogous to the claimed invention. The existing combination teaches a method of performing a neural architecture search by evaluating and modifying a plurality of evolving child networks. The claimed invention improves upon this method by parallelizing candidate architecture search and training. Singh teaches a method of parallelizing candidate architecture search and training, applicable to The existing combination. A person of ordinary skill in the art would have recognized that parallelizing the search and candidate model training of The existing combination would lead to the predictable result of processing, modifying, and searching through candidate networks independently, and would improve the known device by decreasing computation time needed to perform candidate network operations (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 18, the rejection of claim 17 is incorporated. Zhou further discloses a system, wherein the evaluation indicator values are used as rewards in a reinforcement learning process of the second neural network:
“In one or more embodiments, a value network 140 takes in network embedding of the generated target network 145 and data distributions to approximate the reward by ascertain metrics (evaluation indicator values), such as network accuracy 150 and training time 155-although other metrics may also be determined” (Zhou, [0046])
“In this section, embodiments of the overall reinforcement learning (RL) framework of RENA and the corresponding search space are explained. In one or more embodiments, the framework comprises a policy network to generate an action or actions that define the neural network architecture. In one or more embodiments, the environment outputs the performance of the trained neural network, as well as its resource use. In one or more embodiments, a policy gradient with accumulated rewards was used to train the policy network” (Zhou, [0056])
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (RESOURCE-EFFICIENT NEURAL ARCHITECTS, filed 3/8/2019, US 2019/0354837 A1), hereafter referred to as Zhou, in view of Wen et al. (Neural Predictor for Neural Architecture Search, published 12/2/2019, arXiv:1912.00848v1), hereafter referred to as Wen, and further in view of Singh et al. (LEARNING TO SEARCH DEEP NETWORK ARCHITECTURES, filed 9/5/2018, US 2020/0074296 A1), hereafter referred to as Singh, and Tan et al. (COMPOUND MODEL SCALING FOR NEURAL NETWORKS, filed 1/23/2020, US 2020/0234132 A1), hereafter referred to as Tan.
Regarding claim 7, the rejection of claim 1 is incorporated. While Zhou and Singh fail to disclose the further limitations of the claim, Tan discloses a method, comprising:
providing, to a user, an application programming interface; and further generating, for the user through the application programming interface, the neural network architectures:
“This specification describes a system implemented as computer programs on one or more computers in one or more locations that determines, from a baseline architecture, a final architecture for a neural network” (Tan, [0005]).
“the system 100 can receive the baseline architecture 102 and the target resource usage data 104 as an upload from a remote user of the system over a data communication network, e.g., using an application programming interface (API) made available by the system 100. As another example, the system 100 can receive an input from a user specifying which data that is already maintained by the system 100 should be used as data identifying the baseline architecture 102 and as the target resource usage data 104” (Tan, [0023]).
“The system then selects, as the final values for the baseline depth, width and resolution coefficients, the search values that are associated with the maximum performance score among performance scores of all search candidate architectures that have been generated” (Tan, [0053]).
“After the depth, width, and resolution coefficients (d, w, r) are generated for the compound coefficient, the system 100 (generator) generates the final architecture 126” (Tan, [0039]).
Tan relates to neural architecture searches and are analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to use an API that allows data inputs and parameters for network architecture search to be transmitted by users, as disclosed by Tan. Such an API would allow the neural architecture search to be constrained based on individual user needs, and would allow resulting final architectures to be sent directly to the corresponding requesting users as data or graphical displays. See Tan, [0040], [0045], [0073], and [0077].
Regarding claim 15, the rejection of claim 9 is incorporated. While Zhou and Singh fail to disclose the further limitations of the claim, Tan discloses a system, wherein the processor is further configured to execute the instructions to cause the at least one computer to:
provide an application programming interface to a user; and further generate the neural network architectures for the user through the application programming interface:
“This specification describes a system implemented as computer programs on one or more computers in one or more locations that determines, from a baseline architecture, a final architecture for a neural network” (Tan, [0005]).
“the system 100 can receive the baseline architecture 102 and the target resource usage data 104 as an upload from a remote user of the system over a data communication network, e.g., using an application programming interface (API) made available by the system 100. As another example, the system 100 can receive an input from a user specifying which data that is already maintained by the system 100 should be used as data identifying the baseline architecture 102 and as the target resource usage data 104” (Tan, [0023]).
“The system then selects, as the final values for the baseline depth, width and resolution coefficients, the search values that are associated with the maximum performance score among performance scores of all search candidate architectures that have been generated” (Tan, [0053]).
Tan relates to neural architecture searches and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to use an API that allows data inputs and parameters for network architecture search to be transmitted by users, as disclosed by Tan. Such an API would allow the neural architecture search to be constrained based on individual user needs, and would allow resulting final architectures to be sent directly to the corresponding requesting users as data or graphical displays. See Tan, [0040], [0045], [0073], and [0077].
Claims 8 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (RESOURCE-EFFICIENT NEURAL ARCHITECTS, filed 3/8/2019, US 2019/0354837 A1), hereafter referred to as Zhou, in view of Wen et al. (Neural Predictor for Neural Architecture Search, published 12/2/2019, arXiv:1912.00848v1), hereafter referred to as Wen, and further in view of Singh et al. (LEARNING TO SEARCH DEEP NETWORK ARCHITECTURES, filed 9/5/2018, US 2020/0074296 A1), hereafter referred to as Singh, and Rieke (What Is Federated Learning?, published 10/13/2019, Nvidia Blogs, retrieved from https://blogs.nvidia.com/blog/what-is-federated-learning/).
Regarding claim 8, the rejection of claim 1 is incorporated. Rieke, in combination with Zhou, discloses a method, comprising performing, federated learning on each of N initial child models of the plurality of first child models using M datasets to obtain N child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1:
Rieke: “Frameworks such as NVIDIA FLARE (NVFlare) (model training platform) have enabled enterprises to collaborate by contributing data through federated learning for model improvements” (Rieke, page 2, paragraph 7).
Rieke: “The main concept of federated learning is to train models locally without sharing data, only the model parameters. The aggregator starts with an initial global model and broadcasts the model parameters to all clients. The client node receives the global model parameters and starts training the received model (initial child model) on local data (dataset). Then, the newly trained local model (child model) is sent back to the aggregator node. Only model parameters, no private data, are shared with the aggregator” (Rieke, page 3, paragraphs 1-2).
PNG
media_image4.png
1118
2000
media_image4.png
Greyscale
(Rieke, page 5). An illustration of a case with three local models (N == 3), each with a private dataset (M == 3).
Examiner’s note: If the neural network architectures disclosed by Zhou are each used as the basis for a local model trained with a unique dataset in federated learning, as per Rieke’s disclosed method, then there’s a one-to-one correspondence between the initial child models with the neural network architectures.
Rieke relates to training a plurality of neural network models and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to train the neural network architectures with federated learning, as disclosed by Rieke. Federated learning allows data from a wider assortment of sources to be used, without compromising data security. See Rieke, page 2, paragraphs 1-5.
Regarding claim 20, the rejection of claim 1 is incorporated. Rieke, in combination with Zhou, discloses a method, dividing N initial child models into M groups of initial child models of the plurality of first child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein the M groups of initial child models are in a one-to-one correspondence with M datasets, wherein N is greater than 1, and wherein M is greater than 1; and
training the M groups of initial child models using corresponding datasets to obtain M groups of child models:
Rieke: “Frameworks such as NVIDIA FLARE (NVFlare) (model training platform) have enabled enterprises to collaborate by contributing data through federated learning for model improvements” (Rieke, page 2, paragraph 7).
Rieke: “The main concept of federated learning is to train models locally without sharing data, only the model parameters. The aggregator starts with an initial global model and broadcasts the model parameters to all clients. The client node receives the global model parameters and starts training the received model (initial child model) on local data (dataset). Then, the newly trained local model (child model) is sent back to the aggregator node. Only model parameters, no private data, are shared with the aggregator” (Rieke, page 3, paragraphs 1-2).
PNG
media_image4.png
1118
2000
media_image4.png
Greyscale
(Rieke, page 5). An illustration of a case with three local models (N == 3), each in a private training group (M == 3) with a corresponding set of private data.
Examiner’s note: If the neural network architectures disclosed by Zhou are each used as the basis for a local model trained with a unique dataset in federated learning, as per Rieke’s disclosed method, then there’s a one-to-one correspondence between the initial child models with the neural network architectures.
Rieke relates to training a plurality of neural network models and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to train the neural network architectures with federated learning, as disclosed by Rieke. Federated learning allows data from a wider assortment of sources to be used, without compromising data security. See Rieke, page 2, paragraphs 1-5.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (RESOURCE-EFFICIENT NEURAL ARCHITECTS, filed 3/8/2019, US 2019/0354837 A1), hereafter referred to as Zhou, in view of Wen et al. (Neural Predictor for Neural Architecture Search, published 12/2/2019, arXiv:1912.00848v1), hereafter referred to as Wen, and further in view of Singh et al. (LEARNING TO SEARCH DEEP NETWORK ARCHITECTURES, filed 9/5/2018, US 2020/0074296 A1), hereafter referred to as Singh, and Plumbley et al. (ENSEMBLE MODEL CREATION AND SELECTION, filed 3/29/2019, US 20210117869 A1), hereafter referred to as Plumbley.
Regarding claim 19, the rejection of claim 1 is incorporated. Plumbley, in combination with Zhou, discloses a method, comprising training each of N initial child models of the plurality of first child models using M datasets to obtain N*M child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1:
“The inventors have advantageously developed a system (model training platform) for generating and selecting from a large number of trained models, or a plurality of sets of trained models” (Plumbley, [0077])
“Each of the plurality of models (N > 1 initial child models) are trained on each of the plurality of datasets (M > 1 datasets) forming a plurality of trained models (N*M child models)” (Plumbley, [0089])
Examiner’s note: If the neural network architectures disclosed by Zhou are each used as the basis for the plurality of models trained with a plurality of datasets, as per Rieke’s disclosed method, then there’s a one-to-one correspondence between the initial child models with the neural network architectures.
Plumbley relates to training a plurality of neural networks and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to train each architecture model with a plurality of datasets to produce multiple trained models, as disclosed by Plumbley. From such a set of models, optimal model / dataset combinations can be identified automatically, reducing error from model and dataset mismatch. See Plumbley, [0005 – 0008], [0012].
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (RESOURCE-EFFICIENT NEURAL ARCHITECTS, filed 3/8/2019, US 2019/0354837 A1), hereafter referred to as Zhou, in view of Wen et al. (Neural Predictor for Neural Architecture Search, published 12/2/2019, arXiv:1912.00848v1), hereafter referred to as Wen, and further in view of Singh et al. (LEARNING TO SEARCH DEEP NETWORK ARCHITECTURES, filed 9/5/2018, US 2020/0074296 A1), hereafter referred to as Singh, Rieke (What Is Federated Learning?, published 10/13/2019, Nvidia Blogs, retrieved from https://blogs.nvidia.com/blog/what-is-federated-learning/), and Plumbley et al. (ENSEMBLE MODEL CREATION AND SELECTION, filed 3/29/2019, US 20210117869 A1), hereafter referred to as Plumbley.
Regarding claim 16, the rejection of claim 9 is incorporated. Rieke, in combination with Zhou, discloses a system wherein the processor is further configured to execute the instructions to cause the at least one computer to:
perform federated learning on each of N initial child models of the plurality of first child models using M datasets to obtain N child models, wherein the N initial child models are based on the neural network architectures and are in a one-to-one correspondence with the neural network architectures, wherein N is greater than 1, and wherein M is greater than 1, … or divide the N initial child models into M groups of initial child models, wherein the M groups of initial child models are in a one-to-one correspondence with the M datasets, and training the M groups of initial child models using corresponding datasets to obtain M groups of child models:
Rieke: “The main concept of federated learning is to train models locally without sharing data, only the model parameters. The aggregator starts with an initial global model and broadcasts the model parameters to all clients. The client node receives the global model parameters and starts training the received model (initial child model) on local data (dataset). Then, the newly trained local model (child model) is sent back to the aggregator node. Only model parameters, no private data, are shared with the aggregator” (Rieke, page 3, paragraphs 1-2).
PNG
media_image4.png
1118
2000
media_image4.png
Greyscale
(Rieke, page 5). An illustration of a case with three local models (N == 3), each in a private training group (M == 3) with a corresponding private dataset.
Examiner’s note: If the neural network architectures disclosed by Zhou are each used as the basis for a local model trained with a unique dataset in federated learning, as per Rieke’s disclosed method, then there’s a one-to-one correspondence between the initial child models with the neural network architectures.
Rieke relates to training a plurality of neural network models and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to train the neural network architectures with federated learning, as disclosed by Rieke. Federated learning allows data from a wider assortment of sources to be used, without compromising data security. See Rieke, page 2, paragraphs 1-5.
While Rieke fails to disclose the further limitations of the claim, Plumbley, in combination with Zhou, discloses a system able to train each of the N initial child models using the M datasets to obtain N*M child models:
“The inventors have advantageously developed a system (model training platform) for generating and selecting from a large number of trained models, or a plurality of sets of trained models” (Plumbley, [0077])
“Each of the plurality of models (N > 1 initial child models) are trained on each of the plurality of datasets (M > 1 datasets) forming a plurality of trained models (N*M child models)” (Plumbley, [0089])
Examiner’s note: If the neural network architectures disclosed by Zhou are each used as the basis for the plurality of models trained with a plurality of datasets, as per Rieke’s disclosed method, then there’s a one-to-one correspondence between the initial child models with the neural network architectures.
Plumbley relates to training a plurality of neural networks and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to train each architecture model with a plurality of datasets to produce multiple trained models, as disclosed by Plumbley. From such a set of models, optimal model / dataset combinations can be identified automatically, reducing error from model and dataset mismatch. See Plumbley, [0005 – 0008], [0012].
Response to Arguments
The following responses address arguments and remarks made in the instant remarks dated 04/27/2026.
112 Rejections
In light of the instant amendments, previous rejections under 35 U.S.C. 112(b) have been withdrawn.
101 Rejections
On pages 10-11 of the instant remarks, the Applicant argues that the claimed invention improves on existing technology:
“Critically, the claim further requires that determining the first target
neural network architecture is decoupled from a training process of the plurality of first child models
such that the determining step and the training process are processed in parallel. This decoupling and
parallel processing requirement is a concrete architectural limitation that changes how computational tasks are performed within the system. Rather than waiting for completion of child model training
before performing architecture selection, the claimed method restructures the workflow to eliminate
sequential dependency and enable concurrent execution. The first neural network is trained using
actual evaluation indicator values and then used to train the second neural network, thereby removing
reward delay and reducing hardware-dependent latency. These limitations collectively recite a
specific machine-learning-based system configuration that improves computational efficiency,
reduces search duration, and enhances processor utilization. The claim therefore recites a
technological improvement to computer functionality, not an abstract mental process or generic
automation, and is integrated into a practical application. Accordingly, Applicant respectfully
requests withdrawal of the § 101 rejections.”
Regarding the Applicant’s arguments above, the Examiner respectfully disagrees. Using generic computer technology to accelerate a process does not constitute practical integration through an improvement to technology, as noted in MPEP 2106.05(f): Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, "claiming the improved speed or efficiency inherent with applying the abstract idea on a computer" does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015)
The claimed invention recites the usage of parallelism at a high level of generality. The improvements argued by the Applicant, removing processing delays and increasing processing speed, are improvements inherent to the application of generic parallel processing systems, and are not reflective of a specific improvement indicative of the inventive concept of the claimed invention. Thus, this is insufficient to integrate the recited judicial exceptions into a practical application.
See the 101 rejections section for more detail. No rejections are withdrawn on this basis.
103 Rejections
On pages 13-14 of the instant remarks, the Applicant argues that Zhou and Singh fail to disclose training the first and second neural networks, as claimed:
“The existing combination fails to render obvious claims 1-20 because the
combination of Zhou and Singh fails to 1) train, based on the neural network architectures and the
evaluation indicator values, a first neural network using supervised learning to obtain a trained first
neural network, and train, using the trained first neural network, a second neural network to obtain a
trained second neural network
…
First, claim 1 requires training, based on the neural network architectures and the
evaluation indicator values, a first neural network using supervised learning to obtain a trained first
neural network, and training, using the trained first neural network, a second neural network to obtain
a trained second neural network. Claims 9 and 17 contain similar limitations. Support for the
amendments can be found in paragraphs 6-9, 73, 123, and 129 of the Applicant's Application. To
reject these limitations, the Examiner asserts that paragraphs 46-47 and 89 of Zhou disclose similar
limitations. See Office Action, pp. 19-20. However, Zhou does not use supervised learning to train a
neural network based on evaluation indicator values
…
Zhou, ¶¶ 46-47 and 89 (emphasis added). As shown above, Zhou's value network and policy
network are "trained jointly" via reinforcement learning, with both networks updated
simultaneously based on rewards from completed child models. However, Zhou does not use
supervised learning to train a first neural network based on evaluation indicator values and uses
the trained first neural network to train a second neural network. Thus, Zhou's joint
reinforcement learning framework is architecturally distinct from the present claims two-stage
supervised learning approach. As such, Zhou fails to train, based on the neural network
architectures and the evaluation indicator values, a first neural network using supervised learning
to obtain a trained first neural network, and train, using the trained first neural network, a second
neural network to obtain a trained second neural network, as claimed”
Regarding the assertion that Zhou and Singh fail to render obvious the claimed supervised training of the first neural network, the Examiner agrees. However, upon further search and consideration, “training, based on the neural network architectures and the evaluation indicator values, a first neural network using supervised learning to obtain a trained first neural network” is found to be obvious over Zhou in view of Wen. Zhou discloses training a value network based on its output evaluation values, produced in response to candidate network inputs (Zhou, [0087-0089]). While Zhou fails to disclose using supervised learning in this process, Wen discloses using supervised learning to train a network that outputs evaluation values in response to candidate network inputs (Wen, page 1, left column, Abstract & page 2, right column, paragraph 6). It would have been obvious for one of ordinary skill in the art to substitute Wen’s supervised regression network for Zhou’s reinforcement-learned value network with predictable effects (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding the assertion that Zhou fails to render obvious “training, using the trained first neural network, a second neural network to obtain a trained second neural network”, the Examiner disagrees. Zhou discloses training a policy network (second neural network) using the outputs of the value network (first neural network).
See the 103 rejections section for more detail. No rejections are withdrawn on these grounds.
On pages 12-15 of the instant remarks, the Applicant argues that Zhou and Singh fail to disclose the claimed parallel processing:
“The existing combination fails to render obvious claims 1-20 because the
combination of Zhou and Singh fails to … 2) determine the first target neural network architecture is
decoupled from a training process of the plurality of first child models such that determining the first
target neural network architecture and the training process are processed in parallel
…
Second, claim 1 requires determining the first target neural network architecture is decoupled
from a training process of the plurality of first child models such that determining the first target
neural network architecture and the training process are processed in parallel. Claims 9 and 17
contain similar limitations. Support for the amendments can be found in paragraphs 6-9 and 73 of
the Applicant's Application. To reject these limitations, the Examiner acknowledges that Zhou fails
to disclose these limitations, but asserts that paragraph 15 of Singh discloses similar limitations. See
Office Action, pp. 28-29. However, Singh does not decouple concurrent execution of determining
the first target neural network architecture and a training process of the plurality of first child models
…
Singh, ¶ 15 ( emphasis added). As shown above, Singh teaches general parallelization m a
distributed RL framework. However, Singh does not decouple determining the first target neural
network architecture from a training process of the plurality of first child models such that
determining the first target neural network architecture and the training process are processed in
parallel. As such, Singh fails to determine the first target neural network architecture is decoupled
from a training process of the plurality of first child models such that determining the first target
neural network architecture and the training process are processed in parallel, as claimed. Thus,
the existing combination fails to disclose at least one limitation of independent claims 1,
9, and 17, and consequently fails to render obvious claims 1-20.”
Regarding the Applicant’s arguments above, the Examiner respectfully disagrees. Regarding “wherein determining the first target neural network architecture is decoupled from a training process of the plurality of first child models such that determining the first target neural network architecture and the training process are processed in parallel", Singh discloses a decentralized neural architecture search, where multiple independent agents train and search through a space of candidate models in parallel (Singh, [0015]). This decouples the processing (training and evaluation) of each candidate relative to the other candidates in the search (Singh, [0057]), allowing one candidate to be evaluated while the search continues on other candidates in parallel. The parallel processing disclosed by Singh is commensurate in scope with claim language.
No rejections are withdrawn on this basis. See the 103 rejections section for more detail.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Wistuba et al. (A Survey on Neural Architecture Search, published 6/18/2019, arXiv:1905.01392v2) provides a comprehensive overview of neural architecture search fundamentals and known techniques.
McDonnell et al. (Automated neural network generation using fitness estimation, filed 7/30/2019, US 10685286 B1) teaches a method of using a model quality estimator based on an RNN to score prospective architectures in evolutionary NAS.
Zoph et al. (NEURAL ARCHITECTURE SEARCH WITH REINFORCEMENT LEARNING, published 2/15/2017, arXiv:1611.01578v2) discloses the use a controller searching through network architectures, trained with reinforcement learning and utilizing attention mechanisms
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaron P Gormley whose telephone number is (571)272-1372. The examiner can normally be reached Monday - Friday 12:00 PM - 8:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AG/Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights