Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the following communication: Non-Provisional Application filed Oct. 6, 2023.
Claims 1-20 are pending in the case. Claims 1 10, and 19 are independent claims.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “(a) performing one or more evolutionary operations on an initial population of neural architectures to generate offspring neural architectures” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(b) evaluating performance of each of the offspring neural architectures to obtain at least one evaluation value of the offspring neural architecture with respect to a performance metric” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion).
Yes, the limitation “(c) adjusting the evaluation values of the offspring neural architectures based on at least one constraint on the evaluation values;” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(d) selecting at least one of the offspring neural architectures as a new population of neural architectures using a selection strategy” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(e) outputting the new population of neural architectures as a last population of neural architectures when a stopping criterion is achieved” See MPEP § 2106.04(a)(2)(III) is mere insignificant extra solution activity and something the courts have recognized as being well-understood, routine and conventional.
Yes, the limitation “(f) iterating steps (a) to (d) with the new population of neural architectures being taken as the initial population of neural architectures when the stopping criterion is not achieved yet” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “[a]n evolutionary neural architecture search (ENAS) method” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation ““[a]n evolutionary neural architecture search (ENAS) method” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claims 2-9 are dependent on claim 1 and includes all the limitations of claim 1. Therefore, claims 2-9 recite the same abstract idea. The claims recite additional limitations do not add any meaningful limits beyond the abstract idea.
As to claim 10:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “(a) performing one or more evolutionary operations on an initial population of neural architectures to generate offspring neural architectures” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(b) evaluating performance of each of the offspring neural architectures to obtain at least one evaluation value of the offspring neural architecture with respect to a performance metric” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion).
Yes, the limitation “(c) adjusting the evaluation values of the offspring neural architectures based on at least one constraint on the evaluation values;” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(d) selecting at least one of the offspring neural architectures as a new population of neural architectures using a selection strategy” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(e) outputting the new population of neural architectures as a last population of neural architectures when a stopping criterion is achieved” See MPEP § 2106.04(a)(2)(III) is mere insignificant extra solution activity and something the courts have recognized as being well-understood, routine and conventional.
Yes, the limitation “(f) iterating steps (a) to (d) with the new population of neural architectures being taken as the initial population of neural architectures when the stopping criterion is not achieved yet” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “[a]n apparatus, comprising circuitry configured to perform an evolutionary neural architecture search (ENAS) method” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “[a]n apparatus, comprising circuitry configured to perform an evolutionary neural architecture search (ENAS) method” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claims 11-18 are dependent on claim 1 and includes all the limitations of claim 10. Therefore, claims 11-18 recite the same abstract idea. The claims recite additional limitations do not add any meaningful limits beyond the abstract idea.
As to claim 19:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “(a) performing one or more evolutionary operations on an initial population of neural architectures to generate offspring neural architectures” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(b) evaluating performance of each of the offspring neural architectures to obtain at least one evaluation value of the offspring neural architecture with respect to a performance metric” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion).
Yes, the limitation “(c) adjusting the evaluation values of the offspring neural architectures based on at least one constraint on the evaluation values;” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(d) selecting at least one of the offspring neural architectures as a new population of neural architectures using a selection strategy” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “(e) outputting the new population of neural architectures as a last population of neural architectures when a stopping criterion is achieved” See MPEP § 2106.04(a)(2)(III) is mere insignificant extra solution activity and something the courts have recognized as being well-understood, routine and conventional.
Yes, the limitation “(f) iterating steps (a) to (d) with the new population of neural architectures being taken as the initial population of neural architectures when the stopping criterion is not achieved yet” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “[a] non-transitory machine-readable storage medium, storing instructions which, when executed by a processor, causes the processor to execute” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “[a] non-transitory machine-readable storage medium, storing instructions which, when executed by a processor, causes the processor to execute” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
Claim 20 dependent on claim 19 and includes all the limitations of claim 19. Therefore, claim 20 recites the same abstract idea. The claims recite additional limitations do not add any meaningful limits beyond the abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Risto et al. 2017 “Evolving Deep Neural Networks” (hereinafter Risto) in view of Tan et al. “MnasNet: Platform-Aware Neural Architecture Search for Mobile ” (hereinafter Tan) 2019.
With respect to independent clam 1, Risto teaches an evolutionary neural architecture search (ENAS) method (see Abstract - “This paper proposes an automated method, CoDeepNEAT, for optimizing deep learning architectures through evolution. By extending existing neuroevolution methods to topology, components, and hyperparameters”), comprising the following steps of:
(a) performing one or more evolutionary operations on an initial population of neural architectures to generate offspring neural architectures (see e.g. Section 3.1-3.2 – “First, a population of chromo-somes (each represented by a graph) with minimal complexity is created. Over generations, structure (i.e. nodes and edges) is added to the graph incrementally through mutation. During crossover, historical markings are used to determine how genes of two chro-mosomes can be lined up.” “In CoDeepNEAT, two populations of modules and blueprints are evolved separately”);
(b) evaluating performance of each of the offspring neural architectures to obtain at least one evaluation value of the offspring neural architecture with respect to a performance metric (see e.g. Section 3.1– “During fitness evaluation, each chromosome is converted into a DNN. These DNNs are then trained for a fixed number of epochs. After training, a metric that indicates the network’s performance is returned back to DeepNEAT and assigned as fitness to the corresponding chromosome in the population.”);
(d) selecting at least one of the offspring neural architectures as a new population of neural architectures using a selection strategy (see e.g. Section 3.1 – “The population is divided into species (i.e. subpopulations) based on a similarity metric. Each species grows proportionally to its fitness and evolution occurs separately in each species.”);
(e) outputting the new population of neural architectures as a last population of neural architectures when a stopping criterion is achieved (see e.g. Section 3.3 – “After 72 generations of evolution, the best network in the population was returned.”); and
(f) iterating steps (a) to (d) with the new population of neural architectures being taken as the initial population of neural architectures when the stopping criterion is not achieved yet (see e.g. Section 3.1 – “Over generations, structure (i.e. nodes and edges) is added to the graph incrementally through mutation. During crossover, historical markings are used to determine how genes of two chro-mosomes can be lined up.” “During fitness evaluation, each chromosome is converted into a DNN. These DNNs are then trained for a fixed number of epochs.”).
Risto does not expressly show (c) adjusting the evaluation values of the offspring neural architectures based on at least one constraint on the evaluation values. However, Tan teaches similar feature. Tan teaches that evaluation/objective is modified by constraint, such as latency (“we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. “ – page 2820) Tan discloses the performance metric and the constraint variable are used to adjust the evaluation (“Given a model m, let ACC(m) denote its accuracy on the target task, LAT (m) denotes the inference latency on the target mobile platform, and T is the target latency … we simply use accuracy as the objective value if measured latency is less than the target latency T ; otherwise, we sharply penalize the objective value to discourage mod-els from violating latency constraints. The bottom figure (α = β = −0.07) treats the target latency T as a soft con-straint, and smoothly adjusts the objective value based on the measured latency.” See page 2822).
Both Risto and Tan are directed to evolutionary architecture search methods. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Risto and Tan in front of them to modify the system of Risto to include the above feature. The motivation to combine Risto and Tan comes from Tan. Tan discloses the motivation to adjust the evaluation so that evaluation can be more accurate (see e.g. page 2820-2822). This motivation for combination also applies to the remaining claims which depend on this combination.
With respect to dependent clam 2, the modified Risto teaches adjusting the evaluation values of the neural architectures includes at least one of performing a clip algorithm to apply an enough-bound on the evaluation values of the offspring neural architectures and performing an extinct algorithm to apply a must-bound on the evaluation values of the offspring neural architectures (see e.g., Tan page 2820 - “an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency.” Page 2822 – “Figure 3 shows the objective function with two typical values of (_, _). In the top figure with (_ = 0, _ = −1),we simply use accuracy as the objective value if measured latency is less than the target latency T; otherwise, we sharply penalize the objective value to discourage models from violating latency constraints.” Tan does not expressly show “clip algorithm” and “extinct algorithm,” however these algorithms are well-known in the art to enforce constraints in evolutionary optimzation).
With respect to dependent clam 3, the modified Risto teaches at least one of the enough-bound and the must-bound is constant for every iteration of steps (a) to (d) (see e.g., Tan Page 2822 - “Given a model m, let ACC(m) denote its accuracy on the target task, LAT(m) denotes the inference latency on the target mobile platform, and T is the target latency.” The fixed target constraint corresponds to the “must-bound”).
With respect to dependent clam 4, the modified Risto teaches at least one of the enough-bound and the must-bound varies for at least two consecutive iterations of steps (a) to (d) (see e.g., Section 2 and Tan Page 2822 – based on the teaching of Risto and Tan, it would have been obvious to include this feature because the evolutionary search is iterative and objective is adjusted based on latency).
With respect to dependent clam 5, the modified Risto teaches the at least one of the enough-bound and the must-bound increases gradually as steps (a) to (d) iterate (see e.g., Section 2 and Tan Page 2822, 2823).
With respect to dependent clam 6, the modified Risto teaches the must-bound increases gradually as steps (a) to (d) iterate (see e.g., Section 2 and Tan Page 2822, 2823).
With respect to dependent clam 7, the modified Risto teaches the enough-bound gradually increases as steps (a) to (d) iterate (see e.g., Section 2 and Tan Page 2822, 2823 – the examiner notes that it would have been obvious to include this feature because the disclosed adjustment has no limitation).
With respect to dependent clam 8, the modified Risto teaches the evolutionary operations include at least one of crossover and mutation (see e.g., Section 3.1, 4 – “DeepNEAT differs from NEAT in that each node in the chromo-some no longer represents a neuron, but a layer in a DNN. Each node contains a table of real and binary valued hyperparameters that are mutated through uniform Gaussian distribution and ran-dom bit-flipping, respectively. These hyperparameters determine the type of layer (such as convolutional, fully connected, or recur-rent) and the properties of that layer (such as number of neurons, kernel size, and activation function).”).
With respect to dependent clam 9, the modified Risto teaches the selection strategy includes one of non-dominated soring, elitism, discarding the worst, roulette wheel selection and tournament selection (The examiner notes that elitism and tournament selection is well-known in the art).
Claim 10 is rejected for the similar reasons discussed above with respect to claim 1.
Claim 11 is rejected for the similar reasons discussed above with respect to claim 2.
Claim 12 is rejected for the similar reasons discussed above with respect to claim 3.
Claim 13 is rejected for the similar reasons discussed above with respect to claim 4.
Claim 14 is rejected for the similar reasons discussed above with respect to claim 5.
Claim 15 is rejected for the similar reasons discussed above with respect to claim 6.
Claim 16 is rejected for the similar reasons discussed above with respect to claim 7.
Claim 17 is rejected for the similar reasons discussed above with respect to claim 8.
Claim 18 is rejected for the similar reasons discussed above with respect to claim 9.
Claim 19 is rejected for the similar reasons discussed above with respect to claim 1.
Claim 20 is rejected for the similar reasons discussed above with respect to claim 2.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert. denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEIYONG WENG whose telephone number is (571)270-1660. The examiner can normally be reached on Mon.-Fri. 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Ell, can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/PEI YONG WENG/Primary Examiner, Art Unit 2141
Prosecution Insights