Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. The request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for Continued Examination under 37 CFR 1.114, the fee set forth in 37 CFR 1.17(e) has been paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed 12/19/2025 has been entered. An action on the RCE follows.
Summary of claims
3. Claims 1-20 are pending,
Claims 1 and 11 are amended,
Claims 1, 11 are independent claims,
Claims 1-20 are rejected.
Remarks
4. Applicant’s arguments, see Remarks, filed on 12/19/2025, with respect to the rejection(s) of claim(s) 1-20 under 103 have been fully considered and are not persuasive in view of new rejection ground(s).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
5. Claims 1-5, 7, 10-15, 17, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mohammad Salameh et al (US Publication 20230096654 A1, hereinafter Salameh), and in view of Vijay Vasudevan et al (US Publication 20190286984 A1, hereinafter Vasudevan), and Anthony Piergiovanni et al (US Publication 20220366257 A1, hereinafter Piergiovanni).
As for independent claim 1, Salameh discloses: A method for performing a neural architecture search (Salameh: Abstract, A method and system for generating neural architectures to perform a particular task. An actor neural network, as part of a continuous action reinforcement learning (RL) agent, generates a randomized continuous actions parameters to encourage exploration of a search space to generate candidate architectures without bias), comprising: sampling a discrete network search space a first time (Salameh: [0013], (i) generating, by an actor neural network having actor parameters in accordance with current values of the actor parameters, a set of continuous neural network architecture parameters comprising score distributions over possible values for configuring a plurality of architecture cells of a trained search space; (ii) discretizing the set of continuous architecture parameters into a set of discrete neural network architecture parameters), determining a differential architecture network sampled from a super-network by applying a continuous relaxation of the discrete network search space over respective operators in the super-network (Salameh: [0060], NAS algorithms aim to learn a set of architectural parameters that parameterize a candidate neural network. The architectural parameters are connected to different operations at different locations within the candidate neural network. Architecture cells form the building blocks of the supernet. Once trained, the architecture cells may be connected, as specified by architectural parameters, to form neural networks; [0078], during each training epoch, the supernet 312 processes each sample of data in training data 302 in batches. Before a batch is processed by the supernet, at step 502, a continuous representation of a candidate architecture is randomly sampled. In some embodiments, the random sampling policy is blind to the performance of different candidate architectures may avoid any bias towards any specific type of candidate architectures; [0090]-[0091], in some embodiments, the continuous action a.sub.t may be defined as per Equation (3):
α.sub.t=μ(s.sub.t)+Z.sub.t Equation (3)
Where Z.sub.t is a small randomized noise following a uniform distribution Uniform(+ξ, ξ) added to the output of actor neural network 322 to encourage search space exploration by introducing a degree of randomness to the generation of action a.sub.t which may lead to a different discretized version that allows a new candidate architecture to be sampled) …, updating a distribution of the discrete network search space based on the reward (Salameh: Abstract, A critic neural network, as part of the continuous action RL agent, learns a mapping of the continuous action to a reward using modified Deep Deterministic Policy Gradient (DDPG) with quantile loss function by sampling a list of top performing architectures), and determining an updated differential architecture network based on the reward (Salameh: Abstract, The actor neural network is updated with the learned mapping).
Salameh does not clearly disclose a proxy accuracy or proxy complexity of the differential architecture network, in an analogous art of neural architecture search, Vasudevan discloses: based on a proxy accuracy and a proxy complexity of the differential architecture network (Vasudevan: [0029], the performance metric can represent a level of accuracy of candidate architecture on the proxy validation set. The system 100 then determines, based on the proxy performance metrics of the candidate architectures, a respective score label for each candidate architecture in the surviving set of candidate architectures 110);
Salameh and Vasudevan are analogous arts because they are in the same field of endeavor, neural architecture search. Therefore, it would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Salameh using the teachings of Vasudevan to include calculating a score based on the proxy performance metrics. It would provide Salameh’s method with enhanced capabilities of improving the efficiency and accuracy of neural architecture search.
Further, Salameh did not clearly disclose using learnable weights for the operators, in another analogous art of neural architecture search, Piergiovanni discloses: wherein the continuous relaxation defines learnable weights for the respective operators…using the learnable weights for the respective operators (Piergiovanni: [0054], in order to learn novel efficient video architecture, the search process can maximize the following equation, where the input is the set of variables/parameters defining a neural network architecture. N is the network configuration, which is defined by particular values chosen for that network for the searchable parameters of the search space 12. θ is the learnable parameters of the network (|θ| is the number of parameters in the network), and P is a hyperparameter controlling the maximum size of the network. custom-character(N.sub.θ) computes the runtime of the network on a device, given the network N with its weight values θ, and R is the maximum desired computational runtime);
Salameh and Piergiovanni are analogous arts because they are in the same field of endeavor, neural architecture search. Therefore, it would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Salameh using the teachings of Piergiovanni to include using learnable weight values. It would provide Salameh’s method with enhanced capabilities of improving the efficiency and accuracy of neural architecture search.
As for claim 2, Salameh-Vasudevan-Piergiovanni discloses: sampling the discrete network search space a second time based on the reward to improve a sampling accuracy of the discrete network search space (Salameh: [0115], Advantageously, the critic network 324 may act as a performance predictor, producing an approximation of the reward which can be mapped back into an accuracy estimation).
As for claim 3, Salameh-Vasudevan-Piergiovanni discloses: wherein sampling the discrete network search space the first or second time includes determining discrete components comprising at least one of a layer (Salameh: [0070], The architecture data 306 may define one or more parameters of the child neural network, including the number of layers, operations performed by each of the layers, connectivity between the layers of the neural network (i.e. which layer/cell receives inputs from which other layer/cell of the neural network), a number of channels (Salameh: [0035], In some or all examples, each experience tuple is comprised of the state, action, and reward (s.sub.t, a.sub.t, r.sub.t) for each step t, wherein the state s.sub.t defines a set of channel-wise average of discrete neural network architecture parameters, the action a.sub.t is a set of continuous neural architecture parameters, and the reward r.sub.t defines the reward), and an input or output feature map (Salameh: [0060], where each node 102 is a latent representation (e.g. a feature map in convolution networks) and each directed edge (i,j) 104 is associated with some operation o.sup.(i,j) that transforms 102).
As for claim 4, Salameh-Vasudevan-Piergiovanni discloses: performing a differentiable neural architecture search (DARTS) based on the determined discrete components (Salameh: [0079], the sampled continuous representation α, comprising of continuous architecture parameter values for configuring the architecture cells 100 of the supernet 312, is mapped to a discrete representation α.sub.t.sup.d. FIG. 6 illustrates an example pseudo-code representation 600 of an algorithm that can be implemented at step 504 for discretization of the continuous sampled architecture representation for DARTS or DARTS-based search spaces).
As for claim 5, Salameh-Vasudevan-Piergiovanni discloses: simultaneously performing multiple differentiable neural architecture searches (DARTSs) in parallel based on the determined discrete components (Salameh: [0079], the sampled continuous representation α, comprising of continuous architecture parameter values for configuring the architecture cells 100 of the supernet 312, is mapped to a discrete representation α.sub.t.sup.d. FIG. 6 illustrates an example pseudo-code representation 600 of an algorithm that can be implemented at step 504 for discretization of the continuous sampled architecture representation for DARTS or DARTS-based search spaces).
As for claim 7, Salameh-Vasudevan-Piergiovanni discloses: wherein the discrete network search space is sampled by predicting (Salameh: [0021], for each experience tuple in the batch, performing operations comprising: predicting a reward of the candidate architecture based on a current mapping; determining a check loss using quantile regression as a function of the predicted reward and the reward from each experience tuple; and updating the current mapping to minimize the check loss) one or more of a number of layers, a number of initial channels, an operations space, or a use of reduction cells (Salameh: [0060], Each architecture cell 100 may receive outputs and/or states of previous cell as inputs. The output of architecture cell 100 may be obtained by applying a reduction operation, such as concatenation, to some or all of the intermediate nodes 102; [0103], an example final state matrix 900 of a normal cell and a reduction cell based on K=500 top performing architectures located by CADAM on the CIFAR-10 benchmark using a PC-DARTS supernet).
As for claim 10, Salameh-Vasudevan-Piergiovanni discloses: wherein the discrete network search space is sampled based on reinforcement learning (RL) (Salameh: Abstract, A method and system for generating neural architectures to perform a particular task. An actor neural network, as part of a continuous action reinforcement learning (RL) agent).
As per claims 11-15, 17, 20, they recites features that are substantially same as those features claimed by claims 1-5, 7, 10, thus the rationales for rejecting claims 1-5, 7, 10 are incorporated herein.
6. Claims 6, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Salameh and Vasudevan and Piergiovanni as applied on claims 1 and 11, and further in view of Neil Matthew Tinmouth Houlsby et al (US Publication 20220092416 A1, hereinafter Houlsby).
As for claim 6, Salameh-Vasudevan-Piergiovanni does not clearly disclose a floating point operations complexity and pixel-wise measurement, Houlsby discloses: wherein at least one non-differentiable measurement is combined with the reward and comprises at least one of a floating point operations (FLOPs) complexity (Houlsby: [0059], an ordered collection of numeric values, e.g., a vector of floating point or other numeric values, having a fixed dimensionality), an area per pixel (Houlsby: [0077], when the outputs are images, the accuracy measure can be the pixel-wise mean intersection-over-union (mIOU) of the trained instance over the validation data set), a chip area, or an indication of memory consumption.
Salameh and Houlsby are analogous arts because they are in the same field of endeavor, neural architecture search. Therefore, it would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Salameh using the teachings of Houlsby to include measurement accuracy can be pixel-wise mean intersection-over-union. It would provide Salameh’s method with enhanced capabilities of improving the efficiency and accuracy of neural architecture search.
As per claim 16, it recites features that are substantially same as those features claimed by claim 6, thus the rationales for rejecting claim 6 are incorporated herein.
7. Claims 8, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Salameh and Vasudevan and Piergiovanni as applied on claims 1 and 11, and further in view of Arash Vahdat et al (US Publication 20210073612 A1, hereinafter Vahdat).
As for claim 8, Salameh-Vasudevan-Piergiovanni does not clearly disclose Monte-Carlo function, Vahdat discloses: wherein the discrete network search space is sampled based on a Monte-Carlo tree search function (Vahdat: [0082], Monte Carlo estimate is computed by drawing samples from p.sub.φ(z). In at least one embodiment, since we compute training/validation loss in an objective function using a mini-batch of data, we can choose to set a number of architecture samples to a value between one and a number of samples in mini-batch (such as batch size). In this section, we review an effect of choosing a number of architecture samples on variance and efficiency of search, in an embodiment).
Salameh and Vahdat are analogous arts because they are in the same field of endeavor, neural architecture search. Therefore, it would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Salameh using the teachings of Vahdat to include using Monte Carlo function. It would provide Salameh’s method with enhanced capabilities of improving the efficiency and accuracy of neural architecture search.
As per claim 18, it recites features that are substantially same as those features claimed by claim 8, thus the rationales for rejecting claim 8 are incorporated herein.
8. Claims 9, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Salameh and Vasudevan and Piergiovanni as applied on claims 1 and 11, and further in view of Shalinia Mukhopadhyay et al (US Publication 20230334330 A1, hereinafter Mukhopadhyay).
As for claim 9, Salameh-Vasudevan-Piergiovanni does not clearly disclose aging evolutionary search function, Mukhopadhyay discloses: wherein the discrete network search space is sampled based on an aging evolutionary (AE) search function (Mukhopadhyay: [0038], A variety of search techniques exist in EA NAS, but one that is preferred is the Aging Evolutionary Search in the literature).
Salameh and Mukhopadhyay are analogous arts because they are in the same field of endeavor, neural architecture search. Therefore, it would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Salameh using the teachings of Mukhopadhyay to include using aging evolutionary search function. It would provide Salameh’s method with enhanced capabilities of improving the efficiency and accuracy of neural architecture search.
As per claim 19, it recites features that are substantially same as those features claimed by claim 9, thus the rationales for rejecting claim 9 are incorporated herein.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hua Lu whose telephone number is 571-270-1410 and fax number is 571-270-2410. The examiner can normally be reached on Mon-Fri 9:00 am to 6:00 pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Baderman can be reached on 571-272-3644. The fax phone number for the organization where this application or proceeding is assigned is 703-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hua Lu/
Primary Examiner, Art Unit 2118