Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed 03/12/2026. The status of the claims are as follows:
Claims 1-20 are pending.
Claims 1, 8, and 15 are amended.
Response to Amendments
The Office Action is in response to Applicant’s communication filed 03/12/2026 in response to office action mailed 12/12/2025. The Applicant’s remarks and any amendments to the claims or specification have been considered with the results that follow.
Response to Arguments
Regarding 35 U.S.C. § 101
The Examiner has considered Applicant’s arguments and amendments regarding the rejection of claims 1-20 under 35 U.S.C. § 101 and finds them persuasive.
Applicant argues that the pending claims are directed to an improvement in neural architecture search (NAS), including using a gradient-based search result as an input or seed for a sampling method search in order to accelerate the sampling search and reduce the time required to find an optimal neural network architecture. Applicant further points to the Specification’s discussion of the problems with prior NAS techniques and the disclosed improvement of combining gradient-based searching with sampling-based searching.
Upon reconsideration, the Examiner agrees that the record supports withdrawal of the prior § 101 rejection. In particular, the claims, as amended, recite a specific NAS sequence in which a first search space comprising candidate layers is obtained, a gradient-based search is performed in the first search space to obtain a first architecture, and a sampling method search of the first search space is performed using the first architecture as an initial sample. The claims therefore recite more than merely obtaining information at a high level of generality, The ordered combination reflects the disclosed NAS improvement of using a result of the gradient-based search to initialize the sampling method search.
Accordingly, the rejection of claims 1-20 under 35 U.S.C. § 101 is withdrawn.
Regarding 35 U.S.C. § 103
The Examiner has considered Applicant’s arguments and amendments regarding the rejection of claims 1-20 under 35 U.S.C. § 103 and finds them unpersuasive.
Applicant argues that amended independent claims 1, 8, and 15 patentably distinguish over Abdelfattah in view of Tan because the claims now recite “performing a sampling method search of the first search space utilizing the first architecture as an initial sample”. Applicant contends that Abdelfattah performs its searching algorithm only on a subset of models selected during warmup, whereas the claimed invention allegedly uses a selected architecture as a seed for searching the engine first search space. Applicant further argues that modifying Abdelfattah in view of Tan would render Abdelfattah unsuitable for its intended purposes because Abdelfattah allegedly depends on reducing the search space to a subset.
These arguments are not persuasive.
As an initial matter, the amended language does not require exhaustively training, evaluating, or searching every possible model in the first search space. The claim recites a “sampling method search of the first search space”, not an exhaustive search of every architecture in the first search space. A sampling or evolutionary search may operate over a defined search space while beginning from one or more selected initial samples or seed architectures. Thus, the fact that an initial subset or initial sample is selected does not, by itself, mean that the subsequent sampling search is not a search “of” the broader search space.
Abdelfattah teaches NAS over a predefined architecture search space A, and further explains that because training all models in A is infeasible, NAS is implemented as an iterative process in which models are trained and used to influence selection of further models, with a searching function proposing new architecture based on the history of previous architectures. Abdelfattah also teaches that the searching algorithm may include Aging Evolution, REINFORCE, Random Search, or a GCN-based binary predictor. Accordingly, Abdelfattah teaches an iterative NAS search in which selected and scored models are used by a searching algorithm to guide later architecture proposals within the architecture search framework. See updated § 103 rejection below.
Applicant’s argument that Abdelfattah searches only a reduced subset is also not commensurate with the scope of the amended claims. The claims do not require that the sampling method search begin from all architectures in the first search space. Rather, the claims expressly require that the sampling method search us the first architecture “as an initial sample”. Thus, the claims themselves contemplate that the sampling method search is initialized using a selected architecture. Abdelfattah’s selection of high-scoring candidate models for use by a subsequent NAS searching algorithm is consistent with this claimed use of an initial sample or seed.
Nor would the proposed combination render Abdelfattah unsuitable for its intended purpose. Abdelfattah recognizes that training all models in a search space is infeasible and therefore uses search and selection techniques to reduce computational burden. The rejection does not propose modifying Abdelfattah to exhaustively train every model in the search space. Instead, the rejection relies on Tan for the known NAS teaching that architectures may be iteratively sampled within a search space to generate new architectures, and on Tan’s layer/block-based search-space structure. Using a high-performing architecture identified by Abdelfattah’s gradient=related scoring as an initial sample for a later sampling/evolutionary search over the applicable search space would have predictably improved efficiency by starting the sampling search from a promising architecture rather than an arbitrary or poor initial sample.
Applicant’s argument also does not account for the ordinary operation of evolutionary and sampling-based NAS methods. In such methods, an initial population or seed architecture may be selected from a search space, while mutation, sampling, or proposal operations are still performed with respect to the search space. This is inconsistent with the claimed language, which requires use of the first architecture as an initial sample for the sampling method search. The amended phrase “of the first search space” therefore does not distinguish over the combination of Abdelfattah and Tan, because the combination teaches or suggests selecting a promising architecture from the NAS search space and using that architecture as a starting point for a subsequent sampling/evolutionary search operating within the NAS search space.
Applicant further argues that Tan mostly refers generally to prior-art iterative sampling and does not provide a proper motivation to modify Abdelfattah. This argument is not persuasive. Tan teaches that NAS techniques may iteratively sample architectures within a search space to generate new architectures and further teaches layer/block-based search spaces having candidate layer configurations. Abdelfattah teaches selecting and scoring architectures and applying NAS searching algorithms, including Aging Evolution and Random Search. A person of ordinary skill in the art would have had reason to use Abdelfattah’s scored, selected, or gradient-guided architecture as an initial sample for Tan’s iterative sampling search because both references address the same NAS problem of efficiently identifying high-performing neural network architectures while reducing unnecessary search cost. The combination merely applies a known sampling/evolutionary NAS technique using a known beneficial initialization strategy, namely beginning the search from a promising architecture rather than from an arbitrary initial architecture.
For the same reasons, Applicant’s arguments regarding independent claims 8 and 15 are not persuasive. Claims 8 and 15 recite apparatus and computer-readable-medium analogs of claim 1 and do not include additional limitations that patentably distinguish over Abdelfattah in view of Tan. Similarly, the dependent claim remain unpatentable for the reasons set forth in the updated § 103 rejections below.
Accordingly, the rejections of claims 1-20 under 35 U.S.C. § 103 over Abdelfattah in view of Tan is maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Abdelfattah et al., (US20220101089A1) in view of Tan et al., (US20200143227Al).
Regarding claim 1, Abdelfattah in view of Tan teach a method for performing a neural architecture search (NAS), the method comprising;
“obtaining a first search space comprising a plurality of candidate layers for a neural network architecture;” – Abdelfattah teaches this limitation in part. Abdelfattah describes NAS as an optimization over a predefined search space A of architectures and teaches that:
“a is an architecture from the predefined search space A (set of architecture which is considered when searching)” (Abdelfattah, p. 1, ¶[0004])
Abdelfattah’s method explicitly includes obtaining a set of candidate models from the search space:
“A computer-implemented method using a searching algorithm to design a neural network architecture is provided, the method including: obtaining a plurality of neural network models; selecting a first subset of the plurality of neural network models; applying the searching algorithm to the selected subset of models; and identifying an optimal neural network architecture…” (Abdelfattah, § Abstract)
Thus, Abdelfattah teaches obtaining a search space of candidate neural architectures/models (a “first search space”), and obtaining a plurality of models from that space.
“performing a gradient-based search in the first search space to obtain a first architecture;” – Abdelfattah teaches this limitation. Abdelfattah’s NAS procedure uses gradient-related quantities as the scoring signal for architectures and applies them within an iterative search algorithm:
“The score may be obtained using an approximate scoring function. For example, the score may be obtained by calculating a gradient of a training loss function. The score may be obtained for a single batch of data…” (Abdelfattah, p. 2, ¶[0019])
Abdelfattah then shows that these scores are used in the NAS loop:
“obtaining a plurality of neural network models; selecting a first subset of the plurality of neural network models; applying the searching algorithm to the selected subset of models; and identifying an optimal neural network architecture by repeating the selecting and applying for a fixed number of iterations; wherein at least one score indicative of validation loss for each model is used in or alongside at least one of the selecting and applying.” (Abdelfattah, § Abstract)
Abdelfattah also lists search algorithms such as REINFORCE and other policy-gradient methods:
“… the following algorithms are considered: Aging Evolution, REINFORCE with LSTM-based policy network, Random search, GCN-based binary predictor.” (Abdelfattah, p. 4, ¶[0042])
These are gradient-based with respect to an architecture selection policy. A POSITA would understand that as performing a gradient-based search over the search space
A
to select architectures; choosing the best-scoring architecture from this gradient-guided phase corresponds to obtaining a “first architecture”.
“performing a sampling method search ” – Abdelfattah teaches this limitation in part. Abdelfattah teaches applying a searching algorithm to selected models:
“applying the searching algorithm to the selected subset of models” (Abdelfattah, § Abstract)
Abdelfattah further teaches that the searching algorithm may include sampling-type NAS search algorithms, including Aging Evolution and Random Search:
“The searching algorithm may include any appropriate algorithm… For example, the searching algorithm may be selected from Aging Evolution, REINFORCE with LSTM-based policy network. Random search, GCN-based binary predictor but is not limited to these algorithms.” (Abdelfattah, p. 2, ¶[0016])
Abdelfattah also teaches that NAS is performed over a predefined search space A:
“a is an architecture from the predefined search space A (set of architecture which is considered when searching)” (Abdelfattah, pg. 1, ¶[0004])
Abdelfattah further teaches that NAS is implemented as an iterative process in which trained models influence selection of further models and a searching function proposes new architectures based on prior architectures:
“Training all models in A is infeasible and thus, NAS is usually implemented as an iterative process where in each iteration some models are trained in order to get their
L
v
a
l
values, which are later used to influence selection of further models, which are then again trained, and so on.” (Abdelfattah, p. 1, ¶[0005])
“a searching function which proposes new architectures (being given history of previous ones)” (Abdelfattah, p. 1, ¶[0005])
Abdelfattah further teaches selecting and ranking candidate models based on scope:
“Prior to selecting the first subset, the method may, for example, comprise selecting a sample of the plurality of neural network models, obtaining the score which is indicative of validation loss for each model in the sample, and ranking the models within the sample based on the obtained score.” (Abdelfattah, p. 2, ¶[0022])
“The first subset may then be selected from the ranked models, e.g. by selecting the highest ranked models.” (Abdelfattah, p. 2, ¶[0022])
“and obtaining a second architecture as an output of the sampling method search.” – Abdelfattah teaches this limitation. Abdelfattah teaches that the iterative search identifies an optimal architecture as the output of the NAS procedure:
“… identifying an optimal neural network architecture by repeating the selecting and applying for a fixed number of iterations; wherein at least one score indicative of validation loss for each model is used in or alongside at least one of the selecting and applying.” (Abdelfattah, § Abstract)
This optimal architecture is the final result of the sampling-based NAS algorithm and corresponds to the claimed “second architecture as an output of the sampling method search”.
Abdelfattah does not teach:
“… a plurality of candidate layers… “
“… of the first search space utilizing the first architecture as an initial sample;”
Tan, however, teaches these limitations:
“… a plurality of candidate layers… “ – Tan clarifies that such NAS search spaces are defined in terms of candidate layers. Tan explains that the network is composed of multiple blocks, each with a sub-search space of candidate layer configurations, where the searchable parameters include:
“… a number of identical layers included in the block and an operation to be performed by each of the number of identical layers…” (Tan, p. 2, ¶[0014])
A POSITA would therefore understand Abdelfattah’s search space
A
as an NAS search space comprising candidate layers/blocks as in Tan, satisfying this limitation.
“… utilizing the first architecture as an initial sample;” – Tan teaches that NAS search techniques may iteratively search and sample architectures within a search space:
“Existing neural architecture search techniques often work by iteratively searching within a search space that defines the bounds of the search.” (Tan, p. 1, ¶[0008])
“For example, a search technique can include iteratively sampling architectures within the search space to generate new architectures.” (Tan, p. 1, ¶[0008])
Thus, Tan teaches the portion of the limitation requiring the sampling search to be performed “of the first search space”, i.e., by “iteratively sampling architectures within the search space”. Abdelfattah teaches selecting a highest-ranked model/architecture based on scoring and applying a NAS searching algorithm to selected models, and Tan teaches iterative sampling of architectures within the search space. Therefore, it would have been obvious to a POSITA to use Abdelfattah’s selected/high-scoring architecture as the initial sample for Tan’s sampling method search within the first search space.
A POSITA would have been motivated to make this modification because using a high-scoring architecture identified by Abdelfattah’s gradient-based or gradient-scored phase as the initial sample for a later sampling search would predictably start the sampling search from a promising architecture, rather than from an arbitrary architecture, thereby improving search efficiency and reducing the number of iterations/evaluations needed to identify an optimal architecture.
Regarding claim 2, Abdelfattah in view of Tan teach the method according to claim 1 wherein:
the obtaining the first search space comprises obtaining a plurality of sub-spaces, including the first search space, each of the plurality of sub-spaces comprising a set of candidate layers; and the performing the gradient- based search comprises performing a plurality of gradient-based searches respectively in the plurality of sub-spaces to obtain a plurality of first architectures.
and the performing the gradient- based search comprises performing a plurality of gradient-based searches respectively in the plurality of sub-spaces to obtain a plurality of first architectures. – Abdelfattah teaches this limitation. Abdelfattah discloses using gradients of a training loss to score candidate models:
“The score may be obtained using an approximate scoring function. For example, the score may be obtained by calculating a gradient of a training loss function. The score may be obtained for a single batch of data…” (Abdelfattah, p. 2, ¶[0019])
These scores are used in the NAS loop:
“obtaining a plurality of neural network models; selecting a first subset of the plurality of neural network models; applying the searching algorithm to the selected subset of models; and identifying an optimal neural network architecture by repeating the selecting and applying for a fixed number of iterations; wherein at least one score indicative of validation loss for each model is used in or alongside at least one of the selecting and applying.” (Abdelfattah, § Abstract)
The searching algorithm itself is a NAS algorithm over that search space
A
:
“The searching algorithm may include any appropriate algorithm… For example, the searching algorithm may be selected from Aging Evolution, REINFORCE with LSTM-based policy network, Random search, GCN-based binary predictor but is not limited to these algorithms.” (Abdelfattah, p. 2, ¶[0016])
Abdelfattah does not teach:
the obtaining the first search space comprises obtaining a plurality of sub-spaces, including the first search space, each of the plurality of sub-spaces comprising a set of candidate layers;
Tan, however, teaches this limitation:
the obtaining the first search space comprises obtaining a plurality of sub-spaces, including the first search space, each of the plurality of sub-spaces comprising a set of candidate layers; - Tan teaches a search space partitioned into multiple layer-wise sub-spaces. Tan explains how such a search space can be factorized into sub-search spaces associated with different blocks, each sub-search space comprising candidate layer configurations:
“The initial network structure includes a plurality of blocks. The method includes associating, by the one or more computing devices, a plurality of sub-search spaces respectively with the plurality of blocks. The sub-search space for each block has one or more searchable parameters associated therewith.” (Tan, p. 1, ¶[0012])
Tan further discloses that:
“The plurality of searchable parameters for each block includes at least a number of identical layers included in the block and an operation to be performed by each of the number of identical layers included in the block.” (Tan, p. 2, ¶[0014])
Thus, Tan teaches that the NAS search space is organized as a plurality of sub-search spaces, each sub-space comprising a set of candidate layers (number and type of layers in the block). A POSITA would have found it obvious to implement Abdelfattah’s predefined search space
A
using Tan’s factorized hierarchical construction, i.e., by obtaining a plurality of layer-based sub-spaces (one per block), with the “first search space” being one of these spaces.
A POSITA would have been motivated to partition Abdelfattah’s search space according to Tan’s factorized hierarchical scheme and to run the known gradient-based scoring/search within each sub-space because this (i) exploits Tan’s teaching that factorizing the search space into block-wise sub-search-spaces with layer parameters balances flexibility and search-space size, and (ii) uses Abdelfattah’s gradient-based scoring to efficiently find good candidates in each sub-space, yielding multiple promising “first architectures” that can then be used in the subsequent sampling/evolutionary search.
Regarding claim 3, Abdelfattah in view of Tan teach the method according to claim 2, wherein the
performing the sampling method search comprises performing the sampling method search utilizing the plurality of first architectures as initial seeds. – Abdelfattah teaches this limitation. Abdelfattah teaches using a selected plurality of promising architectures as the starting set for the subsequent NAS search. Abdelfattah explains that a sample of models is drawn from the plurality of models, scored, and then a “first subset” of the best models is selected:
“selecting a sample of the plurality of neural network models, obtaining the score which is indicative of validation loss for each model in the sample, and ranking the models within the sample based on the obtained score. The first subset may then be selected from the ranked models, e.g. by selecting the highest ranked models. The sample is preferably larger… Such a sample selection may be referred to as a warm-up phase.” (Abdelfattah, p. 2, ¶[0022])
Abdelfattah further states that these selected models are the ones to which the NAS searching algorithm (e.g., Aging Evolution, REINFORCE, Random search) is applied:
“The searching algorithm may include any appropriate algorithm… For example, the searching algorithm may be selected from Aging Evolution, REINFORCE with LSTM-based policy network, Random search, GCN-based binary predictor… Typically each selected model is trained when applying the searching algorithm during a neural architecture search” (Abdelfattah, p. 2, ¶[0016])
In standard evolutionary / sampling-based NAS, the selected top models from the initial population or seeds for the search; subsequent architectures are obtained by sampling/mutating these seeds. Thus, Abdelfattah teaches that the sampling-based NAS stage operates from a selected plurality of promising architectures, i.e., it utilizes that plurality of first architectures as initial seeds for the sampling method search.
A POSITA would have been motivated to use the plurality of first architectures (one per sub-space from claim 2) as the initial seeds for the sampling method search because Abdelfattah already selects and ranks a subset of high-scoring models and then applies the NAS searching algorithm to those selected models. Using the already identified best architectures as the initial population in an evolutionary / sampling search is a routine NAS design choice that accelerates convergence and improves search quality, and represents no more than ordinary predictable use of Abdelfattah’s selected subset within the multi-sub-space framework provided by Tan.
Regarding claim 4, Abdelfattah in view of Tan teach the method according to claim 3, wherein the
sampling method search comprises an evolutionary search algorithm. – Abdelfattah teaches this limitation. Abdelfattah’s “searching algorithm” corresponds to the applicant’s “sampling method search”. Abdelfattah explicitly discloses that this searching algorithm can be an evolutionary NAS algorithm:
“The searching algorithm may include any appropriate algorithm and may include an algorithm which uses artificial intelligence or machine learning. For example, the searching algorithm may be selected from Aging Evolution, REINFORCE with LSTM-based policy network, Random search, GCN-based binary predictor but is not limited to these algorithms.” (Abdelfattah, p. 2, ¶[0016])
“Aging Evolution” is a well-known evolutionary search algorithm used for neural architecture search; thus, Abdelfattah expressly teaches that the sampling/search method used in its NAS framework comprises an evolutionary algorithm.
It would have been obvious to a person of ordinary skill in the art to implement the “sampling method search” of claim 3 using one of the explicitly suggested NAS algorithms in Abdelfattah, including Aging Evolution, which is an evolutionary search algorithm. Choosing among Abdelfattah’s listed algorithms (e.g., evolutionary vs. random vs. RL-based) is a routine design choice within the NAS framework, balancing exploration behavior, convergence speed, and implementation complexity, and yields no unexpected technical effect beyond the ordinary tradeoffs of known NAS controllers.
Regarding claim 5, Abdelfattah in view of Tan teach the method according to claim 4, wherein the
evolutionary search algorithm utilizes a search space . – Abdelfattah teaches this limitation in part. Abdelfattah teaches that its NAS controller (including Aging Evolution) searches over a predefined architecture search space
A
:
“…a is an architecture from the predefined search space A (set of architecture which is considered when searching)…” (Abdelfattah, p. 1, ¶[0004])
Abdelfattah further discloses that the “searching algorithm” (which may be Aging Evolution) operates on these architectures:
“The searching algorithm may include any appropriate algorithm… For example, the searching algorithm may be selected from Aging Evolution, REINFORCE with LSTM-based policy network, Random search, GCN-based binary predictor…” (Abdelfattah, p. 2, ¶[0016])
Thus, Abdelfattah teaches an evolutionary search algorithm (Aging Evolution) that utilizes a NAS search space
A
of candidate architectures.
Abdelfattah does not on its own specify that
A
is formed as a union of multiple sub-spaces, i.e., Abdelfattah does not teach:
“… which is a union of the plurality of sub-spaces.”
Tan, however, teaches this limitation:
“… which is a union of the plurality of sub-spaces.” – Tan teaches that the factorized hierarchical search space is formed from the block-wise sub-search spaces:
“The final search space is a concatenation of all sub search spaces for each block.” (Tan, p. 6, ¶[0063])
“Suppose the network is partitioned into
B
blocks, and each block has a sub search space of size
S
with average
N
layers per block, then the total search space size would be
S
B
, versing the flat per-layer search space with size
S
B
*
N
.” (Tan, p. 6, ¶[0067])
A POSITA would understand that the “final search space” used by the NAS algorithm is obtained by combining (i.e., taking the union/concatenation of) the plurality of sub-search spaces, corresponding to the claimed search space “which is a union of the plurality of sub-spaces”.
Starting from Abdelfattah’s evolutionary NAS over a generic search space
A
, it would have been obvious to a POSITA to instantiate
A
using Tan’s factorized hierarchical search space, in which (i) each block has its own sub-space of candidate layer configuration, and (ii) the overall NAS search space is formed by considering architecture assembled from these block-level options (the union of the sub-spaces). Doing so is simply adopting a known way (Tan) of structuring and parameterizing the search space for an evolutionary NAS algorithm (Abdelfattah) to reduce search complexity and better control architectural variation, with the predictable benefits Tan explicitly identifies (balanced diversity, reduced search cost). There is no change to the underlying evolutionary algorithm; only the organization of its search space is specified.
Regarding claim 6, Abdelfattah in view of Tan teach the method according to claim 4, wherein the
evolutionary search algorithm is repeated over a number of iterations. – Abdelfattah teaches this limitation. Abdelfattah explains that NAS is inherently an iterative search in which the search algorithm is repeatedly applied:
“Training all models in A is infeasible and thus, NAS is usually implemented as an iterative process where in each iteration some models are trained in order to get their
L
v
a
l
values, which are later used to influence selection of further models, which are then again trained, and so on. Being given a maximum number of models which can be trained (T) and a searching function which proposes new architectures (being given history of previous ones)…” (Abdelfattah, p. 1, ¶[0005])
In the high-level method description, Abdelfattah further states that the searching procedure (which includes the NAS algorithm such as Aging Evolution) is repeated for a fixed number of iterations:
“…obtaining a plurality of neural network models; selecting a first subset of the plurality of neural network models; applying the searching algorithm to the selected subset of models; and identifying an optimal neural network architecture by repeating the selecting and applying for a fixed number of iterations; wherein a score indicative of validation loss for each model is used in or alongside at least one of the selecting and applying steps.” (Abdelfattah, p. 1, ¶[0009])
Thus, Abdelfattah expressly discloses that NAS searching algorithm (which, in claim 4 is the evolutionary search algorithm) is run iteratively and repeated for a fixed number of iterations, i.e., “repeated over a number of iterations” as recited in claim 6.
A POSITA would have understood that any evolutionary NAS algorithm (such as Aging Evolution explicitly listed in Abdelfattah) is executed by repeatedly performing selection, variation, and evaluation steps over many iterations, often bounded by a pre-set iteration or model budget T. Running the evolutionary search algorithm ‘over a number of iterations’ is therefore the standard, inherent way such algorithms operate in NAS, and specifying this in claim 6 does not add any non-obvious technical feature beyond Abdelfattah’s explicit teaching of iterative repetition.
Regarding claim 7, Abdelfattah in view of Tan teach the method according to claim 6, wherein the
number of iterations is based on a predetermined threshold. – Abdelfattah teaches this limitation. Abdelfattah describes NAS as an iterative process and explicitly introduces a fixed search budget limiting how many models can be trained:
“…NAS is usually implemented as an iterative process where in each iteration some models are trained in order to get their
L
v
a
l
values, which are later used to influence selection of further models, which are then again trained, and so on.” (Abdelfattah, p. 1, [0005])
Abdelfattah then formulates the NAS problem under this constraint:
“Being given a maximum number of models which can be trained (T) and a searching function which proposes new architectures…“ (Abdelfattah, p. 1., ¶[0005])
In Abdelfattah’s framework, each iteration involves selecting an training some architecture(s), and the process continues until the predefined budget
T
(the “maximum number of models which can be trained”) is exhausted. Because each iteration consumes part of this budget, the total number of iterations is inherently determined by a predetermined threshold
T
.
A POSITA would thus understand that the iteration count of the NAS loop is not arbitrary; it is governed by the pre-specified budget
T
, i.e., a predetermined threshold on the number of models/steps the search is allowed to perform. Choosing such a threshold
T
(e.g., due to compute/time limits or experimental design) is a routine NAS design choice. Accordingly, the claim 7 feature that “the number of iterations is based on a predetermined threshold” is simply the normal way Abdelfattah’s NAS is run: the iterative search is carried until the maximum number of models threshold
T
is reached, which directly corresponds to the claimed “predetermined threshold”.
Because Abdelfattah already teaches an iterative NAS process with the number of iterations governed by a predefined maximum training budget
T
, it would have been obvious at the time of the claimed invention for a POSITA to implement claim 6’s evolutionary search such that “the number of iterations is based on a predetermined threshold”.
Regarding claims 8-14 (apparatus/system claims)
Each of claims 8-14 is the apparatus/system analog of an already-analyzed method claim (8 [Wingdings font/0xDF][Wingdings font/0xE0] 1; 9 [Wingdings font/0xDF][Wingdings font/0xE0] 2; 10 [Wingdings font/0xDF][Wingdings font/0xE0] 3; 11 [Wingdings font/0xDF][Wingdings font/0xE0] 4; 12 [Wingdings font/0xDF][Wingdings font/0xE0] 5; 13 [Wingdings font/0xDF][Wingdings font/0xE0] 6; 14 [Wingdings font/0xDF][Wingdings font/0xE0] 7). In each of claims 8-14, the functional language associated with “at least one memory storing computer-executable instructions” and “at least one processor configured to execute the computer-executable instructions to …” merely recasts the same NAS operations already recited in method claims 1-7, including obtaining a search space of candidate layers, performing gradient-based searches over the search space/sub-spaces to obtain first architectures, performing a sampling/evolutionary search of the first search space using the first architecture/ architectures as initial samples or seeds, obtaining a second architecture and iterating the search with thresholds, as functionality of a generic processor and memory arrangement.
Both Abdelfattah and Tan disclose computer-implemented NAS techniques. Abdelfattah teaches a “computer-implemented method using a searching algorithm to design a neural network architecture”, including “obtaining a plurality of neural network models”, “applying the searching algorithm”, and “identifying an optimal neural network architecture”. Abdelfattah further teaches that the searching algorithm may include “Aging Evolution”, “REINFORCE with LSTM-based policy network”, “Random search”, and “GCN-based binary predictor”. Tan likewise teaches that neural architecture search techniques may “iteratively sample architectures within the search space to generate new architectures”. Thus, a POSITA would have found it obvious to implement the obvious methods of claims 1-7 on ordinary computer hardware using standard processor and memory configurations, since that is the ordinary and expected implementation form for NAS algorithms.
Because each of claims 8-14 merely requires generic computing components configured to perform the same obvious steps already rejected in method claims 1-7, and does not introduce any non-obvious hardware configuration or unconventional computer implementation, claims 8-14 are unpatentable under 35 U.S.C. § 103 over Abdelfattah in view of Tan for the same reasons as their corresponding method claims 1-7, respectively.
Regarding claims 15-20 (computer-readable medium/computer program product claims)
Each of claims 15-20 is the non-transitory computer-readable medium (CRM) analog of an already-analyzed method claim (15 [Wingdings font/0xDF][Wingdings font/0xE0] 1; 16 [Wingdings font/0xDF][Wingdings font/0xE0] 2; 17 [Wingdings font/0xDF][Wingdings font/0xE0] 3; 18 [Wingdings font/0xDF][Wingdings font/0xE0] 4; 19 [Wingdings font/0xDF][Wingdings font/0xE0] 5; 20 [Wingdings font/0xDF][Wingdings font/0xE0] 6). For each of claims 15-20, the functional language associated with “instructions executable by at least one processor to cause the at least one processor to perform a method comprising …” merely causes a generic processor to perform the same NAS operations already found obvious in method claims 1-6, including obtaining a search space of candidate layers, performing a gradient-based search in the first search space to obtain a first architecture, performing a sampling method of the first search space utilizing the first architecture as an initial sample, obtaining a second architecture, and the additional dependent-claim search-space/evolutionary-search features.
The “non-transitory computer-readable recording medium” and “instructions executable by at least one processor” are standard program-product implementation elements. Abdelfattah teaches a “computer implemented method using a searching algorithm to design a neural network architecture”, and Tan teaches NAS techniques that “iteratively sample architectures within the search space to generate new architectures”. A POSITA would have found it obvious to embody the known NAS methods, as in claims 1-6, as program instructions stored on a non-transitory recording medium so that a processor can perform the method, because this is a routine and expected implementation form for software-based NAS algorithms.
Because each of claims 15-20 is the program-product analog of an obvious method claim and merely implements the same NAS search operations as instructions on a generic non-transitory computer-readable medium executed by a generic processor, without adding any non-obvious structural or functional detail, claims 15-20 are unpatentable under 35 U.S.C. § 103 over Abdelfattah in view of Tan for the same reasons as method claims 1-6, respectively.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul Coleman whose telephone number is (571)272-4687. The examiner can normally be reached Mon-Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL COLEMAN/ Examiner, Art Unit 2126
/DAVID YI/ Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights