Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a
judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly
more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-18 are within the four statutory (a process, machine, manufacture or composition of
matter.) Claims 1-6, 13-20 describe a machine and 7-12 describes a process.
With respect to claim 1:
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG
generate customized design space information representing a customized design space of the architecture of the target neural network using the original design space information and the dataset characteristics information, the customized design space being a design space narrower than the original design space. (This is an abstract idea of a "Mental Process." The "generate" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application
Additional elements:
acquire original design space information representing an original design space of an architecture of a target neural network; (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception
The additional elements add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
When considered in combination, these additional elements represent insignificant extra-solution activity, which do not provide an inventive concept.
Therefore, claim 1 is ineligible.
With respect to claim 2:
Step 2A Prong 1: claim 2, which incorporates the rejection of claim 1, recites an additional abstract idea:
extracting a part of the options for each of the one or more factors; and (This is an abstract idea of a "Mental Process." The "extracting" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The extraction could be made manually by an individual.)
generating the customized design space information that includes the extracted part of the options for each of the one or more factors. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 2 is ineligible.
With respect to claim 3:
Step 2A Prong 1: claim 3, which incorporates the rejection of claim 1, recites an additional abstract idea:
extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information; and (This is an abstract idea of a "Mental Process." The "extracting" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The extraction could be made manually by an individual.)
generate the customized design space information based on the design space extracted from the predefined knowledge and the original design space represented by the original design space information. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 3 is ineligible.
With respect to claim 4:
Step 2A Prong 1: claim 4, which incorporates the rejection of claim 1, recites an additional abstract idea:
generate the customized design space information based on the design space output from the model and the original design space represented by the original design space information (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the at least one memory is further configured to store a model that acquires the characteristics of the target dataset and outputs a design space in response to the characteristics of the target dataset being input into the model, (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
inserting the characteristics of the target dataset represented by the dataset characteristics information into the model; (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
acquire the design space output from the model; (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element “the at least one memory…” adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
The additional elements “inserting the characteristics…” and “acquire the design space…” are recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 4 is ineligible.
With respect to claim 5:
Step 2A Prong 1: claim 5, which incorporates the rejection of claim 1, recites an additional abstract idea:
determine characteristics of the representative dataset to generate the dataset characteristics information that represents the determined characteristics of the representative dataset as the characteristics of the target dataset. (This is an abstract idea of a "Mental Process." The "determine" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The determination could be made manually by an individual.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
acquire a representative dataset having characteristics similar to or same as the characteristics of the target datasets; and (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 5 is ineligible.
With respect to claim 6:
Step 2A Prong 1: claim 6, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the characteristics of the target dataset includes a number of classes, a size of input data, a type of input data, a size of bounding box, a distribution of classes, a distribution of the size of input data, or a distribution of the size of bounding box. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 6 is ineligible.
With respect to claim 7:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 7. Therefore, claim 7 is ineligible.
With respect to claim 8:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 8. Therefore, claim 8 is ineligible.
With respect to claim 9:
The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 9. Therefore, claim 9 is ineligible.
With respect to claim 10:
The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 10. Therefore, claim 10 is ineligible.
With respect to claim 11:
The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis that was utilized for claim 5, as described above, is equally applicable to claim 11. Therefore, claim 11 is ineligible.
With respect to claim 12:
The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis that was utilized for claim 6, as described above, is equally applicable to claim 12. Therefore, claim 12 is ineligible.
With respect to claim 13:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 13. Therefore, claim 13 is ineligible.
With respect to claim 14:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 14. Therefore, claim 14 is ineligible.
With respect to claim 15:
The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 15. Therefore, claim 15 is ineligible.
With respect to claim 16:
The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 16. Therefore, claim 16 is ineligible.
With respect to claim 17:
The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis that was utilized for claim 5, as described above, is equally applicable to claim 17. Therefore, claim 17 is ineligible.
With respect to claim 18:
The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis that was utilized for claim 6, as described above, is equally applicable to claim 18. Therefore, claim 18 is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-7, 10-13, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Saboori (US 2022/0335304 A1 with PCT filed November 18th 2019) in view of Kokiopoulou (US 2022/0121906 A1 with PCT filed January 30th 2020).
Regarding claim 1, Saboori teaches
A design space reduction apparatus comprising: at least one memory storing instructions; and at least one processor that is configured to execute the instructions to: ([0012] “In yet another aspect, there is provided a deep neural network optimization engine configured to perform automated design space exploration for deep neural networks, the engine comprising a processor and memory, the memory comprising computer executable instructions for performing the above method.”)
acquire original design space information representing an original design space of an architecture of a target neural network; ([0026] “Turning now to the figures, FIG. 1 illustrates a DNN optimization engine 10 which is configured, as described below, to take an initial DNN 12 and generate or otherwise determine an optimized DNN 14”)
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and
generate customized design space information representing a customized design space of the architecture of the target neural network using the original design space information …, the customized design space being a design space narrower than the original design space. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori does not teach “acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and” or that generating the customized design space is based on the dataset characteristics.
However Kokiopoulou does:
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and ([0025] “the system 100 then generates a target meta-features tensor 104 for the target training dataset 102. The target training dataset 102 includes a plurality of samples and a respective label for each of the samples. For example, if the target machine learning task is an image classification or recognition task, a sample in the dataset 102 can be an image and its respective label can be a ground-truth output that includes scores for each of a set of object classes, with each score representing the likelihood that the image contains an image of an object belonging to the object class. The target meta-features tensor 104 represents features (e.g., characteristics and statistics) of the target training dataset 102. More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”)
Saboori and Kokiopoulou are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou. One would want to do this to reduce computational costs of neural network search (Kokiopoulou [0007]).
Regarding claim 4, Saboori in view of Kokiopoulou teaches claim 1 as outlined above. Saboori further teaches
the at least one memory is further configured to store a model that acquires the characteristics of the target dataset and outputs a design space in response to the characteristics of the target dataset being input into the model, ([0009] “The learning process trains an optimizer agent to adapt large, initial networks into smaller networks of similar performance that satisfy target constraints in a data-driven way.”)
inserting the characteristics of the target dataset represented by the dataset characteristics information into the model; acquire the design space output from the model; generate the customized design space information based on the design space output from the model and the original design space represented by the original design space information. ([0037] “The engine 10 provides for automated optimization of deep learning algorithms. The engine 10 also employs an efficient process for design space exploration 26 of DNNs that can satisfy target computation constraints 19 such as speed, model size, accuracy, power consumption, etc. There is provided a learning process for training optimizer agents that automatically explore design trade-offs starting with large, initial DNNs to produce compact DNN designs in a data-driven way. Once an engineer has trained an initial deep neural network on a training data set to achieve a target accuracy for a task, they would then need to satisfy other constraints for the real-world production environment and computing hardware. The proposed process makes this possible by automatically producing an optimized DNN model suitable for the production environment and hardware 16.”)
Regarding claim 5, Saboori in view of Kokiopoulou teaches claim 1 as outlined above. Kokiopoulou further teaches
acquire a representative dataset having characteristics similar to or same as the characteristics of the target datasets; and determine characteristics of the representative dataset to generate the dataset characteristics information that represents the determined characteristics of the representative dataset as the characteristics of the target dataset. ([0026] “In some implementations, instead of computing above meta-features from the target training dataset 102, the system 100 may process, using a feature generator neural network, the target training dataset 102 to generate the target meta-features tensor. The feature generator neural network has been trained to process a given training dataset to generate a corresponding meta-features tensor for the given training dataset. In these implementations, the target training dataset 102 (or a fraction of the dataset 102) is given as input to the feature generator neural network, and a task embedding is learned directly from samples in the target training dataset 102. The task embedding plays the roles of the meta-features in the target meta-features tensor 104. The feature generator neural network can be part of an evaluator neural network 120 (that is used to evaluate performance of a candidate architecture of the task neural network) and can be jointly trained with the evaluator neural network 120 using a common objective function.”)
Regarding claim 6, Saboori in view of Xue teaches claim 1 as outlined above. Kokiopoulou further teaches:
the characteristics of the target dataset includes a number of classes, a size of input data, a type of input data, a size of bounding box, a distribution of classes, a distribution of the size of input data, or a distribution of the size of bounding box. ([0025] “More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”
Regarding claim 7, Saboori teaches
A control method performed by a computer, comprising: ([0009] “A method for automated optimization, specifically design space exploration, is described.”)
acquire original design space information representing an original design space of an architecture of a target neural network; ([0026] “Turning now to the figures, FIG. 1 illustrates a DNN optimization engine 10 which is configured, as described below, to take an initial DNN 12 and generate or otherwise determine an optimized DNN 14”)
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and
generate customized design space information representing a customized design space of the architecture of the target neural network using the original design space information …, the customized design space being a design space narrower than the original design space. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori does not teach “acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and” or that generating the customized design space is based on the dataset characteristics.
However Kokiopoulou does:
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and ([0025] “the system 100 then generates a target meta-features tensor 104 for the target training dataset 102. The target training dataset 102 includes a plurality of samples and a respective label for each of the samples. For example, if the target machine learning task is an image classification or recognition task, a sample in the dataset 102 can be an image and its respective label can be a ground-truth output that includes scores for each of a set of object classes, with each score representing the likelihood that the image contains an image of an object belonging to the object class. The target meta-features tensor 104 represents features (e.g., characteristics and statistics) of the target training dataset 102. More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”)
Saboori and Kokiopoulou are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou. One would want to do this to reduce computational costs of neural network search (Kokiopoulou [0007]).
Regarding claim 10, Saboori in view of Kokiopoulou teaches claim 7 as outlined above. Saboori further teaches
the computer is configured to store a model that acquires the characteristics of the target dataset and outputs a design space in response to the characteristics of the target dataset being input into the model, ([0009] “The learning process trains an optimizer agent to adapt large, initial networks into smaller networks of similar performance that satisfy target constraints in a data-driven way.”)
inserting the characteristics of the target dataset represented by the dataset characteristics information into the model; acquire the design space output from the model; generate the customized design space information based on the design space output from the model and the original design space represented by the original design space information. ([0037] “The engine 10 provides for automated optimization of deep learning algorithms. The engine 10 also employs an efficient process for design space exploration 26 of DNNs that can satisfy target computation constraints 19 such as speed, model size, accuracy, power consumption, etc. There is provided a learning process for training optimizer agents that automatically explore design trade-offs starting with large, initial DNNs to produce compact DNN designs in a data-driven way. Once an engineer has trained an initial deep neural network on a training data set to achieve a target accuracy for a task, they would then need to satisfy other constraints for the real-world production environment and computing hardware. The proposed process makes this possible by automatically producing an optimized DNN model suitable for the production environment and hardware 16.”)
Regarding claim 11, Saboori in view of Kokiopoulou teaches claim 7 as outlined above. Kokiopoulou further teaches
acquire a representative dataset having characteristics similar to or same as the characteristics of the target datasets; and determine characteristics of the representative dataset to generate the dataset characteristics information that represents the determined characteristics of the representative dataset as the characteristics of the target dataset. ([0026] “In some implementations, instead of computing above meta-features from the target training dataset 102, the system 100 may process, using a feature generator neural network, the target training dataset 102 to generate the target meta-features tensor. The feature generator neural network has been trained to process a given training dataset to generate a corresponding meta-features tensor for the given training dataset. In these implementations, the target training dataset 102 (or a fraction of the dataset 102) is given as input to the feature generator neural network, and a task embedding is learned directly from samples in the target training dataset 102. The task embedding plays the roles of the meta-features in the target meta-features tensor 104. The feature generator neural network can be part of an evaluator neural network 120 (that is used to evaluate performance of a candidate architecture of the task neural network) and can be jointly trained with the evaluator neural network 120 using a common objective function.”)
Regarding claim 12, Saboori in view of Xue teaches claim 7 as outlined above. Kokiopoulou further teaches:
the characteristics of the target dataset includes a number of classes, a size of input data, a type of input data, a size of bounding box, a distribution of classes, a distribution of the size of input data, or a distribution of the size of bounding box. ([0025] “More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”
Regarding claim 13, Saboori teaches
A non-transitory computer-readable storage medium storing a program that causes a computer to perform: ([0011] “In another aspect, there is provided a computer readable medium comprising computer executable instructions for automated design space exploration for deep neural networks.”)
acquire original design space information representing an original design space of an architecture of a target neural network; ([0026] “Turning now to the figures, FIG. 1 illustrates a DNN optimization engine 10 which is configured, as described below, to take an initial DNN 12 and generate or otherwise determine an optimized DNN 14”)
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and
generate customized design space information representing a customized design space of the architecture of the target neural network using the original design space information …, the customized design space being a design space narrower than the original design space. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori does not teach “acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and” or that generating the customized design space is based on the dataset characteristics.
However Kokiopoulou does:
acquire dataset characteristics information representing characteristics of a target dataset that is a collection of data to be analyzed by the target neural network; and ([0025] “the system 100 then generates a target meta-features tensor 104 for the target training dataset 102. The target training dataset 102 includes a plurality of samples and a respective label for each of the samples. For example, if the target machine learning task is an image classification or recognition task, a sample in the dataset 102 can be an image and its respective label can be a ground-truth output that includes scores for each of a set of object classes, with each score representing the likelihood that the image contains an image of an object belonging to the object class. The target meta-features tensor 104 represents features (e.g., characteristics and statistics) of the target training dataset 102. More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”)
Saboori and Kokiopoulou are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou. One would want to do this to reduce computational costs of neural network search (Kokiopoulou [0007]).
Regarding claim 16, Saboori in view of Kokiopoulou teaches claim 13 as outlined above. Saboori further teaches
a model that acquires the characteristics of the target dataset and outputs a design space in response to the characteristics of the target dataset being input into the model, ([0009] “The learning process trains an optimizer agent to adapt large, initial networks into smaller networks of similar performance that satisfy target constraints in a data-driven way.”)
inserting the characteristics of the target dataset represented by the dataset characteristics information into the model; acquire the design space output from the model; generate the customized design space information based on the design space output from the model and the original design space represented by the original design space information. ([0037] “The engine 10 provides for automated optimization of deep learning algorithms. The engine 10 also employs an efficient process for design space exploration 26 of DNNs that can satisfy target computation constraints 19 such as speed, model size, accuracy, power consumption, etc. There is provided a learning process for training optimizer agents that automatically explore design trade-offs starting with large, initial DNNs to produce compact DNN designs in a data-driven way. Once an engineer has trained an initial deep neural network on a training data set to achieve a target accuracy for a task, they would then need to satisfy other constraints for the real-world production environment and computing hardware. The proposed process makes this possible by automatically producing an optimized DNN model suitable for the production environment and hardware 16.”)
Regarding claim 17, Saboori in view of Kokiopoulou teaches claim 13 as outlined above. Kokiopoulou further teaches
acquire a representative dataset having characteristics similar to or same as the characteristics of the target datasets; and determine characteristics of the representative dataset to generate the dataset characteristics information that represents the determined characteristics of the representative dataset as the characteristics of the target dataset. ([0026] “In some implementations, instead of computing above meta-features from the target training dataset 102, the system 100 may process, using a feature generator neural network, the target training dataset 102 to generate the target meta-features tensor. The feature generator neural network has been trained to process a given training dataset to generate a corresponding meta-features tensor for the given training dataset. In these implementations, the target training dataset 102 (or a fraction of the dataset 102) is given as input to the feature generator neural network, and a task embedding is learned directly from samples in the target training dataset 102. The task embedding plays the roles of the meta-features in the target meta-features tensor 104. The feature generator neural network can be part of an evaluator neural network 120 (that is used to evaluate performance of a candidate architecture of the task neural network) and can be jointly trained with the evaluator neural network 120 using a common objective function.”)
Regarding claim 18, Saboori in view of Xue teaches claim 13 as outlined above. Kokiopoulou further teaches:
the characteristics of the target dataset includes a number of classes, a size of input data, a type of input data, a size of bounding box, a distribution of classes, a distribution of the size of input data, or a distribution of the size of bounding box. ([0025] “More specifically, the target meta-features tensor may include one or more of the following meta-features: total number of samples in the target training dataset 102, number of object classes and their distribution, label entropy, total number of features and statistics about the features (min, max, mean or median), mutual information between the features and the labels in the dataset 102, or task id of the target machine learning task.”
Claims 2, 8, 14, are rejected under 35 U.S.C. 103 as being unpatentable over Saboori in view of Kokiopoulou and Dai (NPL: ‘DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search’).
Regarding claim 2, Saboori in view of Kokiopoulou teaches claim 1 as outlined above. Neither Saboori nor Kokiopoulou teaches:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors.
However Dai does:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors. (Page 2 “In the beginning of training, we train it on a subset of target task (only containing easy classes). During training, we progressively prune low performed blocks from our candidate set until we get a compact candidate set for searching on the whole dataset of target task. We consider a loss function with cost constraint which helps find an optimal architecture under search constraint (e.g. , FLOPs).” Here they are using only a portion on the classes implying they had to identify of parts of the design space for training. They use this in finding the optimal architecture.)
Saboori, Kokiopoulou and Dai are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with only using a subset at a time of Dai. One would want to do this to speed the search process up but not using all the data at one time.
Regarding claim 8, Saboori in view of Kokiopoulou teaches claim 7 as outlined above. Neither Saboori nor Kokiopoulou teaches:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors.
However Dai does:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors. (Page 2 “In the beginning of training, we train it on a subset of target task (only containing easy classes). During training, we progressively prune low performed blocks from our candidate set until we get a compact candidate set for searching on the whole dataset of target task. We consider a loss function with cost constraint which helps find an optimal architecture under search constraint (e.g. , FLOPs).” Here they are using only a portion on the classes implying they had to identify of parts of the design space for training. They use this in finding the optimal architecture.)
Saboori, Kokiopoulou and Dai are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with only using a subset at a time of Dai. One would want to do this to speed the search process up but not using all the data at one time.
Regarding claim 14, Saboori in view of Kokiopoulou teaches claim 13 as outlined above. Neither Saboori nor Kokiopoulou teaches:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors.
However Dai does:
the original design space information includes a plurality of options for each of one or more factors of the architecture of the target neural network; extracting a part of the options for each of the one or more factors; and generating the customized design space information that includes the extracted part of the options for each of the one or more factors. (Page 2 “In the beginning of training, we train it on a subset of target task (only containing easy classes). During training, we progressively prune low performed blocks from our candidate set until we get a compact candidate set for searching on the whole dataset of target task. We consider a loss function with cost constraint which helps find an optimal architecture under search constraint (e.g. , FLOPs).” Here they are using only a portion on the classes implying they had to identify of parts of the design space for training. They use this in finding the optimal architecture.)
Saboori, Kokiopoulou and Dai are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with only using a subset at a time of Dai. One would want to do this to speed the search process up but not using all the data at one time.
Claims 3, 9, 15, are rejected under 35 U.S.C. 103 as being unpatentable over Saboori in view of Kokiopoulou and Xue (US 2022/0027739 A1).
Regarding claim 3, Saboori in view of Kokiopoulou teaches claim 1 as outlined above. Saboori further teaches:
generate the customized design space information based on … the original design space represented by the original design space information. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori nor Kokiopoulou teaches “acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information” or that generating the customized design space is based on the predefined knowledge.
However Xue does:
acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information; and ([0016] “As previously noted, historical data resulting from supervised learning may be used to select initial search space 120a. That is, initial search space 120a may be indicated by data resulting from supervised learning as the optimal initial search space 120 for the particular meta features 110.” Here they use historical data (predefined knowledge) to assist in finding the optimal search space.)
Saboori, Kokiopoulou and Xue are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with the historical data implementation of Xue. One would want to do this to be able to reuse data and speed the search process up.
Regarding claim 9, Saboori in view of Kokiopoulou teaches claim 7 as outlined above. Saboori further teaches:
generate the customized design space information based on … the original design space represented by the original design space information. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori nor Kokiopoulou teaches “acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information” or that generating the customized design space is based on the predefined knowledge.
However Xue does:
acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information; and ([0016] “As previously noted, historical data resulting from supervised learning may be used to select initial search space 120a. That is, initial search space 120a may be indicated by data resulting from supervised learning as the optimal initial search space 120 for the particular meta features 110.” Here they use historical data (predefined knowledge) to assist in finding the optimal search space.)
Saboori, Kokiopoulou and Xue are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with the historical data implementation of Xue. One would want to do this to be able to reuse data and speed the search process up.
Regarding claim 15, Saboori in view of Kokiopoulou teaches claim 13 as outlined above. Saboori further teaches:
generate the customized design space information based on … the original design space represented by the original design space information. ([0031] “Using the design space exploration module 26, the engine 10 provides an automated multi-objective design space exploration with respect to defined constraints, where a reinforcement learning based agent explores the design space for a smaller network (student) with similar performance of the given network (teacher) trained on the same task. The agent generates new networks by network transformation operations such as altering a layer (e.g. number of filters), altering the whole network (e.g. adding or removing a layer), etc. This agent can efficiently navigate the design space to yield an architecture which satisfies all the constraints for the target hardware.”)
Saboori nor Kokiopoulou teaches “acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information” or that generating the customized design space is based on the predefined knowledge.
However Xue does:
acquiring predefined knowledge that represents a plurality of associations between characteristics of dataset and a design space of an architecture of a neural network; extracting, from the predefined knowledge, the design space that is associated with the characteristics of dataset that matches the characteristics of the target dataset represented by the dataset characteristics information; and ([0016] “As previously noted, historical data resulting from supervised learning may be used to select initial search space 120a. That is, initial search space 120a may be indicated by data resulting from supervised learning as the optimal initial search space 120 for the particular meta features 110.” Here they use historical data (predefined knowledge) to assist in finding the optimal search space.)
Saboori, Kokiopoulou and Xue are considered analogous art to the claimed invention because they are in the same field of endeavor neural architecture search. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the neural architecture search of Saboori with the meta features of Kokiopoulou with the historical data implementation of Xue. One would want to do this to be able to reuse data and speed the search process up.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL PATRICK GRUSZKA whose telephone number is (571)272-5259. The examiner can normally be reached M-F 9:00 AM - 6:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL GRUSZKA/ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121