DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an input module” as claimed in claim 7 stating “an input module configured to obtain a training dataset, a plurality of block design elements of a plurality of blocks, at least one hyperparameter setting value of each of a plurality of hyperparameters of a neural network, and a test dataset.”
“a computing module” as claimed in claim 7 stating “wherein the computing module is configured to execute a hyperparameter optimization algorithm according to the training dataset, the plurality of block design elements, and the at least one hyperparameter setting value to generate a hyperparameter combination” and “the computing module is further configured to output the hyperparameter combination when the model performance reaches a threshold.”
“a neural network module” as claimed in claim 7 stating “wherein the neural network module is configured to execute a neural network based on the hyperparameter combination and input the test dataset to evaluate a model performance of the neural network.”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 7-12 are rejected under 35 U.S.C. 101 because the claims are directed towards an abstract idea without significantly more.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 1:
Claim 7 recites a system and is thus a machine, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 7 recites:
a computing module communicably connected to the input module, wherein the computing module is configured to execute a hyperparameter optimization algorithm according to the training dataset, the plurality of block design elements, and the at least one hyperparameter setting value to generate a hyperparameter combination, the hyperparameter combination comprises one of the at least one hyperparameter setting value corresponding to each of the plurality of hyperparameters (This limitation is a mental process as it encompasses a human mentally generating hyperparameters and is thus an evaluation.)
Therefore, claim 7 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 7 further recites additional elements of:
A multi-branch network architecture searching system (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
applicable to an electronic device comprising a processor, comprising (This element does not integrate the abstract idea into a practical application because it recites generic computing components on which to perform the abstract idea (see MPEP 2106.05(f)).)
an input module configured to obtain a training dataset, a plurality of block design elements of a plurality of blocks, at least one hyperparameter setting value of each of a plurality of hyperparameters of a neural network, and a test dataset, wherein the training dataset includes a plurality of input data, the plurality of blocks forms an architecture of the neural network, the plurality of blocks is configured to perform a feature extraction on one of the plurality of input data to generate an output data (This element does not integrate the abstract idea into a practical application because it recites the insignificant extra-solution activity of receiving data (see MPEP 2106.05(g)).)
a neural network module communicably connected to the input module and the computing module, wherein the neural network module is configured to execute a neural network based on the hyperparameter combination and input the test dataset to evaluate a model performance of the neural network (This element does not integrate the abstract idea into a practical application because it recites generic computing components on which to perform the abstract idea (see MPEP 2106.05(f)).)
and the computing module is further configured to output the hyperparameter combination when the model performance reaches a threshold (This element does not integrate the abstract idea into a practical application because it recites generic computing components on which to perform the abstract idea (see MPEP 2106.05(f)).)
Therefore, claim 7 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 7 do not provide significantly more than the abstract
idea itself, taken alone and in combination because
A multi-branch network architecture searching system uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
applicable to an electronic device comprising a processor, comprising uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
an input module configured to obtain a training dataset, a plurality of block design elements of a plurality of blocks, at least one hyperparameter setting value of each of a plurality of hyperparameters of a neural network, and a test dataset, wherein the training dataset includes a plurality of input data, the plurality of blocks forms an architecture of the neural network, the plurality of blocks is configured to perform a feature extraction on one of the plurality of input data to generate an output data is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
a neural network module communicably connected to the input module and the computing module, wherein the neural network module is configured to execute a neural network based on the hyperparameter combination and input the test dataset to evaluate a model performance of the neural network uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
and the computing module is further configured to output the hyperparameter combination when the model performance reaches a threshold uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 7 is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 8 recites the same abstract idea as claim 7. Therefore, claim 8 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of:
wherein the plurality of blocks comprises a plurality of normal blocks and a plurality of reduction blocks, each of the plurality of normal blocks is configured to preserve a dimension of one of the plurality of input data, and each of the plurality of reduction blocks is configured to reduce the dimension of the plurality of input data (This element does not integrate the abstract idea into a practical application because it is further modifying the received plurality of blocks defined in claim 7 and thus recites the insignificant extra-solution activity of receiving data.)
Therefore, claim 8 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 8 do not provide significantly more than the abstract
idea itself, taken alone and in combination because
wherein the plurality of blocks comprises a plurality of normal blocks and a plurality of reduction blocks, each of the plurality of normal blocks is configured to preserve a dimension of one of the plurality of input data, and each of the plurality of reduction blocks is configured to reduce the dimension of the plurality of input data is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 8 is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 9 recites the same abstract idea as claim 8. Therefore, claim 9 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of:
wherein the architecture of the neural network comprises a stem and at least one branch (This element does not integrate the abstract idea into a practical application because it is further modifying the architecture of the neural network made up of the received plurality of blocks defined in claim 7 and thus recites the insignificant extra-solution activity of receiving data.))
and the plurality of hyperparameters of the neural network comprises at least one of the following: a number of blocks of the stem, a number of channels of the stem, a number of the at least one branch, a number of blocks of each of the at least one branch, a number of channels of each of the at least one branch, and a position of the plurality of reduction blocks of each of the at least one branch (This element does not integrate the abstract idea into a practical application because it is further modifying the obtained plurality of hyperparameters defined in claim 7 and thus recites the insignificant extra-solution activity of receiving data.))
Therefore, claim 9 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract
idea itself, taken alone and in combination because
wherein the architecture of the neural network comprises a stem and at least one branch is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
and the plurality of hyperparameters of the neural network comprises at least one of the following: a number of blocks of the stem, a number of channels of the stem, a number of the at least one branch, a number of blocks of each of the at least one branch, a number of channels of each of the at least one branch, and a position of the plurality of reduction blocks of each of the at least one branch is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 10 recites the same abstract idea as claim 9. Therefore, claim 10 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of:
wherein the number of blocks of each of the at least one branch is at least one (This element does not integrate the abstract idea into a practical application because it is further modifying the number of blocks defined in claim 9 and thus recites the insignificant extra-solution activity of receiving data.))
Therefore, claim 10 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract
idea itself, taken alone and in combination because
wherein the number of blocks of each of the at least one branch is at least one is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 11 recites:
wherein the hyperparameter optimization algorithm comprises at least one of the following: Tree-Structured Parzen Estimation, Bayesian Optimization, Grid Search, Random Optimization (This element is a mental process as it is further modifying the hyperparameter optimization algorithm used to generate the hyperparameter combination defined in claim 7.)
Therefore, claim 11 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 11 does not further recite any additional elements. Therefore, claim 11 is not
integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 11 does not provide significantly more than
the abstract idea itself, taken alone and in combination. Therefore, claim 11 is subject-matter
ineligible.
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 12 recites the same abstract idea as claim 7. Therefore, claim 12 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 12 further recites additional elements of:
wherein the model performance comprises at least one of the following: model accuracy, false negative rate (FNR), true negative rate (TNR), parameter size, and inference speed (This element does not integrate the abstract idea into a practical application because it is further modifying the model performance evaluated by the neural network module defined in claim 7 and thus recites generic computing components on which to perform the abstract idea.)
Therefore, claim 12 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 12 do not provide significantly more than the abstract
idea itself, taken alone and in combination because
wherein the model performance comprises at least one of the following: model accuracy, false negative rate (FNR), true negative rate (TNR), parameter size, and inference speed uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 12 is subject-matter ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dutta et al. (US20190188537).
Regarding claim 1, Dutta teaches
A multi-branch network architecture searching method, applicable to an electronic device comprising a processor, comprising: obtaining a training dataset, wherein the training dataset includes a plurality of input data (Dutta, paragraph 0006, “The method comprises: storing, in a memory, a training dataset and a validation dataset related to the particular task, the training dataset and the validation dataset each including a plurality of labeled input and output data pairs.”)
obtaining a plurality of block design elements for a plurality of blocks, wherein the plurality of blocks forms an architecture of a neural network (Dutta, paragraph 0023, “The result of the training process is embodied in a plurality of learned parameters, kernel weights, and/or filter values that are used in the various layers of neural network that comprises the machine learning model to perform various operations or functions.” Examiner notes the blocks are the layers and the block design elements are the filter values.)
and the plurality of blocks are configured to perform a feature extraction on the plurality of input data to generate an output data (Dutta, paragraph 0036, “In at least one embodiment, the deep CNN 400 begins with a sequence of m initial convolution filter layers 404. As mentioned above, it will be appreciated that a convolutional layer generally acts as learnable filter that responds to certain features when convolved with an input, producing a filtered output.” Dutta, paragraph 0036, “The initial convolution filter layers 404 receive input data and perform convolution and/or filtering operation on the input to provide filtered and/or processed data as an intermediate output.”)
for each of a plurality of hyperparameters of the neural network, obtaining at least one hyperparameter setting value (Dutta, paragraph 0050, “Additionally, the processor 14 is configured to store in the memory 16 in association with the value(s) of the evaluated performance metric(s) the hyperparameters used, including the selected set of operations that were used in place of the undefined operations of the building block 100, the values selected for k, n, m, and any other hyperparameter.” Examiner notes the hyperparameter setting values are the values selected for each hyperparameter.)
inputting the training dataset, the plurality of block design elements, and the at least one hyperparameter setting value into a hyperparameter optimization algorithm to generate a hyperparameter combination (Dutta, paragraph 0049, “In the training process, the processor 14 is configured to determine optimized values for the parameters, kernels, and/or weights of at least the convolutional layers of the deep CNN 400 using the training dataset 32. In at least one embodiment, the processor 14 is configured to determine the optimized values by minimizing a loss function evaluates an output of the deep CNN 400 with respect to the correct output identified by the labeled training data in the training dataset 32.” Examiner notes the hyperparameter optimization algorithm is the minimizing of a loss function and the hyperparameter combination are the optimized hyperparameter values.)
wherein the hyperparameter combination includes one of the at least one hyperparameter setting value corresponding to each of the plurality of hyperparameters (Dutta, paragraph 0050, “Additionally, the processor 14 is configured to store in the memory 16 in association with the value(s) of the evaluated performance metric(s) the hyperparameters used, including the selected set of operations that were used in place of the undefined operations of the building block 100, the values selected for k, n, m, and any other hyperparameter.” Examiner notes the hyperparameters that are stored in the memory includes the values selected for each hyperparameter, which are the hyperparameter setting values corresponding to each hyperparameter. Examiner further notes the hyperparameter combination comprises the selected values from hyperparameter optimization.)
executing the neural network based on the hyperparameter combination and inputting a test dataset to evaluate a model performance of the neural network (Dutta, paragraph 0049, “In the training process, the processor 14 is configured to determine optimized values for the parameters, kernels, and/or weights of at least the convolutional layers of the deep CNN 400 using the training dataset 32. In at least one embodiment, the processor 14 is configured to determine the optimized values by minimizing a loss function evaluates an output of the deep CNN 400 with respect to the correct output identified by the labeled training data in the training dataset 32.” Dutta, paragraph 0050, “The method 500 continues with a step of evaluating, using the validation dataset, at least one performance metric of the trained deep convolutional neural network (block 590). Particularly, after the deep CNN 400 is trained using the selected set of operations in place of the undefined operations of the building block 100, the processor 14 is configured to evaluate at least one performance metric of trained deep CNN 400 using the validation dataset 32.” Examiner notes the trained neural network consists of the neural network using the optimized hyperparameters, and the test dataset is the validation dataset.)
and outputting the hyperparameter combination when the model performance reaches a threshold. (Dutta, paragraph 0051, “In other embodiments, the processor 14 is configured to end the search process after the evaluated at least one performance metric reaches a predetermined threshold performance. In at least on embodiment, the processor is configured to compared the stored values of the performance metrics to determine which selected set of operations provided the highest performance of all of the trials. In at least on embodiment, the processor 14 is configured to output the selected set of operations corresponding to the trial having a highest performance according to the stored values of the performance metrics.” Dutta, paragraph 0046, “Particularly, the method 500 treats the choice of operations within the building blocks 404 as hyperparameters.” Examiner notes the output hyperparameter combination is the output selected set of operations, which comprises the hyperparameters which were optimized to produce the highest performance.)
Regarding claim 2, Dutta teaches
The multi-branch network architecture searching method of claim 1, wherein the plurality of blocks comprises a plurality of normal blocks and a plurality of reduction blocks (Dutta, paragraph 0048, “the deep CNN 400 is formed with various pooling layer(s) 412, fully connected layer(s) 416, batch normalizations, activation functions, and dropout layers, as discussed above.” Examiner notes the reduction blocks are the pooling layers and the normal blocks are the non-pooling layers.)
each of the plurality of normal blocks is configured to preserve a dimension of one of the plurality of input data (Dutta, paragraph 0037, “In the deep CNN 400, the initial convolution filter layers 404 are followed by a sequence of n building blocks 408. Each of the building blocks 408 include the same structure as the building block 100 and utilize the same selected operations in place of the undefined operations (the selection process is described in detail elsewhere herein), with the same selected filter dimensions k. Much like the convolution filter layers 404, the convolutional layers of each of the building blocks 408 utilize a plurality of convolutional filter parameters, the values of which are learned and/or optimized during a training process.”)
and each of the plurality of reduction blocks is configured to reduce the dimension of one of the plurality of input data (Dutta, paragraph 0038, “In some embodiments, the deep neural network 400 includes a pooling layer 412 configured to pool and/or reduce the dimensionality of the output data provided by the building blocks 408 with a predetermined pool size. In one embodiment, the pooling layer 412 is an average pooling layer or a max pooling layer. In some embodiments, additional pooling layers may be include after various ones of the initial convolution filter layers 404 or after various ones of the building blocks 408.”)
Regarding claim 3, Dutta teaches
The multi-branch network architecture searching method of claim 2, wherein the architecture of the neural network comprises a stem and at least one branch (Dutta, FIG. 5. Dutta, paragraph 0036, “FIG. 5 shows a deep convolutional neural network 400 for performing a particular task. In at least one embodiment, the deep CNN 400 begins with a sequence of m initial convolution filter layers 404.” Dutta, paragraph 0037, “In the deep CNN 400, the initial convolution filter layers 404 are followed by a sequence of n building blocks 408. Each of the building blocks 408 include the same structure as the building block 100.” Dutta, paragraph 0024, “The building block 100 takes the form of a residual network (which may be referred to elsewhere herein as a residual block) comprising a residual branch 104, a skip connection 108 (which may be referred to elsewhere herein as a feedforward branch), and a summation element 112.” Examiner notes the stem comprises the initial convolution filter layers 404 and the branch is the residual branch.)
and the plurality of hyperparameters of the neural network comprises at least one of the following: a number of blocks of the stem, a number of channels of the stem, a number of the at least one branch, a number of blocks of each of the at least one branch, a number of channels of each of the at least one branch, and a position of the plurality of reduction blocks of each of the at least one branch (Dutta, paragraph 0036, “In at least one embodiment, the number of initial convolution filter layers 404 m included in the deep CNN 400 is treated as a hyperparameter which can be predefined or optimized by performing several trials.” Examiner notes one hyperparameter is the number of blocks of the stem, wherein the stem comprises the initial convolution filter layers and the blocks of the stem are each convolutional filter layer of the initial layers.)
Regarding claim 4, Dutta teaches
The multi-branch network architecture searching method of claim 3, wherein the number of the blocks of each of the at least one branch is at least one (Dutta, paragraph 0006, “the residual branch including at least one layer configured to perform at least one undefined operation to provide the output of the residual branch.” Examiner notes the blocks are the layers of each branch.)
Regarding claim 5, Dutta teaches
The multi-branch network architecture searching method of claim 1, wherein the hyperparameter optimization algorithm comprises at least one of the following: Tree- Structured Parzen Estimation, Bayesian Optimization, Grid Search, Random Optimization, Sequential Model-Based Algorithm Configuration, and Metis (Dutta, paragraph 0047, “It will be appreciated that, as the method 500 treats the selection of operations in the residual branch 104 of the building block 100 and/or 404 as hyperparameters, many other optimization methods can be used. However, random search provides the simplest methods for hyperparameter optimization. Compared to iterating over predefined parameter combinations (i.e., grid search), random search shows a good balance between exploration and exploitation, and thus better convergence rate. It is also less sensitive to the prior assumptions on the distribution of hyperparameters which makes it a more robust alternative when applied to many different problems. In addition, random search is naively parallelizable as there is no dependency on historical results.” Examiner notes the hyperparameter optimization algorithm is random optimization.)
Regarding claim 6, Dutta teaches
The multi-branch network architecture searching method of claim 1, wherein the model performance comprises at least one of the following: model accuracy, false negative rate (FNR), true negative rate (TNR), parameter size, and inference speed (Dutta, paragraph 0050, “The method 500 continues with a step of evaluating, using the validation dataset, at least one performance metric of the trained deep convolutional neural network (block 590). Particularly, after the deep CNN 400 is trained using the selected set of operations in place of the undefined operations of the building block 100, the processor 14 is configured to evaluate at least one performance metric of trained deep CNN 400 using the validation dataset 32. It at least one embodiment, the processor 14 is configured to determine an accuracy of the trained deep CNN 400 in providing the correct output corresponding to a given input from the validation dataset 32.” Examiner notes the performance metric is model accuracy.)
Regarding claim 7, Dutta teaches
A multi-branch network architecture searching system applicable to an electronic device comprising a processor, comprising (Dutta, paragraph 0007, “A system for determining a structure of a deep convolutional neural network for performing a particular task is disclosed. The system comprises a data storage device and at least one processor operably connected to the data storage device.”)
an input module configured to obtain a training dataset (Dutta, paragraph 0006, “The method comprises: storing, in a memory, a training dataset and a validation dataset related to the particular task, the training dataset and the validation dataset each including a plurality of labeled input and output data pairs.”)
a plurality of block design elements of a plurality of blocks (Dutta, paragraph 0023, “The result of the training process is embodied in a plurality of learned parameters, kernel weights, and/or filter values that are used in the various layers of neural network that comprises the machine learning model to perform various operations or functions.” Examiner notes the blocks are the layers and the block design elements are the filter values.)
at least one hyperparameter setting value of each of a plurality of hyperparameters of a neural network (Dutta, paragraph 0050, “Additionally, the processor 14 is configured to store in the memory 16 in association with the value(s) of the evaluated performance metric(s) the hyperparameters used, including the selected set of operations that were used in place of the undefined operations of the building block 100, the values selected for k, n, m, and any other hyperparameter.” Examiner notes the hyperparameter setting values are the values selected for each hyperparameter.)
and a test dataset (Dutta, paragraph 0006, “The method comprises: storing, in a memory, a training dataset and a validation dataset related to the particular task, the training dataset and the validation dataset each including a plurality of labeled input and output data pairs.” Examiner notes the test dataset is the validation dataset.)
wherein the training dataset includes a plurality of input data (Dutta, paragraph 0006, “The method comprises: storing, in a memory, a training dataset and a validation dataset related to the particular task, the training dataset and the validation dataset each including a plurality of labeled input and output data pairs.”)
the plurality of blocks forms an architecture of the neural network (Dutta, paragraph 0023, “The result of the training process is embodied in a plurality of learned parameters, kernel weights, and/or filter values that are used in the various layers of neural network that comprises the machine learning model to perform various operations or functions.” Examiner notes the blocks are the layers.)
the plurality of blocks is configured to perform a feature extraction on one of the plurality of input data to generate an output data (Dutta, paragraph 0036, “In at least one embodiment, the deep CNN 400 begins with a sequence of m initial convolution filter layers 404. As mentioned above, it will be appreciated that a convolutional layer generally acts as learnable filter that responds to certain features when convolved with an input, producing a filtered output.” Dutta, paragraph 0036, “The initial convolution filter layers 404 receive input data and perform convolution and/or filtering operation on the input to provide filtered and/or processed data as an intermediate output.”)
a computing module communicably connected to the input module, wherein the computing module is configured to execute a hyperparameter optimization algorithm according to the training dataset, the plurality of block design elements, and the at least one hyperparameter setting value to generate a hyperparameter combination (Dutta, paragraph 0049, “In the training process, the processor 14 is configured to determine optimized values for the parameters, kernels, and/or weights of at least the convolutional layers of the deep CNN 400 using the training dataset 32. In at least one embodiment, the processor 14 is configured to determine the optimized values by minimizing a loss function evaluates an output of the deep CNN 400 with respect to the correct output identified by the labeled training data in the training dataset 32.” Examiner notes the hyperparameter optimization algorithm is the minimizing of a loss function and the hyperparameter combination is the optimized hyperparameter values.)
the hyperparameter combination comprises one of the at least one hyperparameter setting value corresponding to each of the plurality of hyperparameters (Dutta, paragraph 0050, “Additionally, the processor 14 is configured to store in the memory 16 in association with the value(s) of the evaluated performance metric(s) the hyperparameters used, including the selected set of operations that were used in place of the undefined operations of the building block 100, the values selected for k, n, m, and any other hyperparameter.” Examiner notes the hyperparameters that are stored in the memory includes the values selected for each hyperparameter, which are the hyperparameter setting values corresponding to each hyperparameter. Examiner further notes the hyperparameter combination comprises the selected values from hyperparameter optimization.)
a neural network module communicably connected to the input module and the computing module, wherein the neural network module is configured to execute a neural network based on the hyperparameter combination and input the test dataset to evaluate a model performance of the neural network (Dutta, paragraph 0049, “In the training process, the processor 14 is configured to determine optimized values for the parameters, kernels, and/or weights of at least the convolutional layers of the deep CNN 400 using the training dataset 32. In at least one embodiment, the processor 14 is configured to determine the optimized values by minimizing a loss function evaluates an output of the deep CNN 400 with respect to the correct output identified by the labeled training data in the training dataset 32.” Dutta, paragraph 0050, “The method 500 continues with a step of evaluating, using the validation dataset, at least one performance metric of the trained deep convolutional neural network (block 590). Particularly, after the deep CNN 400 is trained using the selected set of operations in place of the undefined operations of the building block 100, the processor 14 is configured to evaluate at least one performance metric of trained deep CNN 400 using the validation dataset 32.” Examiner notes the trained neural network consists of the neural network using the optimized hyperparameters, and the test dataset is the validation dataset.)
and the computing module is further configured to output the hyperparameter combination when the model performance reaches a threshold (Dutta, paragraph 0051, “In other embodiments, the processor 14 is configured to end the search process after the evaluated at least one performance metric reaches a predetermined threshold performance. In at least on embodiment, the processor is configured to compared the stored values of the performance metrics to determine which selected set of operations provided the highest performance of all of the trials. In at least on embodiment, the processor 14 is configured to output the selected set of operations corresponding to the trial having a highest performance according to the stored values of the performance metrics.” Dutta, paragraph 0046, “Particularly, the method 500 treats the choice of operations within the building blocks 404 as hyperparameters.” Examiner notes the output hyperparameter combination is the output selected set of operations, which comprises the hyperparameters which were optimized to produce the highest performance.)
Regarding claim 8, Dutta teaches
The multi-branch network architecture searching system of claim 7, wherein the plurality of blocks comprises a plurality of normal blocks and a plurality of reduction blocks (Dutta, paragraph 0048, “the deep CNN 400 is formed with various pooling layer(s) 412, fully connected layer(s) 416, batch normalizations, activation functions, and dropout layers, as discussed above.” Examiner notes the reduction blocks are the pooling layers and the normal blocks are the non-pooling layers.)
each of the plurality of normal blocks is configured to preserve a dimension of one of the plurality of input data (Dutta, paragraph 0037, “In the deep CNN 400, the initial convolution filter layers 404 are followed by a sequence of n building blocks 408. Each of the building blocks 408 include the same structure as the building block 100 and utilize the same selected operations in place of the undefined operations (the selection process is described in detail elsewhere herein), with the same selected filter dimensions k. Much like the convolution filter layers 404, the convolutional layers of each of the building blocks 408 utilize a plurality of convolutional filter parameters, the values of which are learned and/or optimized during a training process.”)
and each of the plurality of reduction blocks is configured to reduce the dimension of the plurality of input data (Dutta, paragraph 0038, “In some embodiments, the deep neural network 400 includes a pooling layer 412 configured to pool and/or reduce the dimensionality of the output data provided by the building blocks 408 with a predetermined pool size. In one embodiment, the pooling layer 412 is an average pooling layer or a max pooling layer. In some embodiments, additional pooling layers may be include after various ones of the initial convolution filter layers 404 or after various ones of the building blocks 408.”)
Regarding claim 9, Dutta teaches
The multi-branch network architecture searching system of claim 8, wherein the architecture of the neural network comprises a stem and at least one branch (Dutta, FIG. 5. Dutta, paragraph 0036, “FIG. 5 shows a deep convolutional neural network 400 for performing a particular task. In at least one embodiment, the deep CNN 400 begins with a sequence of m initial convolution filter layers 404.” Dutta, paragraph 0037, “In the deep CNN 400, the initial convolution filter layers 404 are followed by a sequence of n building blocks 408. Each of the building blocks 408 include the same structure as the building block 100.” Dutta, paragraph 0024, “The building block 100 takes the form of a residual network (which may be referred to elsewhere herein as a residual block) comprising a residual branch 104, a skip connection 108 (which may be referred to elsewhere herein as a feedforward branch), and a summation element 112.” Examiner notes the stem comprises the initial convolution filter layers 404 and the branch is the residual branch.)
and the plurality of hyperparameters of the neural network comprises at least one of the following: a number of blocks of the stem, a number of channels of the stem, a number of the at least one branch, a number of blocks of each of the at least one branch, a number of channels of each of the at least one branch, and a position of the plurality of reduction blocks of each of the at least one branch (Dutta, paragraph 0036, “In at least one embodiment, the number of initial convolution filter layers 404 m included in the deep CNN 400 is treated as a hyperparameter which can be predefined or optimized by performing several trials.” Examiner notes one hyperparameter is the number of blocks of the stem, wherein the stem comprises the initial convolution filter layers and the blocks of the stem are each convolutional filter layer of the initial layers.)
Regarding claim 10, Dutta teaches
The multi-branch network architecture searching system of claim 9, wherein the number of blocks of each of the at least one branch is at least one (Dutta, paragraph 0006, “the residual branch including at least one layer configured to perform at least one undefined operation to provide the output of the residual branch.” Examiner notes the blocks are the layers of each branch.)
Regarding claim 11, Dutta teaches
The multi-branch network architecture searching system of claim 7, wherein the hyperparameter optimization algorithm comprises at least one of the following: Tree- Structured Parzen Estimation, Bayesian Optimization, Grid Search, Random Optimization, Sequential Model-Based Algorithm Configuration, and Metis (Dutta, paragraph 0047, “It will be appreciated that, as the method 500 treats the selection of operations in the residual branch 104 of the building block 100 and/or 404 as hyperparameters, many other optimization methods can be used. However, random search provides the simplest methods for hyperparameter optimization. Compared to iterating over predefined parameter combinations (i.e., grid search), random search shows a good balance between exploration and exploitation, and thus better convergence rate. It is also less sensitive to the prior assumptions on the distribution of hyperparameters which makes it a more robust alternative when applied to many different problems. In addition, random search is naively parallelizable as there is no dependency on historical results.” Examiner notes the hyperparameter optimization algorithm is random optimization.)
Regarding claim 12, Dutta teaches
The multi-branch network architecture searching system of claim 7, wherein the model performance comprises at least one of the following: model accuracy, false negative rate (FNR), true negative rate (TNR), parameter size, and inference speed (Dutta, paragraph 0050, “The method 500 continues with a step of evaluating, using the validation dataset, at least one performance metric of the trained deep convolutional neural network (block 590). Particularly, after the deep CNN 400 is trained using the selected set of operations in place of the undefined operations of the building block 100, the processor 14 is configured to evaluate at least one performance metric of trained deep CNN 400 using the validation dataset 32. It at least one embodiment, the processor 14 is configured to determine an accuracy of the trained deep CNN 400 in providing the correct output corresponding to a given input from the validation dataset 32.” Examiner notes the performance metric is model accuracy.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Shen et al. (US-20240303492-A1) also teaches optimizing hyperparameters in a neural network until a performance threshold is met.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Zared O. Cohen whose telephone number is (571)270-0531. The examiner can normally be reached M-Th, 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Z.O.C./ Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights