Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
The Examiner encourages Applicant to schedule an interview to discuss issues related to, for example, the rejections noted below under 35 U.S.C § 101 and § 103.
Providing supporting paragraph(s) for each limitation of amended/new claim(s) in Remarks is strongly requested for clear and definite claim interpretations by Examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No CN202010661477.X, filed on 07/10/2020.
Claim Objections
Claim(s) 8, 13-20 is/are objected to because of the following informalities.
Claim(s) 8 is/are objected to because of the following informalities:
it appears that “based relationships between” (line 3) needs to read “based on relationships between” or something else. Appropriate correction is required. In addition, claim(s) 20 is/are objected to for the same reason.
it appears that “the one or more discarded operators” (line 4) needs to read “the one or more of the plurality of discarded operators” or something else. Appropriate correction is required. In addition, claim(s) 20 is/are objected to for the same reason.
Claim(s) 13 is/are objected to because of the following informalities: it appears that “the apparatus” (line 4) needs to read “the neural network building apparatus” or something else. Appropriate correction is required.
Claim(s) 8, 13, 20 each recite(s) limitations that raise issues of indefiniteness as set forth above, and their dependent claims are objected to at least based on their direct and/or indirect dependency from the claims listed above. Appropriate explanation and/or amendment is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 7, 19 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim(s) 7 recite(s) the limitation “the operator” (line 4). There is insufficient antecedent basis for this limitation in the claim. It is not clear which operator it is referring to among “the operators that are not discarded” (line 2). It appears that it may need to read “an operator” or something else. For the purposes of examination, “an operator” is used. In addition, claim(s) 19 is/are rejected for the same reason.
Claim(s) 7, 19 each recite(s) limitations that raise issues of indefiniteness as set forth above, and their dependent claims are rejected at least based on their direct and/or indirect dependency from the claims listed above. Appropriate explanation and/or amendment is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“A neural network building method, comprising:
initializing a search space and a plurality of building blocks, …;
in at least one training round, randomly discarding one or more of the plurality of operators, and updating the plurality of building blocks by using operators that are not discarded; and
…”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the search space comprises a plurality of operators, and wherein the plurality of building blocks constitute a network structure obtained by connecting a plurality of nodes by using the plurality of operators”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
In particular, the claim recites an additional element(s) (“building a target neural network based on the plurality of updated building blocks”). The additional element is recited at such a high level without any details as to how a model is generated such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
The additional elements regarding training are recited at such a high level without any details as to how a model is generated such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
Regarding claim 2
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“building the target neural network based on the plurality of updated building blocks obtained in a last training round”). The additional element is recited at such a high level without any details as to how a model is generated such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional elements regarding training are recited at such a high level without any details as to how a model is generated such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
Regarding claim 3
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“grouping the plurality of operators into a plurality of operator groups based on types of the plurality of operators, wherein
during the random discarding, each of the plurality of operator groups reserves at least one operator”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim does not recite additional elements. Thus, the claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Regarding claim 4
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 3.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the plurality of operator groups have different discard rates, and wherein each of the discard rates indicates a probability that each type of operator in the plurality of operator groups is discarded”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 5
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“wherein the plurality of operator groups are determined based on a quantity of parameters comprised in each type of operator in the plurality of operators”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim does not recite additional elements. Thus, the claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Regarding claim 6
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 3.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the plurality of operator groups comprise a first operator group and a second operator group, wherein none of operators in the first operator group comprises a parameter, and each operator in the second operator group comprises a parameter”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 7
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“when the plurality of building blocks are updated, performing weight attenuation only on a parameter comprised in the operator that is not discarded”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim does not recite additional elements. Thus, the claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Regarding claim 8
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“adjusting architecture parameters of the plurality of updated building blocks based relationships between the one or more discarded operators and the operators that are not discarded”). The additional element is recited at such a high level without any details as to how architecture parameters are adjusted such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. (Note that par 246 states “In the initialized supernetwork, the architecture parameter α is optimized by an Adam optimizer, and the learning rate is set to 0.0003.”)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional elements regarding training are recited at such a high level without any details as to how architecture parameters are adjusted such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
Regarding claim 9
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the plurality of operators comprise at least one of the following: skip connection, average pooling, maximum pooling, separable convolution, dilated separable convolution, or a zero operation”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 10
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“…; and
… classify an image”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“obtaining an image classification training sample”) – the act of receiving data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of receiving data is recited at a high-level of generality (i.e., as a generic act of receiving performing a generic act function of receiving data) such that it amounts no more than a mere act to apply the exception using a generic act of receiving. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
In particular, the claim recites an additional element(s) (“training the target neural network based on the image classification training sample, to obtain an image classification model”). The additional element is recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the image classification model is used to”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of receiving data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
The additional elements regarding training are recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 11
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“…; and
… detect a target from a to-be-processed image”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“obtaining a target detection training sample”) – the act of receiving data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of receiving data is recited at a high-level of generality (i.e., as a generic act of receiving performing a generic act function of receiving data) such that it amounts no more than a mere act to apply the exception using a generic act of receiving. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
In particular, the claim recites an additional element(s) (“training the target neural network based on the target detection training sample, to obtain a target detection model”). The additional element is recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the target detection model is used to”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of receiving data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
The additional elements regarding training are recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 12
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 11.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the target comprises at least one of the following: a vehicle, a pedestrian, an obstacle, a road sign, or a traffic sign”). This is a recitation of a particular type or source of data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 13
The claim recites “A neural network building apparatus, comprising: at least one processor; and one or more memories coupled to the at least one processor and storing program instructions for execution by the at least one processor to cause the apparatus to perform following operations:” to perform precisely the method of Claim 1. As performance of an abstract idea on generic computer components (see MPEP 2106.05(f)) and “Storing and retrieving information in memory” (see MPEP 2106.05(g) on Insignificant Extra-Solution Activity, and MPEP 2106.05(d) on Well-Understood, Routine, Conventional Activity) cannot integrate the abstract idea into a practical application nor provide significantly more than the abstract idea itself, the claim is rejected for reasons set forth in the rejection of Claim 1.
Regarding claim 14
The claim is rejected for the reasons set forth in the rejection of Claim 2 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 15
The claim is rejected for the reasons set forth in the rejection of Claim 3 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 16
The claim is rejected for the reasons set forth in the rejection of Claim 4 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 17
The claim is rejected for the reasons set forth in the rejection of Claim 5 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 18
The claim is rejected for the reasons set forth in the rejection of Claim 6 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 19
The claim is rejected for the reasons set forth in the rejection of Claim 7 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Regarding claim 20
The claim is rejected for the reasons set forth in the rejection of Claim 8 under 35 U.S.C. 101, mutatis mutandis, as reciting an abstract idea without integrating the judicial exception into a practical application nor providing significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 7-9, 13-14, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (PC-DARTS: PARTIAL CHANNEL CONNECTIONS FOR MEMORY-EFFICIENT ARCHITECTURE SEARCH) in view of Bi et al. (GOLD-NAS: Gradual, One-Level, Differentiable)
Regarding claim 1
Xu teaches
A neural network building method, comprising:
(Xu [fig(s) 1] [sec(s) 4] “We perform experiments on CIFAR10 and ImageNet, two most popular datasets for evaluating neural architecture search.”;)
initializing a search space and a plurality of building blocks, wherein the search space comprises a plurality of operators, and wherein the plurality of building blocks constitute a network structure obtained by connecting a plurality of nodes by using the plurality of operators;
(Xu [fig(s) 1] [sec(s) 1] “In this paper, we present a simple yet effective approach named Partially-Connected DARTS (PCDARTS) to reduce the burdens of memory and computation. The core idea is intuitive: instead of sending all channels into the block of operation selection, we randomly sample a subset of them in each step, while bypassing the rest directly in a shortcut. We assume the computation on this subset is a surrogate approximating that on all the channels. Besides the tremendous reduction in memory and computation costs, channel sampling brings another benefit – operation search is regularized and less likely to fall into local optima. However, PC-DARTS incurs a side effect, where the selection of channel connectivity would become unstable as different subsets of channels are sampled across iterations. Thus, we introduce edge normalization to stabilize the search for network connectivity by explicitly learning an extra set of edge-selection hyper-parameters. By sharing these hyper-parameters throughout the training process, the sought network architecture is insensitive to the sampled channels across iterations and thus is more stable.” [sec(s) 3.1] “DARTS decomposes the searched network into a number (L) of cells. Each cell is represented as a directed acyclic graph (DAG) with N nodes, where each node defines a network layer. There is a pre-defined space of operations denoted by O, in which each element, o(·), is a fixed operation (e.g., identity connection, and 3 × 3 convolution) performed at a network layer. Within a cell, the goal is to choose one operation from O to connect each pair of nodes.” [sec(s) 3.4] “In comparison, our approach preserves all operators and instead performs sub-sampling on the channel dimension. This strategy works better in particular on large-scale datasets like ImageNet.” [sec(s) 4.2] “In the search scenario, the over-parameterized network is constructed by stacking 8 cells (6 normal cells and 2 reduction cells), and each cell consists of N = 6 nodes. We train the network for 50 epochs, with the initial number of channels being 16.” [sec(s) 4.4] “we go one step further by enlarging the search space, allowing a larger number of nodes to appear in each cell – the original DARTS-based space has 6 nodes, and here we allow 5, 6 and 7 nodes. From 5 to 6 nodes, the performance of all three algorithms goes up, while from 6 to 7 nodes, DARTS-v2 suffers a significant accuracy drop, while PC-DARTS mostly preserves it performance.”;)
in at least one training round, [randomly discarding] one or more of the plurality of operators, and updating the plurality of building blocks by using operators that are not discarded; and
(Xu [fig(s) 1] [sec(s) 1] “The core idea is intuitive: instead of sending all channels into the block of operation selection, we randomly sample a subset of them in each step, while bypassing the rest directly in a shortcut. We assume the computation on this subset is a surrogate approximating that on all the channels. Besides the tremendous reduction in memory and computation costs, channel sampling brings another benefit – operation search is regularized and less likely to fall into local optima. However, PC-DARTS incurs a side effect, where the selection of channel connectivity would become unstable as different subsets of channels are sampled across iterations. Thus, we introduce edge normalization to stabilize the search for network connectivity by explicitly learning an extra set of edge-selection hyper-parameters. By sharing these hyper-parameters throughout the training process, the sought network architecture is insensitive to the sampled channels across iterations and thus is more stable.” [sec(s) 3.1] “There is a pre-defined space of operations denoted by O, in which each element, o(·), is a fixed operation (e.g., identity connection, and 3 × 3 convolution) performed at a network layer. Within a cell, the goal is to choose one operation from O to connect each pair of nodes.” [sec(s) 4.1] “The entire search stage is accomplished in an end-to-end manner. For fair comparison, the operation space O remains the same as the convention, which contains 8 choices, i.e., 3×3 and 5×5 separable convolution, 3×3 and 5×5 dilated separable convolution, 3×3 max-pooling, 3×3 average-pooling, skip-connect (a.k.a., identity), and zero (a.k.a., none).”;)
building a target neural network based on the plurality of updated building blocks.
(Xu [fig(s) 1] [sec(s) 1] “The core idea is intuitive: instead of sending all channels into the block of operation selection, we randomly sample a subset of them in each step, while bypassing the rest directly in a shortcut. We assume the computation on this subset is a surrogate approximating that on all the channels. Besides the tremendous reduction in memory and computation costs, channel sampling brings another benefit – operation search is regularized and less likely to fall into local optima. However, PC-DARTS incurs a side effect, where the selection of channel connectivity would become unstable as different subsets of channels are sampled across iterations. Thus, we introduce edge normalization to stabilize the search for network connectivity by explicitly learning an extra set of edge-selection hyper-parameters. By sharing these hyper-parameters throughout the training process, the sought network architecture is insensitive to the sampled channels across iterations and thus is more stable.” [sec(s) 3.1] “There is a pre-defined space of operations denoted by O, in which each element, o(·), is a fixed operation (e.g., identity connection, and 3 × 3 convolution) performed at a network layer. Within a cell, the goal is to choose one operation from O to connect each pair of nodes.” [sec(s) 5] “This research delivers two important messages that are important for future research. First, differentiable architecture search seems to suffer even more significant instability compared to conventional neural network training, and so it can largely benefit from both (i) regularization and (ii) a larger batch size. This work shows an efficient way to incorporate these two factors in a single pipeline.”;)
However, Xu does not appear to explicitly teach:
in at least one training round, [randomly discarding] one or more of the plurality of operators, and updating the plurality of building blocks by using operators that are not discarded; and
Bi teaches
in at least one training round, randomly discarding one or more of the plurality of operators, and updating the plurality of building blocks by using operators that are not discarded; and
(Bi [sec(s) B] “To produce the random search baseline, we randomly prune out operators from the super-network until the architecture fits the hardware constraint (e.g., FLOPs). It is possible that the architecture becomes invalid during the random pruning process, and we discard such architectures. Each random search process collects 24 architectures and we train each of them for 100 epochs and pick up the 13 best one for an entire 600-epoch re-training. As reported in the paper, we perform random search three times and the best architecture reports an average accuracy of 3.31 ± 0.50%.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Xu with the random operator discarding of Bi.
One of ordinary skill in the art would have been motived to combine in order to provide a flexible interface for injecting different kinds of resource constraints, and save the efforts and computational costs of developers.
(Bi [sec(s) 5] “GOLD-NAS provides a flexible interface for injecting different kinds of resource constraints. It also enjoys an intriguing ability of producing a Pareto-front during one search procedure. This can save the efforts and computational costs of developers.”)
Regarding claim 2
The combination of Xu, Bi teaches claim 1.
wherein the building of the target neural network based on the plurality of updated building blocks comprises: (See claim 1)
Xu further teaches
building the target neural network based on the plurality of updated building blocks obtained in a last training round.
(Xu [fig(s) 1] [sec(s) 1] “The core idea is intuitive: instead of sending all channels into the block of operation selection, we randomly sample a subset of them in each step, while bypassing the rest directly in a shortcut. We assume the computation on this subset is a surrogate approximating that on all the channels. Besides the tremendous reduction in memory and computation costs, channel sampling brings another benefit – operation search is regularized and less likely to fall into local optima. However, PC-DARTS incurs a side effect, where the selection of channel connectivity would become unstable as different subsets of channels are sampled across iterations. Thus, we introduce edge normalization to stabilize the search for network connectivity by explicitly learning an extra set of edge-selection hyper-parameters. By sharing these hyper-parameters throughout the training process, the sought network architecture is insensitive to the sampled channels across iterations and thus is more stable.” [sec(s) 3.1] “There is a pre-defined space of operations denoted by O, in which each element, o(·), is a fixed operation (e.g., identity connection, and 3 × 3 convolution) performed at a network layer. Within a cell, the goal is to choose one operation from O to connect each pair of nodes.” [sec(s) 5] “This research delivers two important messages that are important for future research. First, differentiable architecture search seems to suffer even more significant instability compared to conventional neural network training, and so it can largely benefit from both (i) regularization and (ii) a larger batch size. This work shows an efficient way to incorporate these two factors in a single pipeline.”;)
Regarding claim 7
The combination of Xu, Bi teaches claim 1.
wherein the updating of the plurality of building blocks by using the operators that are not discarded comprises: (See claim 1)
Bi further teaches
when the plurality of building blocks are updated, performing weight attenuation only on a parameter comprised in the operator that is not discarded.
(Bi [sec(s) B] “B.4 Details of Random Search Experiments To produce the random search baseline, we randomly prune out operators from the super-network until the architecture fits the hardware constraint (e.g., FLOPs). It is possible that the architecture becomes invalid during the random pruning process, and we discard such architectures. Each random search process collects 24 architectures and we train each of them for 100 epochs and pick up the 13 best one for an entire 600-epoch re-training. As reported in the paper, we perform random search three times and the best architecture reports an average accuracy of 3.31 ± 0.50%.” [sec(s) C] “During the re-training process, the total number of epochs is set to be 250. The batch size is set to be 1,024 (eight cards). We use an SGD optimizer with an initial learning rate of 0.5 (decayed linearly after each epoch till 0), a momentum of 0.9 and a weight decay of 3 × 10−5. The search process takes around 3 days on eight NVIDIA Telsa-V100 GPUs.”;
Examiner notes that paragraph 249 of the Instant Specification describes “A momentum method is used for updating, a momentum parameter is set to 0.9, and a weight attenuation coefficient is set to 0.00003.”
Examiner notes that paragraph 22 of the Instant Specification describes “With reference to the first aspect, in some implementations of the first aspect, when the plurality of building block are updated, weight attenuation may be performed only on a parameter included in the operator that is not discarded. This can effectively avoid excessive regularization.”
Examiner notes that paragraph 210 of the Instant Specification describes “For a discarded operator, a weight attenuation operation may not be performed on the operator during weight attenuation, to avoid excessive regularization. In other words, weight attenuation can be performed only on an operator that is not discarded to avoid excessive regularization.”
Examiner notes that paragraph 246 of the Instant Specification describes “For the initialized supernetwork, the following initialization parameters are set: a learning rate is set to 0.0375, a gradient with momentum is used, the momentum is set to 0.9, weight attenuation is set to 0.0003, and a model weight w is optimized and updated by using a stochastic gradient descent method”)
The combination of Xu, Bi is combinable with Bi for the same rationale as set forth above with respect to claim 1.
Regarding claim 8
The combination of Xu, Bi teaches claim 1.
Bi further teaches
wherein the method further comprises:
adjusting architecture parameters of the plurality of updated building blocks based relationships between the one or more discarded operators and the operators that are not discarded.
(Bi [sec(s) B] “B.4 Details of Random Search Experiments To produce the random search baseline, we randomly prune out operators from the super-network until the architecture fits the hardware constraint (e.g., FLOPs). It is possible that the architecture becomes invalid during the random pruning process, and we discard such architectures. Each random search process collects 24 architectures and we train each of them for 100 epochs and pick up the 13 best one for an entire 600-epoch re-training. As reported in the paper, we perform random search three times and the best architecture reports an average accuracy of 3.31 ± 0.50%.” [sec(s) 3.6] “we investigate the performance of random search. Following prior work [19, 22], we individually sample 24 valid architectures from the new space and evaluate the performance in a 100-epoch validation process (for technical details, please refer to Appendix B.4). The best architecture is taken into a standard re-training process. We perform random search three times, each of which takes 4 GPU-days, and report an average error of 3.31 ± 0.50%, number of parameters of 2.30 ± 0.49M, and FLOPs of 368 ± 73M. This is far behind the Pareto fronts shown in Figure 1, indicating the strong ability of GOLD-NAS in finding efficient architectures.”;)
The combination of Xu, Bi is combinable with Bi for the same rationale as set forth above with respect to claim 1.
Regarding claim 9
The combination of Xu, Bi teaches claim 1.
Xu further teaches
wherein the plurality of operators comprise at least one of the following: skip connection, average pooling, maximum pooling, separable convolution, dilated separable convolution, or a zero operation.
(Xu [sec(s) 4.1] “To this end, the trainnig set is partitioned into two parts, with the first part used for optimizing network parameters, e.g., convolutional weights, and the second part used for optimizing hyper-parameters. The entire search stage is accomplished in an end-to-end manner. For fair comparison, the operation space O remains the same as the convention, which contains 8 choices, i.e., 3×3 and 5×5 separable convolution, 3×3 and 5×5 dilated separable convolution, 3×3 max-pooling, 3×3 average-pooling, skip-connect (a.k.a., identity), and zero (a.k.a., none).” [sec(s) 4.4] “This largely owes to the regularization mechanism introduced by PC-DARTS, which (i) forces it to adjust to dynamic architectures, and (ii) avoids the large pruning gap after search, brought by the none operator.”;)
Regarding claim 13
The claim is an apparatus claim corresponding to the method claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 14
The claim is an apparatus claim corresponding to the method claim 2, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 19
The claim is an apparatus claim corresponding to the method claim 7, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 20
The claim is an apparatus claim corresponding to the method claim 8, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Claim(s) 3-4, 6, 15-16, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (PC-DARTS: PARTIAL CHANNEL CONNECTIONS FOR MEMORY-EFFICIENT ARCHITECTURE SEARCH) in view of Bi et al. (GOLD-NAS: Gradual, One-Level, Differentiable) in view of Elsken et al. (EFFICIENT MULTI-OBJECTIVE NEURAL ARCHITECTURE SEARCH VIA LAMARCKIAN EVOLUTION)
Regarding claim 3
The combination of Xu, Bi teaches claim 1.
wherein the randomly discarding one or more of the plurality of operators comprises: (See claim 1)
Xu further teaches
grouping the plurality of operators into a plurality of operator groups based on types of the plurality of operators, wherein
(Xu [sec(s) 4.1] “To this end, the trainnig set is partitioned into two parts, with the first part used for optimizing network parameters, e.g., convolutional weights, and the second part used for optimizing hyper-parameters. The entire search stage is accomplished in an end-to-end manner. For fair comparison, the operation space O remains the same as the convention, which contains 8 choices, i.e., 3×3 and 5×5 separable convolution, 3×3 and 5×5 dilated separable convolution, 3×3 max-pooling, 3×3 average-pooling, skip-connect (a.k.a., identity), and zero (a.k.a., none).” [sec(s) 4.4] “This largely owes to the regularization mechanism introduced by PC-DARTS, which (i) forces it to adjust to dynamic architectures, and (ii) avoids the large pruning gap after search, brought by the none operator.”;)
However, the combination of Xu, Bi does not appear to explicitly teach:
during the random discarding, each of the plurality of operator groups reserves at least one operator.
Elsken teaches
during the random discarding, each of the plurality of operator groups reserves at least one operator.
(Elsken [fig(s) 1] [sec(s) 3] “We now discuss two specific classes of network operators, namely network morphisms and approximate network morphisms. Operators from these two classes will later on serve as mutations in our evolutionary algorithm. … Hence, we now generalize the concept of network morphisms to also cover operators that reduce the capacity of a neural architecture. We say an operator T is an approximate network morphism (ANM) with respect to a neural network Nw with parameters w if Nw(x) ≈ (TN)w˜(x) for every x ∈ X. We refer to Appendix A.1.2 for a formal definition. In practice we simply determine w˜ so that N˜ approximates N by using knowledge distillation (Hinton et al., 2015). In our experiments, we employ the following ANM’s: (i) remove a randomly chosen layer or a skip connection, (ii) prune a randomly chosen convolutional layer (i.e., remove 1/2 or 1/4 of its filters), and (iii) substitute a randomly chosen convolution by a depthwise separable convolution. Note that these operators could easily be extended by sophisticated methods for compressing neural networks (Han et al., 2016; Cheng et al., 2018).” [sec(s) 5.1] “The set of operators to generate child networks we consider in our experiments are the three network morphism operators (insert convolution, insert skip connection, increase number of filters), as well as the three approximate network morphism operators (remove layer, prune filters, replace layer) described in Section 3. The operators are sampled uniformly at random to generate children. The experiment ran for approximately 5 days on 16 GPUs in parallel. The resulting Pareto front consists of approximately 300 neural network architectures.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Xu, Bi with the reserving operator of Elsken.
One of ordinary skill in the art would have been motived to combine in order to speed up the training of novel architectures, and be able to find competitive models and cells both in terms of accuracy and of resource efficiency.
(Elsken [sec(s) 6] “We have proposed LEMONADE, a multi-objective evolutionary algorithm for architecture search. The algorithm employs a Lamarckian inheritance mechanism based on (approximate) network morphism operators to speed up the training of novel architectures. Moreover, LEMONADE exploits the fact that evaluating several objectives, such as the performance of a neural network, is orders of magnitude more expensive than evaluating, e.g., a model’s number of parameters. Experiments on CIFAR-10 and ImageNet64x64 show that LEMONADE is able to find competitive models and cells both in terms of accuracy and of resource efficiency.”)
Regarding claim 4
The combination of Xu, Bi, Elsken teaches claim 3.
Bi further teaches
wherein the plurality of operator groups have different discard rates, and wherein each of the discard rates indicates a probability that each type of operator in the plurality of operator groups is discarded.
(Bi [fig(s) 4] [sec(s) 3.4] “Motivated by this, we propose to add regularization to the process of super-network training so that to penalize the architectures that use more computational resources. This mechanism is similar in a few prior work that incorporated hardware constraints to the search algorithm [3, 33, 34], but the goal of our method is to use the power of regularization to suppress the weight of some operators so that they can be pruned. Note that the risk of discretization error grows as the number and the strength (weights) of pruned operators. So, a safe choice is to perform pruning multiple times, in each of which only the operators with sufficiently low weights can be removed. We elaborate the details in the following subsection.” [sec(s) 3.5] “To satisfy the condition that the weights of pruned operators are sufficiently small, we design a gradual pruning algorithm. The core idea is to start with a low regularization coefficient and increase it gradually during the search procedure. Every time the coefficient becomes larger, there will be some operators (those having higher redundancy) being suppressed to low weights. Pruning them out causes little drop in training accuracy. This process continues till the super-network becomes sufficiently small. During the search process, The architectures that survive for sufficiently long are recorded, which compose the set of Pareto-optimal architectures.” [sec(s) B.2] “In Figure 4, we visualize the search procedures on CIFAR10 using the hyper-parameters of η = 0 and η = 1. We can observe that, as the search procedure goes, weak operators are pruned out from the super-network and the FLOPs of the network gradually goes down. With η = 1, the rate of pruning is much faster. More interestingly, λ, the balancing coefficient, zigzags from a small value to a large value. In each period, λ first goes up to force some operators to have lower weights (during this process, nothing is pruned and the architecture remains unchanged), and then goes down as pruning takes effect to eliminate the weak operators. Each local maximum (just before the pruning stage) corresponds to a Pareto-optimal architecture.”; e.g., “pruning multiple times, in each of which only the operators with sufficiently low weights can be removed” read(s) on “probability that each type of operator in the plurality of operator groups is discarded”.)
The combination of Xu, Bi, Elsken is combinable with Bi for the same rationale as set forth above with respect to claim 1.
Regarding claim 6
The combination of Xu, Bi, Elsken teaches claim 3.
Xu further teaches
wherein the plurality of operator groups comprise a first operator group and a second operator group, wherein none of operators in the first operator group comprises a parameter, and each operator in the second operator group comprises a parameter.
(Xu [sec(s) 4.1] “To this end, the trainnig set is partitioned into two parts, with the first part used for optimizing network parameters, e.g., convolutional weights, and the second part used for optimizing hyper-parameters. The entire search stage is accomplished in an end-to-end manner. For fair comparison, the operation space O remains the same as the convention, which contains 8 choices, i.e., 3×3 and 5×5 separable convolution, 3×3 and 5×5 dilated separable convolution, 3×3 max-pooling, 3×3 average-pooling, skip-connect (a.k.a., identity), and zero (a.k.a., none).” [sec(s) 4.4] “This largely owes to the regularization mechanism introduced by PC-DARTS, which (i) forces it to adjust to dynamic architectures, and (ii) avoids the large pruning gap after search, brought by the none operator.”; e.g., “zero (a.k.a., none)” read(s) on “first operator group”. In addition, e.g., “5×5 separable convolution” read(s) on “second operator group”.)
Regarding claim 15
The claim is an apparatus claim corresponding to the method claim 3, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 16
The claim is an apparatus claim corresponding to the method claim 4, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 18
The claim is an apparatus claim corresponding to the method claim 6, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Claim(s) 5, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (PC-DARTS: PARTIAL CHANNEL CONNECTIONS FOR MEMORY-EFFICIENT ARCHITECTURE SEARCH) in view of Bi et al. (GOLD-NAS: Gradual, One-Level, Differentiable) in view of Elsken et al. (EFFICIENT MULTI-OBJECTIVE NEURAL ARCHITECTURE SEARCH VIA LAMARCKIAN EVOLUTION) in view of WENG et al. (Automatic Convolutional Neural Architecture Search for Image Classification Under Different Scenes)
Regarding claim 5
The combination of Xu, Bi, Elsken teaches claim 3.
However, the combination of Xu, Bi does not appear to explicitly teach:
wherein the plurality of operator groups are determined based on a quantity of parameters comprised in each type of operator in the plurality of operators.
WENG teaches
wherein the plurality of operator groups are determined based on a quantity of parameters comprised in each type of operator in the plurality of operators.
(WENG [sec(s) II] “The choice of operations needs to meet three conditions: uniqueness, fewer parameters and fast calculation. Fewer parameters means that simple POs will be put into the search space in order to consume less GPU or RAM memory resources during the search process, fast calculation is vital to speed up both architecture search and model running, and uniqueness here means that each operation has some unique properties that cannot be replaced by the others. Large receptive fields, such as a 5 × 5 size convolution and a 7 × 7 size convolution, can be replaced by stacking 3 × 3 size convolutions along the depth. Therefore, all convolution operations and pooling operations are limited by a 3 × 3 size. We analyze the CNN architectures that have achieved significant performance in image classification, image segmentation and object detection in recent years. The following POs are selected. • none • identity • average-pooling • max-pooling • separable convolution • dilation convolution • squeeze-and-excitation • channel shuffle convolution”;
Examiner notes that paragraph 20 of the Instant Specification describes “The grouping manner may be considered as a special case of grouping based on a quantity of operators. In other words, an operator that does not include a parameter is the first group, and an operator that includes a parameter is the second group. If a threshold is set for grouping, the threshold is equal to 0”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Xu, Bi, Elsken with the quantity of parameters for operators of WENG.
One of ordinary skill in the art would have been motived to combine in order to provide a more effective framework for solving different image classification problems using convolutional neural architecture search (CNAS), which can automatically learn the best CNN architecture for a specific dataset, and be useful for reducing the search space further.
(WENG [sec(s) IV] “We proposed a more effective framework for solving different image classification problems using convolutional neural architecture search (CNAS), which can automatically learn the best CNN architecture for a specific dataset. We show that both the channel shuffle convolution operation and squeeze-and-excitation operation are almost the only two operations selected for normal cells in classifying tasks after searching. This result may be useful for reducing the search space further. Although the performance of a sparse architecture will decrease a bit compare to a dense architecture, the number of parameters of the model will reduced by approximately 40%, which is very useful in some IOT scenarios in which hardware resources are limited (e.g., mobile phones and embedded deviced).”)
Regarding claim 17
The claim is an apparatus claim corresponding to the method claim 5, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Claim(s) 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (PC-DARTS: PARTIAL CHANNEL CONNECTIONS FOR MEMORY-EFFICIENT ARCHITECTURE SEARCH) in view of Bi et al. (GOLD-NAS: Gradual, One-Level, Differentiable) in view of Wong et al. (MicronNet: A Highly Compact Deep Convolutional Neural Network Architecture for Real-Time Embedded Traffic Sign Classification)
Regarding claim 10
The combination of Xu, Bi teaches claim 1.
However, the combination of Xu, Bi does not appear to explicitly teach:
wherein the method further comprises:
obtaining an image classification training sample; and
training the target neural network based on the image classification training sample, to obtain an image classification model, wherein the image classification model is used to classify an image
Wong teaches
wherein the method further comprises:
obtaining an image classification training sample; and
(Wong [sec(s) Abs] “In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies.” [sec(s) II.C-D] “The proposed MicronNet network was trained for 60,000 iterations in the Caffe framework with a training batch size of 50. Stochastic gradient descent with momentum and exponential decay was utilized as the training policy with the base learning rate set to 0.007, the momentum set to 0.9, the learning rate decay step size set to 1000, and the learning rate decay rate set to 0.9996. A l2 weight decay with rate 0.00001 was also used on the filters and matrices.” [sec(s) III.A] “The German traffic sign recognition benchmark (GTSRB) [20] used for evaluation purposes in this paper consists of color images of traffic signs (one traffic sign per image, with a total of 43 types of traffic signs) with image sizes varying from 15 × 15 to 250 × 250 pixels. There are a total of 39,209 color images in the training set and a total of 12,630 images in the test set”;)
training the target neural network based on the image classification training sample, to obtain an image classification model, wherein the image classification model is used to classify an image.
(Wong [fig(s) 4] [sec(s) Abs] “In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies.” [sec(s) II.C-D] “For the MicronNet network architecture, all parameters are characterized with half precision floating-point data representations after training to enable further model size reductions while still achieving strong performance. Alongside the use of fixed-point parameter precision for embedded applications, the utilization of half-precision floating-point parameter precision for deep neural networks has seen widespread adoption for embedded applications and hardware-accelerated in a wide range of embedded processors, including the Nvidia Tegra family of embedded processors as well as widely-used ARM embedded processors such as the Cortex-A53 high efficiency processor tested in this study. In additional, we also produced a variant of the proposed MicronNet network architecture with 16-bit fixed-point data representation for comparison purposes. … Here, we will discuss the training policy for learning the proposed MicronNet network. The proposed MicronNet network was trained for 60,000 iterations in the Caffe framework with a training batch size of 50. Stochastic gradient descent with momentum and exponential decay was utilized as the training policy with the base learning rate set to 0.007, the momentum set to 0.9, the learning rate decay step size set to 1000, and the learning rate decay rate set to 0.9996. A l2 weight decay with rate 0.00001 was also used on the filters and matrices.” [sec(s) III.A] “The German traffic sign recognition benchmark (GTSRB) [20] used for evaluation purposes in this paper consists of color images of traffic signs (one traffic sign per image, with a total of 43 types of traffic signs) with image sizes varying from 15 × 15 to 250 × 250 pixels. There are a total of 39,209 color images in the training set and a total of 12,630 images in the test set” [sec(s) III.C] “To study where the proposed MicronNet encounters difficulties, we examine some of the traffic images from the GTSRB test dataset that has been misclassified by the proposed MicronNet (see Fig. 4). It can be observed that in the example misclassified traffic images, the sign is either heavily motion blurred (left), partially occluded (middle), or exhibit poor illumination (right). The identification of such misclassifications can provide good insight into the weaknesses of a network”; e.g., “MicronNet” read(s) on “target neural network”.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Xu, Bi with the target NN training of Wong.
One of ordinary skill in the art would have been motived to combine in order to greatly improve the efficiency of the resulting deep convolutional neural network while maintaining strong accuracy.
(Wong [sec(s) II] “the overall network architecture of the proposed MicronNet network for real-time embedded traffic sign recognition is inspired by the network macroarchitecture described in [1] and takes the following microarchitecture-level and macroarchitecture-level design considerations and optimization strategies into account to greatly improve the efficiency of the resulting deep convolutional neural network while maintaining strong accuracy (see Figure 2): • Optimizing microarchitectures of each convolutional layer via numerical optimization for reduced number of parameters • Incorporating spectral augmentations to produce a spectral-spatial macroarchitecture that further reduces number of parameters and computational complexity while maintaining strong accuracy • Optimizing parameter precision for reduced model size while maintaining strong accuracy”)
Regarding claim 11
The combination of Xu, Bi teaches claim 1.
However, the combination of Zhao, Lee does not appear to explicitly teach:
wherein the method further comprises:
obtaining a target detection training sample; and
training the target neural network based on the target detection training sample, to obtain a target detection model, wherein the target detection model is used to detect a target from a to-be-processed image.
Wong teaches
wherein the method further comprises:
obtaining a target detection training sample; and
(Wong [sec(s) Abs] “In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies.” [sec(s) II.C-D] “The proposed MicronNet network was trained for 60,000 iterations in the Caffe framework with a training batch size of 50. Stochastic gradient descent with momentum and exponential decay was utilized as the training policy with the base learning rate set to 0.007, the momentum set to 0.9, the learning rate decay step size set to 1000, and the learning rate decay rate set to 0.9996. A l2 weight decay with rate 0.00001 was also used on the filters and matrices.” [sec(s) III.A] “The German traffic sign recognition benchmark (GTSRB) [20] used for evaluation purposes in this paper consists of color images of traffic signs (one traffic sign per image, with a total of 43 types of traffic signs) with image sizes varying from 15 × 15 to 250 × 250 pixels. There are a total of 39,209 color images in the training set and a total of 12,630 images in the test set”;)
training the target neural network based on the target detection training sample, to obtain a target detection model, wherein the target detection model is used to detect a target from a to-be-processed image.
(Wong [sec(s) Abs] “In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies.” [sec(s) II.C-D] “For the MicronNet network architecture, all parameters are characterized with half precision floating-point data representations after training to enable further model size reductions while still achieving strong performance. Alongside the use of fixed-point parameter precision for embedded applications, the utilization of half-precision floating-point parameter precision for deep neural networks has seen widespread adoption for embedded applications and hardware-accelerated in a wide range of embedded processors, including the Nvidia Tegra family of embedded processors as well as widely-used ARM embedded processors such as the Cortex-A53 high efficiency processor tested in this study. In additional, we also produced a variant of the proposed MicronNet network architecture with 16-bit fixed-point data representation for comparison purposes. … Here, we will discuss the training policy for learning the proposed MicronNet network. The proposed MicronNet network was trained for 60,000 iterations in the Caffe framework with a training batch size of 50. Stochastic gradient descent with momentum and exponential decay was utilized as the training policy with the base learning rate set to 0.007, the momentum set to 0.9, the learning rate decay step size set to 1000, and the learning rate decay rate set to 0.9996. A l2 weight decay with rate 0.00001 was also used on the filters and matrices.” [sec(s) III.A] “The German traffic sign recognition benchmark (GTSRB) [20] used for evaluation purposes in this paper consists of color images of traffic signs (one traffic sign per image, with a total of 43 types of traffic signs) with image sizes varying from 15 × 15 to 250 × 250 pixels. There are a total of 39,209 color images in the training set and a total of 12,630 images in the test set”; e.g., “MicronNet” read(s) on “target neural network”.)
The combination of Xu, Bi is combinable with Wong for the same rationale as set forth above with respect to claim 10.
Regarding claim 12
The combination of Xu, Bi, Wong teaches claim 11.
Wong further teaches
wherein the target comprises at least one of the following: a vehicle, a pedestrian, an obstacle, a road sign, or a traffic sign.
(Wong [sec(s) Abs] “In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies.” [sec(s) II.C-D] “The proposed MicronNet network was trained for 60,000 iterations in the Caffe framework with a training batch size of 50.” [sec(s) III.A] “The German traffic sign recognition benchmark (GTSRB) [20] used for evaluation purposes in this paper consists of color images of traffic signs (one traffic sign per image, with a total of 43 types of traffic signs) with image sizes varying from 15 × 15 to 250 × 250 pixels. There are a total of 39,209 color images in the training set and a total of 12,630 images in the test set”; e.g., “MicronNet” read(s) on “target neural network”.)
The combination of Xu, Bi, Wong is combinable with Wong for the same rationale as set forth above with respect to claim 10.
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xie et al. (Exploring Randomly Wired Neural Networks for Image Recognition) teaches generating randomly wired NNs.
Laube et al. (Prune and Replace NAS) teaches removing the worst operators.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEHWAN KIM whose telephone number is (571)270-7409. The examiner can normally be reached Mon - Fri 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEHWAN KIM/Examiner, Art Unit 2129