Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 12-21 are presented for examination.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: P and 13. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Page 8 line 15 recites “In a fourth step (104)of the method”; however, it should recite - - in a fourth step (104) of the method --.
Appropriate correction is required.
Claim Objections
Claim 14 is objected to because of the following informalities:
Claim 14 recites “input of the depth-wise the convolutional layer”; however, it should recite - - input of the depth-wise convolutional layer - -.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12 and 14-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 12
Step 1: The claim recites “A computer-implemented method for training a neural network, wherein the neural network is configured to”; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: The claim recites, inter alia:
determine an output signal based on an input signal, the training method comprising the following steps: These limitations recite a mentally performable process of observation to determine an output signal based on an input signal, the training method comprising the following steps.
training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer.
Thus, this claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A computer-implemented method for training a neural network, wherein the neural network is configured to: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include a recitation of invoking a computer or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 14
Step 1: a process, as in claim 12.
Step 2A Prong 1: the claim recites, inter alia:
wherein the variance is determined according to the formula: 1 V = wherein KW is a kernel width of filters of the convolutional layer, Kh is a kernel height of filters of the convolutional layer, and k is the number of filters applied at each depth of the input of the depth-wise the convolutional layer: These limitations recite a mathematical concept of a formula/equation.
Thus, this claim further recites the judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 15
Step 1: a process, as in claim 12.
Step 2A Prong 1: the claim recites, inter alia:
wherein the predefined probability distribution is a Normal distribution: These limitations recite a mathematical concept of a formula/equation.
Thus, this claim further recites the judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 16
Step 1: a process, as in claim 15.
Step 2A Prong 1: the claim recites, inter alia:
wherein the predefined probability distribution has an expected value of zero: These limitations recite a mathematical concept of a formula/equation.
Thus, this claim further recites the judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 17
Step 1: The claim recites “A computer-implemented method for determining an output signal based on an input signal, wherein the method comprises the following steps”; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: the claim recites, inter alia:
determining an output signal based on an input signal, wherein the method comprises the following steps: These limitations recite a mentally performable process of observation to determine an output signal based on an input signal, wherein the method comprises the following steps.
training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer.
determining the output signal using the input signal: These limitations recite a mentally performable process of observation to determine the output signal.
Thus, the claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A computer-implemented method for: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
obtaining a neural network that has been trained by: These additional elements are recited at a high level of generality and merely recite insignificant extra-solution activity of obtaining a neural network that has been trained by. See MPEP 2106.05(g).
using the obtained neural network: These additional elements are recited at a high level of generality and merely recite using the obtained neural network without details of how it is to be accomplished, e.g. no description of inventive steps of how using the obtained neural network works other than only that it happens, and is equivalent of “apply it” 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include insignificant extra-solution activity of obtaining a neural network that has been trained by which is well-understood, routine and conventional activity similar to presenting offers and gathering statistics. See MPEP 2106.05(d)(II). Further additional elements include invoking computers or other machinery to apply the underlying judicial exception. Further additional elements include a recitation of the worst “apply it” (or an equivalent). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 18
Step 1: The claim recites “A computer-implemented neural network, wherein the neural network is configured to:”; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: the claim recites, inter alia.
determine an output signal based on an input signal, wherein the neural network is trained by: These limitations recite a mentally performable process of observation to determine an output signal based on an input signal, wherein the neural network is trained by.
training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer.
Thus, this claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A computer-implemented neural network, wherein the neural network is configured to: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include a recitation of invoking a computer or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 19
Step 1: The claim recites “A control system configured to use a neural network:”; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: the claim recites, inter alia.
wherein the neural network is trained by training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer, where in the control system is configured to determine a control signal for controlling an actuator of a technical system and/or a display of a technical system. These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer, where in the control system is configured to determine a control signal for controlling an actuator of a technical system and/or a display of a technical system.
Thus, this claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A control system configured to use a neural network: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include a recitation of invoking a computer or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 20
Step 1: The claim recites “A training system configured to train a neural network:”; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: the claim recites, inter alia.
determine an output signal based on an input signal, the training system being configured to: These limitations recite a mentally performable process of observation to determine an output signal based on an input signal, the training system being configured to.
train parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer
Thus, this claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A training system configured to train a neural network: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
wherein the neural network is configured to: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include a recitation of invoking a computer or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 21
Step 1: The claim recites “A non-transitory machine-readable storage medium on which is stored a computer program for training a neural network, wherein the neural network is configured to determine an output signal based on an input signal, the computer program, when executed by a computer, causing the computer to perform the following steps:”; therefore, these claims are directed to the statutory category of an article of manufacture.
Step 2A Prong 1: the claim recites, inter alia.
determine an output signal based on an input signal: These limitations recite a mentally performable process of observation to determine an output signal based on an input signal.
training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: These limitations recite a mathematical relationship similar to organizing information and manipulating information, e.g. initializing a depth-wise convolutional layer based on values drawn from a predefined probability distribution, wherein a variance of the probability distribution is characterized by a reciprocal of a square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer
Thus, this claim further recites a judicial exception.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A non-transitory machine-readable storage medium on which is stored a computer program for training a neural network, wherein the neural network is configured to… the computer program, when executed by a computer, causing the computer to perform the following steps: These additional elements are recited at a high level of generality and merely recite generic computer components invoked as a tool to carry out the abstract idea. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include a recitation of invoking a computer or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 12-18 and 20-21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Anonymous authors (SCALING LAWS FOR THE PRINCIPLED DESIGN, INITIALIZATION, AND PRECONDITIONING OF RELU NET- WORKS, 25 September 2019), hereinafter Anonymous.
Regarding claim 12, Anonymous teaches: A computer-implemented method for training a neural network, wherein the neural network is configured to determine an output signal based on an input signal, the training method comprising the following steps: (Anonymous; [Page 9, 8.1 Convolutional experiments], Briefly described, a neural network tested using CIFAR- 10 indicating images being measured using sensors to obtain sensor data as input and determine an output through a probability value of standard deviation):
training parameters of a depth-wise convolutional layer of the neural network:
(Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5, Remark 6], Briefly described, parameters of a depth-wise convolutional layer of the neural network being trained):
wherein the depth-wise convolutional layer is initialized based on values drawn from a
predefined probability distribution: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5], Briefly described, depth-wise convolutional layer is initialized based on values from previous probability equations):
wherein a variance of the probability distribution is characterized by a reciprocal of a
square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization], Briefly described, Equation 2 consisting of a reciprocal of a square root of filters applied at each depth of an input of the depth-wise convolutional layer to characterize the variance of the probability distribution; [Page 9, 8.1 Convolutional experiments], Briefly described, a variety of filter sizes in a convolutional layer; [Page 15, E Conditioning of ResNets without normalization layers], Briefly described, variance based off depth input).
Regarding claim 13, Anonymous teaches the method of claim 12 wherein the input signal characterizes a measurement obtained from a sensor: (Anonymous; [Page 9, 8.1 Convolutional experiments], Briefly described, CIFAR-10 testing where images are measured using sensors to obtain sensor data for an input signal).
Regarding claim 14, Anonymous teaches the method of claim 12 wherein the variance is determined according to the formula: 1 V = wherein KW is a kernel width of filters of the convolutional layer, Kh is a kernel height of filters of the convolutional layer, and k is the number of filters applied at each depth of the input of the depth-wise the convolutional layer: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Briefly described, Equation 2 to determine the variance through a formula that scales kernel sizes based on filters of the convolution layers using a reciprocal of a square root serving the same function).
Regarding claim 15, Anonymous teaches the method of claim 12 wherein the predefined probability distribution is a Normal distribution: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5], Briefly described, a symmetrical distribution which normal distributions fall under).
Regarding claim 16, Anonymous teaches the method of claim 15 wherein the predefined probability distribution has an expected value of zero: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5], Briefly described, a predefined probability distribution with a mean of 0, and the distribution is uncorrelated and symmetrically distributed meaning zero is an expected value).
Regarding claim 17, it is a method claim that corresponds to method claim 12. Therefore, it is rejected for the same reason as claim 12 above.
Regarding claim 18, it is a system claim that corresponds to method claim 12. Therefore, it is rejected for the same reason as claim 12 above.
Regarding claim 20, it is a system claim that corresponds to method claim 12. Therefore, it is rejected for the same reason as claim 12 above.
Regarding claim 21, it is a computer-readable storage media claim that corresponds to method claim 12. Therefore, it is rejected for the same reason as claim 12 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Anonymous authors (SCALING LAWS FOR THE PRINCIPLED DESIGN, INITIALIZATION, AND PRECONDITIONING OF RELU NET- WORKS, 25 September 2019), hereinafter Anonymous, and in view of Springer et al. (WO 2021045976 A1, filed 08/28/2020), hereinafter Springer.
Regarding claim 19, Anonymous teaches: training parameters of a depth-wise convolutional layer of the neural network: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5, Remark 6], Briefly described, parameters of a depth-wise convolutional layer of the neural network being trained):
wherein the depth-wise convolutional layer is initialized based on values drawn from a
predefined probability distribution: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization, Proposition 5], Briefly described, depth-wise convolutional layer is initialized based on values from previous probability equations):
wherein a variance of the probability distribution is characterized by a reciprocal of a
square root of a number of filters applied at each depth of an input of the depth-wise convolutional layer: (Anonymous; [Page 5, 6. Preconditioning of neural networks via initialization], Briefly described, Equation 2 consisting of a reciprocal of a square root of filters applied at each depth of an input of the depth-wise convolutional layer to characterize the variance of the probability distribution; [Page 9, 8.1 Convolutional experiments], Briefly described, a variety of filter sizes in a convolutional layer; [Page 15, E Conditioning of ResNets without normalization layers], Briefly described, variance based off depth input).
However, Anonymous fails to expressly teach – A control system configured to use a neural network, wherein the neural network is trained by; and – where in the control system is configured to determine a control signal for controlling an actuator of a technical system and/or display of a technical system.
In the same field of endeavor, Springer teaches:
A control system configured to use a neural network, wherein the neural network is
trained by: (Springer; [0111], Briefly described, a control system for controlling a vehicle; [0115], Briefly described, trained neural networks in which a control system utilizes):
where in the control system is configured to determine a control signal for controlling an
actuator of a technical system and/or display of a technical system: (Springer; [0111], Briefly described, a control system for controlling a vehicle and determines signals to control the system like vehicle brakes though brake actuators; [0113], Briefly described, controllers may provide output to control the display of a technical system through a human-machine interface).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated - A control system configured to use a neural network, wherein the neural network is trained by; and – where in the control system is configured to determine a control signal for controlling an actuator of a technical system and/or display of a technical system as suggested by Anonymous and Springer. Doing so would be desirable because having an actuator coupled with the neural network used for applying the mathematical formula through each convolutional layer adds control to the system. A display present to see the actions taken by the technical system by the neural network helps understand the signal processes causing the actions to take place. The combination of Anonymous and Springer provides a human-machine interface to the model so the outcome of the measurements of convolutional layers upon receiving signals as input can have a visual component to display the actions of the actuator.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kolter et al. (EP 3798913 A1, filed 09/24/2019) teaches in Page 5 Paragraph an actuator control signal where the actuator is for controlling a technical system like an autonomous robot or a manufacturing machine.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROLANDO PATRICK VIRREIRA whose telephone number is (571)270-1570. The examiner can normally be reaches Monday – Friday, 8:30AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions. Contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROLANDO PATRICK VIRREIRA/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights