DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is responsive to Applicant’s claims filed 055/30/2024.
Claims 1-20 are currently pending and have been examined here.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101. The claims are drawn to ineligible patent subject matter, because the claims are directed to a recited judicial exception to patentability (an abstract idea), without claiming something significantly more than the judicial exception itself.
Claims are ineligible for patent protection if they are drawn to subject matter which is not within one of the four statutory categories, or, if the subject matter claimed does fall into one of the four statutory categories, the claims are ineligible if they recite a judicial exception, are directed to that judicial exception, and do not recite additional elements which amount to significantly more than the judicial exception itself. Alice Corp. v. CLS Bank Int'l, 375 U.S. ___ (2014). Accordingly, claims are first analyzed to determine whether they fall into one of the four statutory categories of patent eligible subject matter. Then, if the claims fall within one of the four statutory categories, it must be determined whether the claims are directed to a judicial exception to patentability (i.e., a law of nature, a natural phenomenon, or an abstract idea). In determining whether a claim is directed to a judicial exception, the claim is first analyzed to determine whether the claim recites a judicial exception. If the claim does not recite one of these exceptions, the claim is directed to patent eligible subject matter under 35 U.S.C. 101. If the claim recites one of these exceptions, the claim is then analyzed to determine whether the claim recites additional elements that integrate the exception into a practical application of that exception. Claims which integrate the exception into a practical application of that exception are directed to patent eligible subject matter under 35 U.S.C. 101. If the claim fails to integrate the exception into a practical application of that exception, the claim is directed to an abstract idea. Finally, if the claims are directed to a judicial exception to patentability, the claims are then analyzed determine whether the claims are directed to patent eligible subject matter by reciting meaningful limitations which transform the judicial exception into something significantly more than the judicial exception itself. If they do not, the claims are not directed towards eligible subject matter under 35 U.S.C. § 101.
Regarding independent claims 1, 9, and 13 the claims are directed to one of the four statutory categories (each is directed to a process). The claimed invention of independent claims 1, 9, and 13 is directed to a judicial exception to patentability, an abstract idea. The claims include limitations which recite elements which can be properly characterized under at least one of the following groupings of subject matter recognized as abstract ideas by MPEP 2106.04(a):
Mathematical Concepts: mathematical relationships, mathematical formulas or equations, and mathematical calculations;
Certain methods of organizing human activity: fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes: concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
Claims 1, 9, and 13, as a whole, recite the following limitations:
defining a problem to be solved using a neural network; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could define a problem to be solved using a neural network)
providing data to be input to the neural network; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could provide data)
splitting the data into a training data set and a test data set, the test data set and the training data set being mutually exclusive; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could split data into a training and a test set which are mutually exclusive)
splitting a validation data set from the training data set, the validation data set and the training data set being mutually exclusive; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could split validation data into two mutually exclusive sets)
determining a decision plurality of decision vectors and a weight plurality of weight vectors; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could determine a plurality of decision vectors and weight a plurality of weight vectors; furthermore, the broadest reasonable interpretation of this limitation recites mathematical concepts since the weighting of vectors sets forth and describes mathematical operations in form of matrix algebra)
pairing individual decision vectors from the decision plurality of decision vectors with corresponding individual weight vectors from the weight plurality of weight vectors; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could pair vectors with weight vectors)
constructing a data structure consisting of decision vector orientations, specified by angles relative to weight vectors, for input data from a validation data set neither used for training or testing the neural network; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could construct a data structure specified by angles relative to weight vectors for this data)
determining which decision vectors in the data structure are within a specified spatial neighborhood of a test or operational data decision vector under evaluation; (claims 1, 9, 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could determine which vectors are within a spatial neighborhood of a vector under evaluation)
estimating at least one Bayesian probability from a plurality of class counts of vectors in the data structure and a plurality of prior class distributions that the neural network predictions are correct; (claims 1, 9; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could estimate a Bayesian probability in this fashion; furthermore, the broadest reasonable interpretation of this limitation recites mathematical concepts since the estimation of a Bayesian probability sets forth and describes mathematical operations and formulas)
and using the at least one Bayesian probability set to provide a human or a machine decision maker with the likelihood information about the neural network's prediction needed to make a risk informed decision. (claims 1, 9 13; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could provide a human with a likelihood of a neural network’s prediction)
estimating a plurality of conditionally informed probability confidence estimation probabilities from at least one class count of vectors in the data structure and at least one prior class distribution that the neural network predictions are correct; (claims 1, 9; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could estimate a plurality of probabilities in this fashion)
computing plural distribution parameters from the parametric function; (claims 1, 9; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could estimate compute plural distribution parameters from a parametric function; furthermore, the broadest reasonable interpretation of this limitation recites mathematical concepts since the computation of plurality distribution parameters from a parametric function sets forth and describes mathematical operations and formulas)
computing plural probabilities that the values from the plural distribution parameters are correct; (claims 1, 9; the broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could estimate a plurality of probabilities in this fashion)
The above elements, as a whole, recite mental processes since, but for the requirement to implement the above steps on a set of generic computer components or “apply” the abstract idea using a machine learning model, the entirety of the above set of steps could be performed by a human using their mind, pen and paper, and simple observation, evaluation, and judgment.
Moving forward, the above recited abstract idea is not integrated into a practical application.
The added limitations do not represent an integration of the abstract idea into a practical application because:
the claims represent mere instructions to implement an abstract idea on a computer, and merely use a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).
the claims merely add insignificant extra-solution activity to the judicial exception (activity which can be characterized as incidental to the primary purpose or product that is merely a nominal or tangential addition to the claim). See MPEP 2106.05(g) and/or
the claims represent mere general linking of the use of the judicial exception to a particular technological environment or field of use. See MPEP 2016.05(h)
Beyond those limitations which recite the abstract idea, the following limitations are added:
training the neural network to have weight parameters and bias parameters to minimize plural aggregate differences between at least one prediction from the neural network and at least one truth datum contained within the training data set; (claim 1; the broadest reasonable interpretation of this limitation represents the mere requirement to “apply” the abstract idea of solving a problem using a neural network since the limitation merely recites training a neural network at a high level of generality, the neural network is used in its ordinary capacity to perform a prediction or solve a problem, and since the limitation recites the outcome or solution of training the neural network without describing how the solution is brought forth)
The claims, as a whole, are directed to the abstract idea(s) which they recite. The claim limitations do not present improvements to another technological field, nor do they improve the functioning of a computer or another technology. Nor do the claim limitations apply the judicial exception with, or by use of a particular machine. The claims do not effect a transformation or reduction of a particular article to a different state or thing. See MPEP 2106.05(c). None of the hardware in the claims "offers a meaningful limitation beyond generally linking 'the use of the [method] to a particular technological environment' that is, implementation via computers” such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See MPEP 2106.05(e); Alice Corp. v. CLS Bank Int’l (citing Bilski v. Kappos, 561 U.S. 610, 611 (U.S. 2010)). Therefore, because the claims recite a judicial exception (an abstract idea) and do not integrate the judicial exception into a practical application, the claims, as a whole, are directed to the judicial exception.
Turning to the final prong of the test (Step 2B), independent claims 1, 9, and 13 do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because there are no meaningful limitations which transform the exception into a patent eligible application.
As outlined above, the claim limitations do not present improvements to another technological field, nor do they improve the functioning of a computer or another technology. Nor do the claim limitations apply the judicial exception with, or by use of a particular machine. The claims do not effect a transformation or reduction of a particular article to a different state or thing. See MPEP 2106.05(c). None of the hardware in the claims "offers a meaningful limitation beyond generally linking 'the use of the [method] to a particular technological environment' that is, implementation via computers” such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See MPEP 2106.05(e); Alice Corp. v. CLS Bank Int’l (citing Bilski v. Kappos, 561 U.S. 610, 611 (U.S. 2010)).
Furthermore, no specific limitations are added which represent something other than what is well-understood, routine, and conventional activity in the field. See MPEP 2106.05(d). Besides performing the abstract idea itself, the generic computer components only serve to perform the court-recognized well-understood computer functions of receiving or transmitting data over a network, performing repetitive calculations, electronic record keeping, and storing and retrieving information in memory. See MPEP 2106.05(d). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. Their collective functions merely provide conventional computer implementation. The specification details any combination of a generic computer system program to perform the method. Generically recited computer elements do not add a meaningful limitation to the abstract idea because they would be routine in any computer implementation and because the Alice decision noted that generic structures that merely apply the abstract ideas are not significantly more than the abstract ideas. Therefore, independent claims 1, 9, and 13 are rejected under 35 U.S.C. §101 as being directed to ineligible subject matter.
Claims 2-8, 10-12, and 14-20, recite the same abstract idea as their respective independent claims.
The following additional features are added in the dependent claims:
Claim 2:
wherein the step of splitting the data into a training data set and a test data set comprises randomly splitting the data.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could randomly split data.
Claim 3:
sequestering the test data set while splitting the data into the training data set and the test data set.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could sequester test data while splitting data.
Claim 4:
wherein the step of determining the decision plurality of decision vectors and the weight plurality of weight vectors comprises making the determinations from data in the validation data set.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could make determinations from data in a validation set.
Claim 5:
wherein the decision vector spatial orientations are saved in a data structure for later reference to be used to estimate probabilities for test or operational data.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could save orientations in a data structure for later reference.
Claim 6:
wherein the problem is defined using a neural network classifier model.
The broadest reasonable interpretation of this limitation represents the mere requirement to “apply” the abstract idea of solving a problem using a neural network classifier model since the limitation merely recites the neural network at a high level of generality, the neural network is used in its ordinary capacity to perform a prediction or solve a problem, and since the limitation recites the outcome or solution of using the neural network without describing how the solution is brought forth.
Claim 7:
wherein the neural network classifier model defines a type of data input and a plurality of classes of the data input.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could define a type of input and plurality of classes to be input into a model.
Claim 8:
wherein the type of data comprises visual images.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could define images as a type of input into a model.
Claim 10:
wherein the orientation of decision vectors relative to the weight vectors are stored in a data structure.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could store this information in a data structure.
Claim 11:
wherein the plural aggregate differences are expressed as a cross-entropy loss.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could express differences as a cross-entropy loss.
Claim 12:
wherein the type of data comprises visual images.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could perform the abstract idea above using visual image data.
Claim 14:
computing a calibration error for at least one of the estimated probability.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could compute a calibration error for an estimated probability.
Claim 15:
wherein the step of computing a calibration error comprises computing the calibration error comprises computing the calibration error for a statistically significant number of samples.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could compute a calibration error for a statistically significant number of samples.
Claim 16:
wherein the step of minimizing plural aggregate differences comprises optimizing the weight parameters and bias parameters using a gradient descent.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could optimize weighted parameters and bias parameters using a gradient descent. Furthermore, the use of a gradient descent algorithm sets forth and describes a mathematical formula to be used to minimize differences and therefore this limitation recites mathematical concepts.
Claim 17:
wherein the gradient descent is a stochastic gradient descent.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could optimize weighted parameters and bias parameters using a stochastic gradient descent. Furthermore, the use of a stochastic gradient descent algorithm sets forth and describes a mathematical formula to be used to minimize differences and therefore this limitation recites mathematical concepts.
Claim 18:
wherein the step of training the neural network is terminated when the aggregate differences for the validation data set reach a minimum value, in order to avoid overfitting the model.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could stop training of a model when differences reach a minimum value.
Claim 19:
wherein the conditionally informed probability confidence estimation uses Bayes' rule.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could use Bayes’ rule to estimate probability confidence.
Claim 20:
further comprising the step of providing a human or machine decision maker with the likelihood information about the neural network's prediction needed to make a risk informed decision.
The broadest reasonable interpretation of this limitation recites mental processes since a human using their mind, pen and paper, and simple observation, evaluation, and judgment could provide a human with this information.
The above limitations do not represent a practical application of the recited abstract idea. The claim limitations do not present improvements to another technological field, nor do they improve the functioning of a computer or another technology. Nor do the claim limitations apply the judicial exception with, or by use of a particular machine. The claims do not effect a transformation or reduction of a particular article to a different state or thing. See MPEP 2106.05(c). None of the hardware in the claims "offers a meaningful limitation beyond generally linking 'the use of the [method] to a particular technological environment' that is, implementation via computers” such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See MPEP 2106.05(e); Alice Corp. v. CLS Bank Int’l (citing Bilski v. Kappos, 561 U.S. 610, 611 (U.S. 2010)). Therefore, because the claims recite a judicial exception (an abstract idea) and do not integrate the judicial exception into a practical application, the claims are also directed to the judicial exception.
Furthermore, the added limitations do not direct the claim to significantly more than the abstract idea. No specific limitations are added which represent something other than what is well-understood, routine, and conventional activity in the field. See MPEP 2106.05(d). Accordingly, none of the dependent claims 2-8, 10-12, and 14-20, individually, or as an ordered combination, are directed to patent eligible subject matter under 35 U.S.C. 101.
Please see MPEP §2106.05(d)(II) for a discussion of elements that the Courts have recognized as well-understood, routine, conventional, activity in particular fields.
Please see MPEP §2106 for examination guidelines regarding patent subject matter eligibility.
Novelty/Non-obviousness
Regarding the novelty/non-obviousness of claims 1, 9, and 13, the prior art does not appear to explicitly teach, in the context of the systems and methods recited for estimating confidence in a neural network. that a data structure of decision vector orientations may be created specified by angles relative to weight vectors, decision vectors within a specified spatial neighborhood of a decision vector under evaluation may be determined, and a probability from a plurality of class counts of vectors in the data structure that the neural network is correct may be determined. Such an ordered combination of elements, in combination with the other elements of the claim, is not taught or suggested by the prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMETT K WALSH whose telephone number is (571)272-2624. The examiner can normally be reached Mon.-Fri. 6 a.m. - 4:45 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jessica Lemieux can be reached at 571-270-3445. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMMETT K. WALSH/Primary Examiner, Art Unit 3626
Prosecution Insights