DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant's amendments and remarks, filed 04/14/2026, are acknowledged. Rejections and/or objections not reiterated from previous office actions are hereby withdrawn. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Status of Claims
Claims 1-8, 11, 15-21, 25, 27-29 are under examination.
Claims 9, 10, 24 are withdrawn.
Claims 12, 13, 14, 22, 23, 26, 30 are cancelled.
Priority
Applicant’s claim for the benefit of priority under 35 U.S.C. 119(a)-(d) is acknowledged. This application claims the right of priority under 35 U.S.C. 371 to International Application No. PCT/US2020/034475, filed May 26, 2020, which claims priority to and the benefit of United States Provisional Patent Application Number 62/857,774, filed June 5, 2019.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The following rejection is modified in view of applicant’s amendments.
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
The United States Patent and Trademark Office published revised guidance on the application of 35 U.S.C. § 101. USPTO’s 2019 Revised Patent Subject Matter Eligibility Guidance (“Guidance”). Under the Guidance, in determining what concept the claim is “directed to,” we first look to whether the claim recites:
(1) any judicial exceptions, including certain groupings of abstract ideas (i.e., mathematical concepts, certain methods of organizing human activity such as a fundamental economic practice, or mental processes) (Guidance Step 2A, Prong 1); and
(2) additional elements that integrate the judicial exception into a practical application (see MPEP § 2106.05(a)-(c), (e)-(h)) (Guidance Step 2A, Prong 2).
Only if a claim (1) recites a judicial exception and (2) does not integrate that exception into a practical application, do we then look to whether the claim contains an “‘inventive concept’ sufficient to ‘transform’” the claimed judicial exception into a patent-eligible application of the judicial exception. Alice, 573 U.S. at 221 (quoting Mayo, 566 U.S. at 82). In so doing, we thus consider whether the claim:
(3) adds a specific limitation beyond the judicial exception that are not “well-understood, routine and conventional in the field” (see MPEP § 2106.05(d)); or 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019).
(4) simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.(Guidance Step 2B). See Guidance, 84 Fed. Reg. at 54-56.
Guidance Step 1:
The instant invention (claim 15 being representative) is directed to a method for performing out-of-distribution detection (i.e. a process). Thus, the claims are directed to one of the statutory categories of invention. MPEP 2106.03.
A. Guidance Step 2A, Prong 1
The Revised Guidance instructs us first to determine whether any judicial exception to patent eligibility is recited in the claim. The Revised Guidance identifies three judicially-excepted groupings identified by the courts as abstract ideas: (1) mathematical concepts, (2) certain methods of organizing human behavior such as fundamental economic practices, and (3) mental processes. Regarding claim(s) 1, the claimed steps that are part of the abstract idea are as follows:
Claim 15
training, by the one or more computing devices, a first machine-learned model using the set of first training data;
perturbing, by the one or more computing devices, one or more first training examples of the plurality of first training examples to generate one or more second training examples;
training, by the one or more computing devices, a second machine-learned model using a set of second training data that comprises the one or more second training examples;
inputting, by the one or more computing devices, a data input into the first machine- learned model that has been trained on the set of first training data to generate a first likelihood value for the data input, wherein the data input comprises image data, audio data, textual data, sensor data, genomic data, or molecule data;
inputting, by the one or more computing devices, the data input into the second machine-learned generative model that has been trained on the set of second training data to generate a second likelihood value for the data input;
screening, by the one or more computing devices, the data input from processing by one or more downstream system components based at least in part on a likelihood ratio value for the data input based at least in part on the first likelihood value and the second likelihood value.
Mental Processes
With regards to perturbing training examples, this step is recited at a high level of generality (without any technological details directed to how it is performed). In addition, scientists routinely modify or manipulate (i.e. perturb) data using their brains. As such, this step encompasses a mental process of observing data and/or manipulating data. MPEP 2106.04(a)(2), section III.
With regards to inputting data (to generate a likelihood value), this step is recited at a high level of generality (without any technological details directed to how it is performed and without any specificity with regards to the type of model being used). In addition, scientists routinely input data into equations (i.e. models) to generate additional data and this can be done in the mind. As such, this step encompasses a mental process of observing data and manipulating data to generate additional data. MPEP 2106.04(a)(2), section III.
With regards to screening data (based on likelihood values), this step is recited at a high level of generality (without any technological details directed to how it is performed). In addition, scientists routinely input data into equations (i.e. models) to generate additional data and this can be done in the mind. As such, this step encompasses a mental process of observing data, performing analysis, and making a decision. MPEP 2106.04(a)(2), section III.
It is important to note that “Claims that recite performing information analysis as well as the collection and manipulation of information related to such analysis, have been determined by our reviewing court to be an abstract concept that is not patent eligible. See SAP, 898 F.3d, 1165, 1167, 1168 (Claims reciting "[a] method for providing statistical analysis" (id. at 1165) were determined to be "directed to an abstract idea" (id. at 1168)); see also Content Extraction & Transmission LLC v. Wells Fargo Bank, Nat'l Ass 'n, 776 F.3d 1343, 1345, 1347 (Fed. Cir. 2014) (finding the "claims generally recite ... extracting data ... [and] recognizing specific information from the extracted data" and that the "claims are drawn to the basic concept of data recognition"). "As many cases make clear, even if a process of collecting and analyzing information is limited to particular content or a particular source, that limitation does not make the collection and analysis other than abstract." SAP, 898 F.3d at 1168 (internal quotation marks omitted))." [Step 2A, Prong 1: YES].
Mathematical Concept
With regards to training the models, these steps are recited at a high level of generality (without any technological details or specificity with regards to how they are performed, the structure of the model, or how the model operates). Moreover, the artisan would recognize that training a model is an explicit mathematic process of assigning and adjusting weights of parameters that define the structure of the model. This position is supported by Goodfellow et al. ("Explaining and harnessing adversarial examples”, Published as a conference paper at ICLR 2015, pp.1-11), which explicitly teaches that training a model explicitly requires mathematically relating data [Sections 3 and 4]. This position is further supported by the specification, which teaches mathematical models/algorithms associated with “generative” models [see at least pages 6-8]. As such, this step recites mathematical correlations and/or calculations. MPEP 2106.04(a)(2) Section I.
With regards to inputting data (for the intended use of generating likelihood values), this step is recited at a high level of generality (without any technological details directed to how it is performed and without any specificity with regards to the type of model being used). In addition, the model is being used to calculate likelihood values. As such, this step recites mathematical correlations and/or calculations. MPEP 2106.04(a)(2) Section I.
Similar to the ineligible claims at issue for In re: Board of Trustees of the Leland Stanford Junior University, 991 F.3d 1245 (Fed. Cir. 2021), the instant claims are written effectively as a method for mathematically manipulating or relating data to ascertain additional data. While no specific equation is being claimed, Applicant is reminded that there is no particular word or set of words that indicates a claim recites a mathematical calculation. See MPEP 2106.04(a)(2). Therefore, when read in light of applicant’s own specification, the claims are directed to mathematical concepts. See MPEP 2106.04 and 2106.05(II). [Step 2A, Prong 1: YES].
B. Guidance Step 2A, Prong 2
This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional steps/elements recited in the claim beyond the judicial exception, and (2) evaluating those additional steps/elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d).
In this case, the additional steps/elements recited in the claim beyond the judicial exception are as follows:
obtaining, by one or more computing devices, a set of training data comprising a plurality of training examples;
“machine-learned model” (stored in a computer-readable media) configured to receive and process…data input…to generate…likelihood values…;
“downstream system components” ; one or more “processors”; one or more “computer readable media”
With regard the claimed ‘obtaining’ step, this is nothing more than collecting data for use by the abstract idea. Accordingly, this step amounts to extra-solution activity and is not indicative of an integration into a practical application. See MPEP 2106.05(g).
With regards to the claimed models, these are recited at a high of generality without any details regarding structure or how they operate to achieve the claimed functions (i.e. generating likelihood scores). Accordingly, these provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). Notably, MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. In addition, the claimed models also merely indicate a field of use or technological environment in which the judicial exception is performed. See MPEP 2106.05(h). As such, these additional elements fail to add an inventive concept to the claims.
With regard to the claimed processor, components, and computer-readable media, these features are generically recited and merely used as tools to obtain information and perform the abstract idea. Moreover, applicant is reminded that “generic computer components such as a computer and database do not satisfy the inventive concept requirement.” See MPEP 2106.05(f) and 2106.05(h). In addition, the courts have explained that the use of generic computer elements do not alone transform an otherwise abstract idea into patent-eligible subject matter. See DDR Holdings (Fed. Cir. 2014). Therefore, the additionally recited steps/elements amount to insignificant extra-solution activity that does not apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Even when viewed in combination, these additional steps/elements do not integrate the recited judicial exception into a practical application. See MPEP 2106.04(d)(1) for a list of considerations when evaluating whether additional elements integrate a judicial exception into a practical application. [Step 2A, Prong 2: NO].
C. Guidance Step 2B:
This part of the eligibility analysis evaluates whether the claim as a whole amount to significantly more than the recited exception i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
As discussed above, the non-abstract steps/elements amount to nothing more than insignificant extra-solution activity. A review of the specification teaches a plurality of routine and conventional computer elements and devices for storing and processing data [Figure 1, 0063-0066] as well as conventional or commercially available deep learning/generative models, e.g. Glow and PixelNCC [page 7]. In other words, the specification does not provide any evidence to suggest applicant has invented the claimed models or processors being used. In addition, as set forth above, the courts have explained that the use of generic computer elements do not alone transform an otherwise abstract idea into patent-eligible subject matter. See DDR Holdings (Fed. Cir. 2014). Therefore, even upon reconsideration, there is nothing unconventional with regards to the above non-abstract elements. See MPEP 2106.05(d)(Part II). Thus, the independent claim(s) as a whole do not amount to significantly more than the exception itself. Therefore, the claim(s) is/are not patent eligible. [Step 2B: NO].
D. Dependent Claims
Dependent claims 2-8, 11, 16-21, 25, 27-29 have also been considered under the two-part analysis but do not include additional steps/elements appended to the judicial exception that are sufficient to amount to significantly more than the judicial exception(s) for the following reasons. In particular, claims 2-8, 11, 16-21, 25, 27-29 are all entirely directed to limitations that further limit the specificity of the abstract idea or the nature of the data being used by the abstract idea. Accordingly, these claims are also directed to an abstract idea for the reasons set forth above (Step 2A, prong 1, analysis). Therefore, the instantly rejected claims are not drawn to eligible subject matter as they are directed to an abstract idea without significantly more.
Response to Arguments
Applicant’s arguments, filed 04/14/2026, have been fully considered but are not persuasive for the following reasons.
Applicant provides an extensive summary of the USPTO Memorandum from Charles Kim (Aug. 4th, 2025) and the revised MPEP sections in light of Ex Parte Desjardins. The examiner is highly familiar with these documents and applicant is thanked for reiterating its contents.
Applicant argues that the examiner has not met the burden for establishing prima facie unpatentability of the instant claims and failed to account for specific limitations detailing how the second machine learned model is trained (e.g. through perturbation of one or more first training examples). In response, as set forth in the Step 2A (prong 1) analysis, the examiner clearly identified claim limitations that constitute the judicial exception (i.e. the training, perturbing, inputting, and screening steps) and provided sufficient reasoning as to why these steps fall within the mental process and/or mathematical concept of abstract ideas. Accordingly, the examiner maintains that he has properly identified and explained the judicial exception in a manner that is consistent with the Step 2A, prong 1 analysis.
Applicant argues that the examiner has not given sufficient weight to the evidence on record demonstrating the claimed invention provides a technical solution to a technical problem, e.g. existing models can have failure modes related to at out- of-distribution (OOD) detection when they assign higher likelihoods to OOD inputs than in- distribution inputs due to confounding background statistics…The confounding effect of GC-content makes the likelihood less reliable as a score for OOD detection, because an OOD input may result in a higher likelihood than an in-distribution input." Applicant's Specification, at [0031], [0033]; "proposed likelihood ratio method can leverage both a semantic model that learns the semantic features of the in-distribution training examples and a background model that learns to correct for the background statistics of the training examples." Applicant's Specification, at [0020]. And, by using the improved OOD detection systems of the present disclosure to screen inputs prior to processing by downstream system components (e.g., a classifier), the unnecessary and undesirable processing of OOD inputs by such downstream components can be reduced. Thus, computing resources such as processor usage, memory usages, network bandwidth, etc., can be conserved. Applicant's Specification, at [0026].
In response, the MPEP is clear that the word "improvements" in the context of this consideration is limited to improvements to the functioning of a computer or any other technology/technical field, whether in Step 2A Prong Two or in Step 2B. MPEP 2106.04(d)(1). In this case, applicant has not provided any objective evidence to support the case that the claims result in an improvement to the functioning of a computer and no such evidence is provided in the specification. In fact, in at least one case, the improvements cited in the specification are not commensurate in scope with what is actually being claimed, e.g. confounding effects of GC-content makes the likelihood less reliable as a score for OOD detection. To evaluate an improvement to a computer or technical field, the specification must set forth an improvement in technology and the claim itself must reflect the disclosed improvement. See MPEP 2106.04(d)(1) and 2106.05(a). Applicant has failed to identify any steps/elements appended to the abstract idea that provide for a new technology (Step 2A, prong 2 or Step 2B) or provided any evidence of an unconventional combination of steps. Therefore, applicant is essentially arguing that the improvement is entirely in the realm of abstract ideas and that abstract idea is providing the improvement (by providing “better data”). However, Applicant is reminded that the claimed invention’s use of the ineligible concept to which it is directed (i.e. the abstract idea) cannot supply the inventive concept that renders the invention ‘significantly more’ than that ineligible concept.” BSG Tech LLC v. BuySeasons, Inc., 899 F.3d 1281, 1290 (Fed. Cir. 2018). Similarly, the courts have also instructed that “[t]he different use of a mathematical calculation, even one that yields different or better results, does not render patent eligible subject matter.” Board Of Trustees Of Leland Stanford Junior University, 991 F.3d 1245, 1251 (Fed. Cir. 2021).
As such, the claims do not recite an improvement to computer functionality and do not delineate steps through which the machine learning technology achieves an improvement. See, e.g. Ex Parte Desjardins. For example, "Ex parte Desjardins had claims drawn to the use of a machine learning model, trained on one task with a first set of data, and set parameter weights, then trained again using differing data on a different task, adjusting parameters and weights, while protecting performance of the first task. Further, in Desjardins, the retraining of the particular ML changed the structure of that ML in a way that provided "'[a]n improvement in the functioning of a computer, or an improvement to other technology or technical field,' as discussed in MPEP §§ 2106.04(d)(l) and 2106.05(a). Moreover, the independent claim in Ex parte Desjardins contained specific limitations as to how at least some aspects of the asserted improvements are achieved: "When evaluating the claim as a whole, we discern at least the following limitation of independent claim 1 that reflects the improvement: "adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task." We are persuaded that constitutes an improvement to how the machine learning model itself operates, and not, for example, the identified mathematical calculation." Ex parte Desjardins, p9. In contrast, the instant claims do not clearly set forth the link between the data gathered, the initial training of the models, the structure of the models, and how training (or perturbing) affects the structure to obtain the desired results or asserted improvement. For at least these reasons, absent any evidence to the contrary, the rejection is maintained.
Claim Rejections - 35 USC § 101- Lack of Utility
The following rejection is modified in view of applicant’s amendments.
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 101 because the claimed invention is not supported by either a substantial asserted utility or a well-established utility.
For an invention to be “useful” it must satisfy the utility requirement of section 101. The USPTO’s official interpretation of the utility requirement provides that the utility of an invention has to be (i) specific, (ii) substantial and (iii) credible. See MPEP § 2107 and Fisher, 421 F.3d at 1372, 76 USPQ2d at 1230. In this case, as discussed above, the claimed system/method results in “ screening…the data input from processing by one or more downstream system components based at least in part on a likelihood ratio value for the data input based at least in part on the first likelihood value and the second likelihood value.” One of ordinary skill in the art would recognize that “screening” is the evaluation or investigation of something as part of a methodical survey, to assess suitability for a particular role or purpose. In this case, however, the amended claims do not set forth any boundaries with regards to what the data is actually being screened for, e.g. some type of suitability, particular purpose, or otherwise. Accordingly, the claimed method/system is not associated with any particular real-world context of use and would require carrying out further research to identify or reasonably confirm a "real world" context of use. As such, the claims do not define "substantial utilities". The specification does teach, for example, correcting for background components to significantly improve the accuracy of a system that performs classification of inputs [0023] or reducing the number of inaccurate medical diagnoses [0024]. However, such limitations are not commensurate in scope with what is being claimed. For the above reasons, the claimed invention is not supported by a specific and/or substantial utility.
Claims 1-8, 11, 15-21, 25, 27-29 are also rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph. Specifically, since the claimed invention is not supported by either a specific, substantial, and credible asserted utility or a well-established utility for the reasons set forth above, one skilled in the art clearly would not know how to use the claimed invention. [See MPEP 2106.02].
Response to Arguments
Applicant’s arguments, filed 04/14/2026, have been fully considered but are moot in view of the modified rejection (which is necessitated by amendment).
Claim rejections - 35 USC § 112, 2nd Paragraph
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
The following rejections are necessitated by amendment.
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Claims that depend directly or indirectly from claim(s) 1 and 15 are also rejected due to said dependency.
Claims 1 and 15 recite “…to generate a first likelihood value for the data input” and “…to generate a second likelihood value for the data input.” In each case, it is unclear what the “likelihood value” represents with regards to “data input”, i.e. in what way does data input, per se, have a likelihood. For example, the artisan would recognize that likelihood refers to how well a sample provides support for particular values of a parameter in a model, or that a likelihood function (i.e. a likelihood) measures how well a statistical model explains observed data (by calculating the probability). However, this is not consistent with a likelihood value “for data input” and the specification does not provide any limiting definition that would serve to clarify the scope. Clarification is requested via amendment.
Claims 1 and 15 recite “screening…the data input from processing by one or more downstream system components based at least in part on a likelihood ratio value for the data input based at least in part on the first likelihood value and the second likelihood value.” This limitation is problematic for the following reasons. (1) It is unclear as to the metes and bounds of said screening such that the artisan would recognize what computational operation(s) is/are intended, i.e. in what way is data screened “from processing by one or more downstream system components”. One of ordinary skill in the art would recognize that “screening” is the evaluation or investigation of something as part of a methodical survey, to assess suitability for a particular role or purpose. In this case, however, the amended claims do not set forth any boundaries with regards to what the data is actually being screened for, e.g. some type of suitability, particular purpose, or otherwise. A review of the specification does not provide any limiting definition that would serve to clarify the scope. The specification does teach by identifying and screening OOD inputs at an earlier stage of the processing pipeline, the wasteful application of downstream resources to such OOD inputs can be reduced [0026]. However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is requested via amendment. (2) It is unclear as to the metes and bounds of “downstream system components”. A review of the specification does not provide any limiting definition that would serve to clarify what structural limitation is intended. Clarification is requested via amendment.
Claims 1 and 15 recite first and second “training examples”. It is unclear as to the metes and bounds of the term “examples” such that the artisan would know how to avoid infringement, i.e. in what way are ‘training examples’ different from ‘training data’. A review of the specification does not provide any limiting definition that would serve to clarify the scope. The specification does teach, for example, a training example is discrete (e.g., xa E A) (e.g., A = {A, C, G, T} for genomic sequences, A = {a, ...,255} for images, A = a dictionary of words, graphemes, phonemes, and/or n-grams for textual content, A = a set of possible amplitudes and/or frequencies for audio content, etc.)…” [0049]. However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is again requested via amendment.
Claim 1 recites “a…model trained on…training data comprising a plurality of…training examples which are samples of a…distribution.” Firstly, it is unclear as to the metes and bounds of the “machine-learned model” such that the artisan would recognize what structural limitation is intended. Stated differently, applicant has essentially chosen to claim training black box ‘models’ as a critical aspect of the invention (without defining the models or how they are trained). Secondly, it is unclear as to the metes and bounds of the term “trained” such that the artisan would recognize what computational operations are included or excluded. The specification provides a limited example of training, e.g. backwards propagation of errors [0067] as well as using noise to train background models [0047-0052]. However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. As such, by claiming a method for performing such a specialized function in a computer system without disclosing the internal structure of that system/processor in the form of an algorithm, the claim is indefinite. Clarification is again requested via amendment. Applicant has not provided any illuminating arguments or clarifying amendments that would serve to rectify the issue.
Claim 15 recites “training…a…machine-learned model using the set of…training data.” Firstly, it is unclear as to the metes and bounds of the “machine-learned model” such that the artisan would recognize what structural limitation is intended. Stated differently, applicant has essentially chosen to claim training black box ‘models’ as a critical aspect of the invention (without defining the models or how they are trained). Secondly, it is unclear as to the metes and bounds of the term “trained” such that the artisan would recognize what computational operations are included or excluded. The specification provides a limited example of training, e.g. backwards propagation of errors [0067] as well as using noise to train background models [0047-0052]. However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. As such, by claiming a method for performing such a specialized function in a computer system without disclosing the internal structure of that system/processor in the form of an algorithm, the claim is indefinite. Clarification is again requested via amendment. Applicant has not provided any illuminating arguments or clarifying amendments that would serve to rectify the issue.
Claims 1 and 15 recite “inputting…a data input into the…machine-learned model…to generate a…likelihood value for the data input.” In this case, the claims suggest the (intended use) act of generating a likelihood value without positively reciting any such step. Applicant is reminded that intended use recitations do not impose any limiting effect on the method as claimed. Accordingly, it is unclear what step(s) are required to practice the claimed limitation, e.g. inputting data into a model, generating likelihood values (using the model), both, or otherwise. Clarification is requested via amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 3, 4, 5, 8, 16, 17, 18, 19, 27 are rejected under 35 U.S.C. 103 as being unpatentable over Liang et al. (Enhancing The Reliability of Out-Of-Distribution Image Detection In Neural Networks, ICLR 2018, pp. 1-15).
Regarding claim(s) 1 and 15, Liang teaches methods for detecting out-of-distribution images in neural networks. In particular, Liang teaches obtaining training data and training a neural network to classify N classes [Section 3, Temperature Scaling, and Section 4.1], which reads on obtaining and training a first model as claimed. Liang teaches perturbing the input data (i.e. training data) to decrease the log-softmax score for the true label and force the neural network to make a wrong prediction [Section 3, Input Preprocessing]. Liang does not specifically teach training a second machine-learned model using a set of second training data that comprises the one or more second training examples, as claimed. However, Liang reasonably suggests this limitation because a new out-of-distribution model is trained using a combination of the trained input and pre-processed input data [Section 3, page 3]. Liang teaches inputting image data into the models, calculating the first and second log-softmax scores (i.e. likelihood values), and comparing the scores to a threshold [Section 3, page 3, OOD Detector, Section 4.2].
Liang does not specifically teach screening the data input from processing by one or more downstream system components based at least in part on a likelihood ratio value for the data input based at least in part on the first likelihood value and the second likelihood value. However, Liang reasonably suggests this limitation by calculating evaluation (i.e. screening) metrics to measure the model effectiveness that include FPR, ROC, and detection error [Sections 4.3, 4.4] and since one of ordinary skill in the art would recognize that metrics such as error rates include ratios. Liang does not specifically teach a processor and memory as claimed. However, Liang makes obvious these features since their method requires running computational neural network models that necessarily require a suitably programmed computer.
Regarding claim(s) 2, 3, 16, 17, Liang teaches inputting image data into the models, calculating the first and second log-softmax scores (i.e. likelihood values), and comparing the scores to a threshold [Section 3, page 3, OOD Detector, Section 4.2]. Regarding claim(s) 4, 18, Liang teaches classification data as OOD or not OOD [page 3]. Regarding claim(s) 5, 19, Liang teaches perturbing the input data (i.e. training data) by adding a perturbation (i.e. noise) parameter [Section 3, Input Preprocessing]. Regarding claim(s) 8, Liang teaches a neural network model [Section 3]. Regarding claim(s) 27, Liang teaches training using first and second loss functions [Section 3]. Therefore, it would have been prima facie obvious to one of ordinary skill in the art at the time the application was filed to carry out the claimed methods.
Prior Art Rejection of Indefinite Claims
In view of the indefiniteness and lack of clarity in the instant claims, as set forth in the 35 USC 112(b) rejection above, the Examiner has had difficulty in properly interpreting instant claims. However, to avoid piecemeal prosecution and to give applicant a better appreciation for relevant prior art if the claims are redrafted to avoid the 35 USC 112 rejections, the following prior art made of record and not relied upon is considered pertinent to applicant' s disclosure.
Goodfellow et al. ("Explaining and harnessing adversarial examples”, Published as a conference paper at ICLR 2015, pp.1-11), which teaches methods for analyzing machine learning models for misclassification based on harnessing adversarial examples. In particular, Goodfellow teaches obtaining digital images (represented as x) used as training data [Section 3], which broadly reads on obtaining training data comprising a first training example. Goodfellow teaches training a model and linearly perturbing the model comprising parameters for model input (x), an adversarial input (represented as ƞ) and a cost function, and generating an adversarial example [Sections 4 and 5, and Figure 2], which reads on training a model and perturbing first training examples to generate a second training example, as claimed.
Ren et al. (Proceedings of the 33rd International Conference on Neural Information Processing Systems; December 2019, Article No. 1317, Pages 14707 – 14718) teaches methods of OOD detection based on deep generative models and likelihood scores.
Conclusion
No claims are allowed.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PABLO S WHALEY whose telephone number is (571)272-4425. The examiner can normally be reached between 1pm-9pm EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Anita Coope can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PABLO S WHALEY/Primary Examiner, Art Unit 3619
Prosecution Insights