DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restriction
Applicants’ election without traverse of Species A(i), drawn to claims 6, 7, 20, 21, filed 09/16/2025 is acknowledged. Claims 9, 10, 24 are hereby withdrawn from further consideration pursuant to 37 CFR 1.142(b), as being drawn to a nonelected invention, there being no allowable generic or linking claim.
Status of Claims
Claims 1-8, 11, 15-21, 25, 27-29 are under examination.
Claims 9, 10, 24 are withdrawn.
Claims 12, 13, 14, 22, 23, 26, 30 are cancelled.
Priority
Applicant’s claim for the benefit of priority under 35 U.S.C. 119(a)-(d) is acknowledged. This application claims the right of priority under 35 U.S.C. 371 to International Application No. PCT/US2020/034475, filed May 26, 2020, which claims priority to and the benefit of United States Provisional Patent Application Number 62/857,774, filed June 5, 2019.
Drawings
The drawings filed 12/03/2021 are acceptable.
Information Disclosure Statement(s)
The two (2) information disclosure statement (IDS) document(s) submitted are in compliance with the provisions of 37 CFR 1.97. Accordingly, the IDS document(s) has/have been fully considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
The United States Patent and Trademark Office published revised guidance on the application of 35 U.S.C. § 101. USPTO’s 2019 Revised Patent Subject Matter Eligibility Guidance (“Guidance”). Under the Guidance, in determining what concept the claim is “directed to,” we first look to whether the claim recites:
(1) any judicial exceptions, including certain groupings of abstract ideas (i.e., mathematical concepts, certain methods of organizing human activity such as a fundamental economic practice, or mental processes) (Guidance Step 2A, Prong 1); and
(2) additional elements that integrate the judicial exception into a practical application (see MPEP § 2106.05(a)-(c), (e)-(h)) (Guidance Step 2A, Prong 2).
Only if a claim (1) recites a judicial exception and (2) does not integrate that exception into a practical application, do we then look to whether the claim contains an “‘inventive concept’ sufficient to ‘transform’” the claimed judicial exception into a patent-eligible application of the judicial exception. Alice, 573 U.S. at 221 (quoting Mayo, 566 U.S. at 82). In so doing, we thus consider whether the claim:
(3) adds a specific limitation beyond the judicial exception that are not “well-understood, routine and conventional in the field” (see MPEP § 2106.05(d)); or 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019).
(4) simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.(Guidance Step 2B). See Guidance, 84 Fed. Reg. at 54-56.
Guidance Step 1:
The instant invention (claim 15 being representative) is directed to a method and system for performing out-of-distribution detection (i.e. a process). Thus, the claims are directed to one of the statutory categories of invention. MPEP 2106.03.
A. Guidance Step 2A, Prong 1
The Revised Guidance instructs us first to determine whether any judicial exception to patent eligibility is recited in the claim. The Revised Guidance identifies three judicially-excepted groupings identified by the courts as abstract ideas: (1) mathematical concepts, (2) certain methods of organizing human behavior such as fundamental economic practices, and (3) mental processes. Regarding claim(s) 1, the claimed steps that are part of the abstract idea are as follows:
Claim 15
training, by the one or more computing devices, a machine-learned generative semantic model using the set of in-distribution training data;
perturbing, by the one or more computing devices, one or more in-distribution training examples of the plurality of in-distribution training examples to generate one or more background training examples;
training, by the one or more computing devices, a machine-learned generative background model using a set of background training data that comprises the one or more background training examples;
inputting, by the one or more computing devices, a data input into the machine-learned generative semantic model that has been trained on the set of in-distribution training data;
receiving, by the one or more computing devices, a first likelihood value for the data input as an output of the machine-learned generative semantic model;
inputting, by the one or more computing devices, the data input into the machine-learned generative background model that has been trained on the set of background training data;
receiving, by the one or more computing devices, a second likelihood value for the data input as an output of the machine-learned generative background model;
determining, by the one or more computing devices, a likelihood ratio value for the data input based at least in part on the first likelihood value generated by the machine-learned generative semantic model and the second likelihood value generated by the machine-learned generative background model;
predicting whether the data input is out-of-distribution based at least in part on the likelihood ratio value.
Mental Processes
Under the broadest reasonable interpretation, the above italicized steps require training two nominally recited “generative” models using data (in-distribution and background); inputting “data input” into said models; calculating likelihood values using the trained models and “data input”; and calculating a “likelihood ratio” to determine if the input data is “out-of-distribution” (i.e. making a decision). Notably, claims do not impose any boundaries on how the italicized functions are actually being achieved. In addition, the instant specification describes mathematical models/algorithms associated with the claimed models as well as specific equations for performing the above functions [see at least pages 6-8]. As such, the specification provides sufficient evidence that the claims are directed to an abstract idea since the specific descriptions provided for accomplishing these tasks include only data reception and analysis. Accordingly, but for the recitation of processors, the above steps clearly fall within the mental process groupings of abstract ideas because they cover concepts performed in the human mind, including observation, evaluation, judgment, and opinion. See MPEP 2106.04(a)(2), subsection III [Step 2A, Prong 1: YES].
Mathematical Concept
As summarized above, the claimed method requires training two nominally recited “generative” models using data (in-distribution and background); inputting “data input” into said models; calculating likelihood values using the trained models and “data input”; and calculating a “likelihood ratio” to determine if the input data is “out-of-distribution” (i.e. making a decision). Under the BRI, a model is understood to be a relationship between output or response variables and its corresponding input or independent variables in a dataset. In this case, the “generative models” being used are nominally recited (without any specificity with regards to structure) and interpreted as mathematical concepts and/or a mathematical relationship. This position is further supported by the specification, which teaches mathematical models/algorithms associated with the claimed “generative” models [see at least pages 6-8]. Additionally, the artisan would recognize that steps for determining likelihood values and ratios are mathematical calculations (since likelihood values much be calculated). This position is supported by the specification, which clearly describes specific equations for performing the above functions [see at least pages 7-8]. Similar to the ineligible claims at issue for In re: Board of Trustees of the Leland Stanford Junior University, 991 F.3d 1245 (Fed. Cir. 2021), the instant claims are written effectively as a method for mathematically manipulating or relating data to ascertain additional data. While no specific equation is being claimed, Applicant is reminded that there is no particular word or set of words that indicates a claim recites a mathematical calculation. See MPEP 2106.04(a)(2). Therefore, when read in light of applicant’s own specification, the claims are directed to mathematical concepts. See MPEP 2106.04 and 2106.05(II). [Step 2A, Prong 1: YES].
B. Guidance Step 2A, Prong 2
This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional steps/elements recited in the claim beyond the judicial exception, and (2) evaluating those additional steps/elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d).
In this case, the additional steps/elements recited in the claim beyond the judicial exception are as follows:
obtaining, by one or more computing devices, a set of in-distribution training data comprising a plurality of in-distribution training examples;
“generative semantic model” (stored in a computer-readable media) configured to receive and process…data input…to generate…likelihood values…;
“generative background model” (stored in a computer-readable media) configured to receive and process…data input…to generate…likelihood values…;
one or more “processors”; one or more “computer readable media”
With regard the claimed ‘obtaining’ step, this is nothing more than collecting data for use by the abstract idea. Accordingly, this step amounts to extra-solution activity and is not indicative of an integration into a practical application. See MPEP 2106.05(g).
With regards to the “generative” models, these are recited at a high of generality (without any details regarding structure) and are interpreted as generic computer components used as tools to perform the abstract idea and/or merely indicate instructions to implement the abstract idea on a computer. See MPEP 2106.05(h) and MPEP 2106.05(f). As such, these additional elements fail to add an inventive concept to the claims.
With regard to the claimed processor and computer-readable media, these features are generically recited and merely used as tools to obtain information and perform the abstract idea. Moreover, applicant is reminded that “generic computer components such as a computer and database do not satisfy the inventive concept requirement.” See MPEP 2106.05(f) and 2106.05(h). In addition, the courts have explained that the use of generic computer elements do not alone transform an otherwise abstract idea into patent-eligible subject matter. See DDR Holdings (Fed. Cir. 2014). Therefore, the additionally recited steps/elements amount to insignificant extra-solution activity that does not apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Even when viewed in combination, these additional steps/elements do not integrate the recited judicial exception into a practical application. See MPEP 2106.04(d)(1) for a list of considerations when evaluating whether additional elements integrate a judicial exception into a practical application. [Step 2A, Prong 2: NO].
C. Guidance Step 2B:
This part of the eligibility analysis evaluates whether the claim as a whole amount to significantly more than the recited exception i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
As discussed above, the non-abstract steps/elements amount to nothing more than insignificant extra-solution activity. A review of the specification teaches a plurality of routine and conventional computer elements and devices for storing and processing data [Figure 1, 0063-0066] as well as conventional or commercially available deep learning/generative models, e.g. Glow and PixelNCC [page 7]. In other words, the specification does not provide any evidence to suggest applicant has invented the claimed models or processors being used. In addition, as set forth above, the courts have explained that the use of generic computer elements do not alone transform an otherwise abstract idea into patent-eligible subject matter. See DDR Holdings (Fed. Cir. 2014). Therefore, even upon reconsideration, there is nothing unconventional with regards to the above non-abstract elements. See MPEP 2106.05(d)(Part II). Thus, the independent claim(s) as a whole do not amount to significantly more than the exception itself. Therefore, the claim(s) is/are not patent eligible. [Step 2B: NO].
D. Dependent Claims
Dependent claims 2-8, 11, 16-21, 25, 27-29 have also been considered under the two-part analysis but do not include additional steps/elements appended to the judicial exception that are sufficient to amount to significantly more than the judicial exception(s) for the following reasons. In particular, claims 2-8, 11, 16-21, 25, 27-29 are all entirely directed to limitations that further limit the specificity of the abstract idea or the nature of the data being used by the abstract idea. Accordingly, these claims are also directed to an abstract idea for the reasons set forth above (Step 2A, prong 1, analysis). Therefore, the instantly rejected claims are not drawn to eligible subject matter as they are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 101- Lack of Utility
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 101 because the claimed invention is not supported by either a substantial asserted utility or a well-established utility.
For an invention to be “useful” it must satisfy the utility requirement of section 101. The USPTO’s official interpretation of the utility requirement provides that the utility of an invention has to be (i) specific, (ii) substantial and (iii) credible. See MPEP § 2107 and Fisher, 421 F.3d at 1372, 76 USPQ2d at 1230.
In this case, as discussed above, the claimed system/method results in “predicting whether the data input is out-of-distribution based at least in part on the likelihood ratio value.” However, the claim does not impose any boundaries in the nature of the ‘samples’, “examples”, or the ‘data input’ used by the claimed method/system, i.e. these are purely theoretical. Moreover, this step broadly reads on predicting whether data is “unknown” (absent any limiting definition in the specification for “out-of-distribution”). Accordingly, the claimed method/system is not associated with any particular real-world context of use with regards to type of data or the purpose of the predictive evaluation. As such, the claimed invention would require or constitute carrying out further research to identify or reasonably confirm a "real world" context of use (e.g. evaluating the effectiveness of a model for predicting a medical diagnosis) and therefore does not define "substantial utilities". It is noted that the specification does teach that input data sets could be nucleic acid sequences or sensor data (e.g. images or sound) [0002], as well as training a classifier on “existing bacterial classes” wherein “OOD inputs can also be the contaminations from the bacteria's host genomes such as human, plant, fungi, etc., which also need to be detected and excluded from predictions” [0004, 0005]. However, such limitations are not commensurate in scope with what is being claimed. Therefore, the claimed invention is broader than the disclosure such that it does not require a practical application. For the above reasons, the claimed invention is not supported by a specific and/or substantial utility.
Claims 1-8, 11, 15-21, 25, 27-29 are also rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph. Specifically, since the claimed invention is not supported by either a specific, substantial, and credible asserted utility or a well-established utility for the reasons set forth above, one skilled in the art clearly would not know how to use the claimed invention. [See MPEP 2106.02].
Claim rejections - 35 USC § 112, 2nd Paragraph
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-8, 11, 15-21, 25, 27-29 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Claims that depend directly or indirectly from claim(s) 1 and 15 are also rejected due to said dependency.
Claim 1 recites a “machine-learned generative semantic model trained on a set of in-distribution training data comprising a plurality of in-distribution training examples which are samples of a distribution...”. The above italicized phrase is problematic for the following reasons. (1) It is unclear as to the metes and bounds of the claimed “machine-learned generative semantic model” such that the artisan would recognize what structural limitation is intended. One of ordinary skill in the art would recognize that a “machine learning model” encompasses various categories with different mathematical structures, e.g. regression, non-parametric, decision trees, SVM, supervised, unsupervised, PCA, etc., based on different mathematical parameters which describe the relationships between data. In this case, however, the specification does not provide any limiting definition and the claims do not recite any particular variables or parameters associated with said model that would serve to clarify what mathematical structure (or algorithm) is intended. Clarification is requested via amendment.
(2) It is unclear as to the metes and bounds of “in-distribution training examples” such that the artisan would know how to avoid infringement, i.e. what type of data are encompassed. A review of the specification does not provide any limiting definitions for these terms that would serve to clarify the scope. The specification teaches and in-distribution dataset V of (x, y) pairs sampled from the distribution p*(x, y), where x is the extracted feature vector or raw input [0028]. However, this is not commensurate in scope with what is claimed and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is requested via amendment.
Claim 1 recites a “machine-learned generative background model trained on a set of background training data comprising a plurality of background training examples, one or more background training examples of the plurality of background training examples generated through perturbation of one or more in-distribution training examples of the plurality of in-distribution training examples…”. The above italicized phrase is problematic for the following reasons. (1) It is unclear as to the metes and bounds of the claimed “machine-learned generative background model”. One of ordinary skill in the art would recognize that a “machine learning model” encompasses various categories with different mathematical structures, e.g. regression, non-parametric, decision trees, SVM, supervised, unsupervised, PCA, etc., based on different mathematical parameters which describe the relationships between data. In this case, however, the specification does not provide any limiting definition and the claims do not recite any particular variables or parameters associated with said model that would serve to clarify what mathematical structure (or algorithm) is intended. Clarification is requested via amendment.
(2) It is unclear what computational operations are encompassed by “generated through perturbation of one or more in-distribution training examples of the plurality of in-distribution training examples. A review of the specification does not describe, to any appreciable extent, any algorithms, equations, or prose equivalent that correspond to the claimed function. Clarification is requested via amendment.
(3) It is unclear as to the metes and bounds of the term “background training examples” such that the artisan would know how to avoid infringement, i.e. what is meant by ‘background’ and what type of data is actually encompassed by the examples. A review of the specification does not provide any limiting definitions for these terms that would serve to clarify the scope. The specification does teach input data sets could be nucleic acid sequences or sensor data (e.g. images or sound) [0002]. However, this does not clarify the issue since one of ordinary skill in the art of image analysis would recognize that “background” is associated with an area in an image being analyzed, for example, whereas one of ordinary skill in the art of genomics would not recognize what the ‘background’ is associated with in a nucleic acid sequence. Moreover, the instant claims do not obtain any image or sequence data and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is requested via amendment.
Claim 15 recites “training, by the one or more computing devices, a machine-learned generative semantic model using the set of in-distribution training data.” It is unclear as to the metes and bounds of the term “training” such that the artisan would recognize what computational operations are included or excluded. Such generic functional claim language amounts to descriptions of problems to be solved and covers all means or methods of performing the claimed function. The specification provides a limited example of training, e.g. backwards propagation of errors [0067] However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is requested via amendment. For purposes of examination, this limitation is broadly interpreted as inputting data into said model.
Claim 15 recites “perturbing, by the one or more computing devices, one or more in-distribution training examples of the plurality of in-distribution training examples to generate one or more background training examples.” It is unclear as to the metes and bounds of “perturbing” such that the artisan would recognize what computational operations are included or excluded. A review of the specification provides limited examples of perturbation, e.g. by adding noise to training example or adding regularization [0047, 0052]. However, examples are not limiting definitions and it is improper to import narrowing limitations into the claims. MPEP 2111.01. Clarification is requested via amendment.
Claim 15 recites “receiving, by the one or more computing devices, a first likelihood value for the data input as an output of the machine-learned generative semantic model”. In this case, there is no previous step in the claim for actually calculating a “likelihood value” nor are such values an inherent property of the input data as claimed. Accordingly, there is lack of antecedent basis for “a first likelihood value”. As a result, it is additionally unclear what is meant by “likelihood value for the data input as an output of the…model”, i.e. does the likelihood represent model accuracy, data accuracy, or otherwise. Clarification is requested via amendment.
Claim 15 recites “receiving, by the one or more computing devices, a second likelihood value for the data input as an output of the machine-learned generative background model.” Similarly to the above rejection, there is no previous step in the claim for actually calculating a “likelihood value” nor are such values an inherent property of the input data as claimed. Accordingly, there is lack of antecedent basis for “a second likelihood value”. As a result, it is additionally unclear what is meant by “likelihood value for the data input as an output of the…model”, i.e. does the likelihood represent model accuracy, data accuracy, or otherwise. Clarification is requested via amendment.
Prior Art Rejection of Indefinite Claims
In view of the indefiniteness and lack of clarity in the instant claims, as set forth in the 35 USC 112(b) rejection above, the Examiner has had difficulty in properly interpreting instant claims. However, to avoid piecemeal prosecution and to give applicant a better appreciation for relevant prior art if the claims are redrafted to avoid the 35 USC 112 rejections, the following prior art made of record and not relied upon is considered pertinent to applicant' s disclosure.
Shalev et al. (32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada, pp.1-11) teaches methods for out-of-distribution (OOD) detection in neural networks.
Hendrycks (A Baseline For Detecting Misclassified And Out-Of-Distribution Examples In Neural Networks, ICLR 2017, pp. 1-12), which teaches computational methods for detecting Out-of-Distribution examples in neural networks. In particular, Hendrycks teaches obtaining in-distribution training data comprising a plurality of training and test datasets, and training a semantic learning model [Section 3.2.1 and Table 4].
Bevandic et al. (Discriminative Out-Of-Distribution Detection For Semantic Segmentation, Oct 2018, pp. 1-18), which teaches methods for discriminative detection of OOD pixels in input data, training a dedicated OOD model which discriminates the primary training set from a much larger ”background” dataset which approximates the variety of the visual world.
Liang et al. (Enhancing The Reliability of Out-Of-Distribution Image Detection In Neural Networks, ICLR 2018, pp. 1-15), which teaches methods for detecting out-of-distribution images in neural networks. Liang teaches mathematically defining two distinct data distributions (Px and Qx) defined on an image space X, wherein Px and Qx represent in-distribution and out-distribution data and wherein a neural network is trained on Px [Section 2], which reads on obtaining in-distribution training data.
Conclusion
No claims are allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PABLO S WHALEY whose telephone number is (571)272-4425. The examiner can normally be reached between 1pm-9pm EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Anita Coope can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PABLO S WHALEY/Primary Examiner, Art Unit 3619