DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 2/27/2023. Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 2/27/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
The last sentence of paragraph 49 recites “Disclosed embodiment utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” This recitation is grammatically incorrect and should read “Disclosed embodiments utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” Appropriate correction is required.
Claim Objections
Claims 8-11 and 14-17 are objected to because of the following informalities:
Claim 8 recites “the at least one high-density regions”. This recitation is grammatically incorrect and should read “the at least one high-density region . Appropriate correction is required.
In independent claim 14, the word “and” is missing between “a processor;” and “a memory”. Appropriate correction is required.
Claims 9-11, which each depend directly or indirectly from claim 8, are objected to based on their respective dependencies from claim 8.
Also, claims 15-17, which each depend directly from claim 14, are objected to based on their respective dependencies from claim 14.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Independent claims 1, 14 and 18 each recite “at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data.” (see, lines 4-5 of claim 1, and lines 6-7 of claims 14 and 18). The term “high-density region” is a relative term which renders the claims indefinite. The term “high-density region" is not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In particular, it is unclear what metrics are used for ascertaining the requisite degree of density or what range or amount of densities or density values are covered by the term “high-density” in the phrase “high-density region”. Aside from merely repeating the claim language in paragraphs 4-6 and mentioning general examples in stating “embodiments utilize deep learning to estimate high-density regions of multidimensional, multivariate, and/or multimodal data”, “In embodiments, the CycleGAN is used to identify high-density regions in multidimensional input data.”, “In embodiments, the autoencoder is used to identify high-density regions in multidimensional input data.” and “estimating high-density regions in multidimensional input data of dimension N, where N has a value greater than or equal to 5 is challenging. Disclosed embodiment utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” (see, paragraphs 16, 44, 46 and 48), Applicant’s specification fails to describe, much less define what constitutes a “high-density region”. As such, the specification does not explicitly define what is meant by the recited “high-density region” or provide a standard for ascertaining the requisite degree of the term “high” or density of the claimed “high-density region”. Applicant’s specification also does not provide a standard for ascertaining the requisite degree of density or range of density values in the term “high-density region” recited in claims 1, 14 and 18.
Therefore, one of ordinary skill in the art would not be able to ascertain what “at least one high-density region” would encompass. See MPEP § 2173.05(b). For the purposes of determining patent eligibility and comparison with the prior art, the examiner is interpreting the term “at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data” as one or more regions of input data having a data volume value, measurement or metric exceeding a minimum data volume value, measurement or metric threshold value. Appropriate correction is required.
Also, claims 2-13, 15-17 and 19-20, which depend directly or indirectly from claims 1, 14 and 18, respectively, are rejected under 35 U.S.C. 112(b) as being indefinite under the same rationale as claims 1, 14 and 18.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Examiner’s Note: Independent claim 18 recites “A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor”. According to the original speciation of the applicant, the utilization of computer readable storage media is limited to a non-transitory computer storage medium [i.e., Paragraph 21 discloses “A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media.”].
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding independent claims 1, 14 and 18, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, corresponding to a process, claim 14 is directed to a device comprising a processor and a memory, corresponding to a machine, and claim 18 is directed to a computer program product comprising a computer readable storage medium, corresponding to an article of manufacture, which are each one of the statutory categories.
Step 2A Prong One Analysis: The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including an observation, evaluation, judgment, opinion) based on mathematical concepts (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations).
The claims recite, using respective similar language, the following limitations:
estimating at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data1;
performing a statistical drift detection test on the at least one high-density region; and
in response to detecting a drift, executing at least one mitigation action, wherein the at least one mitigation action includes automatic model retraining - under their broadest reasonable interpretation (BRI), in light of the specification, these estimating and mitigation limitations encompass a mental processes of estimating high-density region and mitigating detected data drift by retraining a generically-recited model responsive to detecting data drift (i.e., evaluation/judgement/opinion to compare a data volume with a threshold based on observed data volume in observed/received input data and evaluation for mitigation needed to respond to observed drift) based on a mathematical concept (mathematical calculations – performing a statistical drift detection test on the at least one high-density region - see, e.g., paragraphs 49 and 58 of the specification disclosing that “a statistical drift detection test is performed in the high-density regions. In embodiments, the drift detection is performed using a Kolmogorov-Smirnov (KS) test … The KS test quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution, or between the empirical distribution functions of two samples.” and “The high-density regions are then analyzed with a mathematical test such as a KS test in order to make a drift detection determination.”).
MPEP 2106.04(a)(2)(II) provides “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
MPEP 2106.04(a)(2)(II) further provides “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea).”
Therefore, the claims recite mental processes based on mathematical concepts.
These steps and operations cover performance of the limitations in the mind (i.e., observation of input data and evaluation, judgement, opinion to estimate a high-density region and mitigate detected data drift by retraining a generically-recited model in response to detecting data drift based on a mathematical concept (mathematical calculations).
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind and mathematical relationships, mathematical formulas or equations, or mathematical calculations but for the recitation of generic computer components (i.e., the generically-recited “neural network” and “model”) then they fall within the “Mental Processes” and “Mathematical concepts” groupings of abstract ideas. Accordingly, claims 1, 14 and 18 recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claims recites these additional elements: A computer-implemented method for monitoring performance of a neural network (claim 1), An electronic computation device comprising: a processor; a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to: <perform operations> (claim 14) and A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to: <perform operations> (claim 18).
The above-noted additional elements in the claims amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f). Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “neural network”, “electronic computation device comprising: a processor; a memory” and “electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor”) cannot meaningfully integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Regarding the “model” and “neural network” no details of the model or neural network are recited, and the model and neural network are recited at a high level of generality and the model and network can be constructed by hand with pen and paper. Aside from repeating the claim language in paragraphs 4-6, and providing general examples in paragraphs 39, 48 and 58 in stating “machine learning system 217 can include, but is not limited to, a convolutional neural network (CNN), Support Vector Machine (SVM), Decision Tree, Recurrent Neural Network (RNN), Long Short Term Memory Network (LSTM), Radial Basis Function Network (RBFN), Multilayer Perceptron (MLP), and/or other suitable neural network type”, “Disclosed embodiment utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” and “a neural network, such as a deep learning neural network, is used to identify one or more high-density regions in the input data.”, applicant’s specification does not further define the claimed “model” or “neural network”. Thus, the claimed “model” and “neural network”, under the BRI, in light of the specification, could be any data model and neural network, which could be constructed and updated/retrained by hand with pen and paper. That is, the “model” and “neural network” limitations give the indication that the model and neural network can be constructed by hand with pen and paper.
The model and neural network are recited at a high level of generality and therefore are being interpreted as performing a mental process based on a mathematical concept on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
The claims also recite, using respective similar language, the additional limitation: obtaining input data, wherein the input data comprises multidimensional data. This is insignificant extra-solution activity that does not add a meaningful limitation to the above-noted abstract idea (mental processes based on mathematical concepts) specified in these claims because “obtaining input data” is mere data gathering (i.e., receiving provided/transmitted input data) (See MPEP § 2106.05(g)).
Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.04(d).
The claims are directed to an abstract idea.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Receiving and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitation of “obtaining input data, wherein the input data comprises multidimensional data” is the well-understood, routine, conventional activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 2, 15 and 19, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method as depending from claim 1, claim 15 is directed to a device as depending from claim 14, and claim 19 is directed to a computer program product as depending from claim 18, thus the analysis for patent eligibilities of claims 1, 14 and 18 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, “wherein the drift detection test comprises a Kolmogorov-Smirnov test.”
This limitation does nothing to alter the fundamental nature of the claims as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what determining the drift detection test includes, i.e., “a Kolmogorov-Smirnov test.”
In light of the specification, (see, e.g., paragraphs 49 and 58 of the specification disclosing that “a statistical drift detection test is performed in the high-density regions. In embodiments, the drift detection is performed using a Kolmogorov-Smirnov (KS) test … The KS test quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution, or between the empirical distribution functions of two samples.” and “The high-density regions are then analyzed with a mathematical test such as a KS test in order to make a drift detection determination.”), this limitation is directed to a mathematical concept and encompasses a mathematical calculation.
Dependent claims 2, 15 and 19, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims covers a mathematical concept.
Thus, this limitation does nothing to alter the analysis of claims 1, 14 and 18.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
There are no additional elements to integrate the abstract idea into a practical application or to impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 and the “electronic computation device” comprising a processor of base claims 14 and 18) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claims 3 and 16, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method as depending from claim 1 and claim 16 is directed to a device as depending from claim 14, thus the analysis for patent eligibilities of claims 1 and 14 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, “wherein the drift detection test comprises a Chi-squared test.”
This limitation does nothing to alter the fundamental nature of the claims as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what determining the drift detection test includes, i.e., “a Chi-squared test.”
In light of the specification, (see, e.g., paragraph 50 of the specification disclosing that “another statistical test, such as the Chi-squared test 336 may be used instead of, or in addition to, the KS test. … The chi-square statistic compares the size of any discrepancies between the expected results and the actual results, given the size of the sample and the number of variables in the relationship.”), this limitation is directed to a mathematical concept and encompasses a mathematical calculation.
Dependent claims 3 and 16, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims covers a mathematical concept.
Thus, this limitation does nothing to alter the analysis of claims 1 and 14.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
There are no additional elements to integrate the abstract idea into a practical application or to impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 and the “electronic computation device” comprising a processor of base claim 14) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claim 4, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein the at least one mitigation action further includes pausing the neural network.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the mitigation action includes, i.e., pausing the generically-recited “neural network.”
Dependent claim 4, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of mitigating detected data drift by halting, stopping or pausing operation of a generically-recited neural network.
Thus, this limitation does nothing to alter the analysis of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
Regarding the “neural network” no details of the neural network are recited, and the neural network are recited at a high level of generality and the model and network can be constructed by hand with pen and paper. Aside from repeating the claim language in paragraphs 4-6, and providing general examples in paragraphs 39, 48 and 58 in stating “machine learning system 217 can include, but is not limited to, a convolutional neural network (CNN), Support Vector Machine (SVM), Decision Tree, Recurrent Neural Network (RNN), Long Short Term Memory Network (LSTM), Radial Basis Function Network (RBFN), Multilayer Perceptron (MLP), and/or other suitable neural network type”, “Disclosed embodiment utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” and “a neural network, such as a deep learning neural network, is used to identify one or more high-density regions in the input data.”, applicant’s specification does not further define the claimed “neural network”. Thus, the claimed “neural network”, under the BRI, in light of the specification, could be any data neural network, which could be constructed and paused/halted by hand with pen and paper.
The neural network is recited at a high level of generality and therefore is being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Merely asserting that a judicial exception is to be carried out on a generic computer cannot meaningfully integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” of base claim 1 and “neural network” do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claim 5, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein the at least one mitigation action includes issuing a data drift alert.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the mitigation action includes, i.e., issuing a data drift alert.
Dependent claim 5, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of issuing an alert message or communication (i.e., a data drift alert).
Thus, this limitation does nothing to alter the analysis of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites the additional element: “issuing a data drift alert.”
This is an insignificant extra-solution activity that is not integrated into the claim as a whole and does not add a meaningful limitation to the above-noted mental process specified in this claim. That is, “issuing a data drift alert” amounts to necessary data outputting (See MPEP § 2106.05(g)).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Also, merely asserting that a judicial exception is to be carried out on a generic computer cannot meaningfully integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Receiving, transmitting and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitation of “issuing a data drift alert” is the well-understood, routine, conventional activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
This claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of the claim is not sufficient to amount to significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 6, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein detecting the drift comprises detecting a data drift.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what detecting the drift includes, i.e., “detecting a data drift.”
Dependent claim 6, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of evaluation/judgement/opinion to detect data drift based on observed data. Thus, this limitation does nothing to alter the analysis of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
There are no additional elements to integrate the abstract idea into a practical application or to impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claim 7, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein detecting the drift comprises detecting a concept drift.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what detecting the drift includes, i.e., “detecting a concept drift.”
Dependent claim 7, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of evaluation/judgement/opinion to detect a concept drift based on observed concepts. Thus, this limitation does nothing to alter the analysis of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
There are no additional elements to integrate the abstract idea into a practical application or to impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claims 8, 17 and 20, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method as depending from claim 1, claim 17 is directed to a device as depending from claim 14, and claim 20 is directed to a computer program product as depending from claim 18, thus the analysis for patent eligibilities of claims 1, 14 and 18 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, “wherein estimating the at least one high-density regions is performed with a second neural network.”
This limitation does nothing to alter the fundamental nature of the claims as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what estimating the at least one high-density region includes, i.e., using a generically-recited “second neural network.”
Dependent claims 8, 17 and 20, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims cover a mental process of estimating the at least one high-density region by using a generically-recited 2nd neural network.
Thus, this limitation does nothing to alter the analysis of claims 1, 14 and 18.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claims recites these additional elements: “performed with a second neural network” (claim 8), “wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to estimate the at least one high-density region using a second neural network” (claim 17) and “wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to estimate the at least one high-density region using a second neural network.” (claim 20).
The above-noted additional elements in the claims amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f). Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “second neural network” and “electronic computation device” comprising a “processor” and a “memory”) cannot meaningfully integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Regarding the “second neural network” no details of the 2nd neural network are recited, and the neural network are recited at a high level of generality and the model and network can be constructed by hand with pen and paper. Aside from providing general examples in paragraphs 45-46, 48 and 58 in stating “the data density machine learning system 232 comprises an autoencoder. An autoencoder is a type of neural network”, “In some embodiments, the autoencoder comprises a deep autoencoder, variational autoencoder, or denoising autoencoder. In embodiments, the autoencoder is used to identify high-density regions in multidimensional input data.”, Disclosed embodiment[s] utilize a dedicated neural network, such as a GAN 334, and/or autoencoder 344 to identify high-density regions.” and “a neural network, such as a deep learning neural network, is used to identify one or more high-density regions in the input data.”, applicant’s specification does not mention, let alone describe the claimed “second neural network”. Thus, the claimed 2nd “neural network”, under the BRI, in light of the specification, could be any data neural network, which could be constructed by hand with pen and paper.
The 2nd neural network is recited at a high level of generality and therefore is being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Merely asserting that a judicial exception is to be carried out on a generic computer cannot meaningfully integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically (i.e., with the recited “second neural network” and the “electronic computation device” comprising a processor and memory) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Regarding claim 9, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method as depending from claim 8, thus the analysis for patent eligibilities of claim 8 and of base claim 1 are incorporated herein.
Step 2A Prongs 1-2: The claim recites “wherein the second neural network comprises an autoencoder.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the generically-recited second neural network includes, i.e., “an autoencoder.”
Dependent claim 9, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea.
The “autoencoder” is recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., a system implementing the “autoencoder”) and amounts to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Also, the limitation “wherein the second neural network comprises an autoencoder” can be considered as “generally linking the use of judicial exception to a particular technological environment or field of use”. See MPEP 2106.05(h).
Thus, this limitation does nothing to alter the analysis of claims 1 and 8.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this dependent claim as a combination does not add anything further than the individual elements.
Mere instructions to apply the mental process electronically (i.e., with the recited “second neural network” and the “autoencoder”) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
Regarding claim 10, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method as depending from claim 8, thus the analysis for patent eligibilities of claim 8 and of base claim 1 are incorporated herein.
Step 2A Prongs 1-2: The claim recites “wherein the second neural network comprises a generative adversarial network.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the generically-recited second neural network includes, i.e., “a generative adversarial network.”
Dependent claim 10, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea.
The 2nd neural network and its “generative adversarial network” are recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., the “computer” of base claim 1 implementing the 2nd neural network and its constituent “generative adversarial network”) and amounts to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Also, the limitation “wherein the second neural network comprises a generative adversarial network” can be considered as “generally linking the use of judicial exception to a particular technological environment or field of use”. See MPEP 2106.05(h).
Thus, this limitation does nothing to alter the analysis of claims 1 and 8.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this dependent claim as a combination does not add anything further than the individual elements.
Mere instructions to apply the mental process electronically (i.e., with the recited “second neural network” and the “generative adversarial network”) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional element does not amount to significantly more than the judicial exception.
Regarding claim 11, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a method as depending from claim 10, thus the analysis for patent eligibilities of claim 8, 10 and of base claim 1 are incorporated herein.
Step 2A Prongs 1-2: The claim recites “wherein the second neural network comprises a CycleGAN neural network.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the generically-recited second neural network includes, i.e., “a CycleGAN neural network.”
Dependent claim 11, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea.
The 2nd neural network and its “CycleGAN neural network” are recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., the “computer” of base claim 1 implementing the 2nd neural network and its constituent “CycleGAN neural network”) and amounts to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Also, the limitation “wherein the second neural network comprises a CycleGAN neural network” can be considered as “generally linking the use of judicial exception to a particular technological environment or field of use”. See MPEP 2106.05(h).
Thus, this limitation does nothing to alter the analysis of claims 1, 8 and 10.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this dependent claim as a combination does not add anything further than the individual elements.
Mere instructions to apply the mental process electronically (i.e., with the recited “second neural network” and the “CycleGAN neural network”) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional element does not amount to significantly more than the judicial exception.
Regarding claim 12, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prongs 1-2: The claim recites “wherein the input data comprises multivariate data.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the obtained input data includes, i.e., “multivariate data.”
As noted above with regard to base claim 1, obtaining “input data” is insignificant extra-solution activity that does not add a meaningful limitation to the above-noted abstract idea (mental processes based on mathematical concepts) specified in the claim because “obtaining input data” is mere data gathering (i.e., receiving provided/transmitted input data) (See MPEP § 2106.05(g)). Thus, obtaining input data “wherein the input data comprises multivariate data” merely recites that the data gathering includes obtaining “multivariate data.”
Dependent claim 12, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea.
Thus, this limitation does nothing to alter the analysis of claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this dependent claim as a combination does not add anything further than the individual elements.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
This claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of the claim is not sufficient to amount to significantly more than the abstract idea.
Accordingly, at Step 2B, the additional element does not amount to significantly more than the judicial exception.
Regarding claim 13, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a method as depending from claim 12, thus the analysis for patent eligibilities of claim 12 and of base claim 1 are incorporated herein.
Step 2A Prongs 1-2: The claim recites “wherein the input data comprises multimodal data.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process based on or combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the obtained input data includes, i.e., “multimodal data.”
As noted above with regard to base claim 1, obtaining “input data” is insignificant extra-solution activity that does not add a meaningful limitation to the above-noted abstract idea (mental processes based on mathematical concepts) specified in the claim because “obtaining input data” is mere data gathering (i.e., receiving provided/transmitted input data) (See MPEP § 2106.05(g)). Thus, obtaining input data “wherein the input data comprises multimodal data” merely recites that the data gathering includes obtaining “multimodal data.”
Dependent claim 13, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea.
Thus, this limitation does nothing to alter the analysis of claims 1 and 12.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this dependent claim as a combination does not add anything further than the individual elements.
Mere instructions to apply the mental process electronically (i.e., with the recited “computer” and “neural network” of base claim 1 do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
This claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of the claim is not sufficient to amount to significantly more than the abstract idea.
Accordingly, at Step 2B, the additional element does not amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-9, 12 and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Tabet et al. (U.S. Patent Application Pub. No. 2023/0126842 A1, hereinafter “Tabet”) in view of non-patent literature Glazer et al. (“Learning High-Density Regions for a Generalized Kolmogorov-Smirnov Test in High-Dimensional Data”, 2012, cited in applicant’s IDS filed 02/27/2023, hereinafter “Glazer”).
With respect to claim 1, Tabet discloses the invention as claimed including a computer-implemented method for monitoring performance of a neural network (see, e.g., paragraphs 6, “The AI/ML model may include at least one of: … a Deep Neural Network”, 60, “Systems and methods described herein are directed to autonomic detection and correction of AI/ML model drift. … the AI/ML, models are then tested against multiple datasets to validate the desired level of AI/ML model performance.” and 79-80, “identify subsets of data that may be monitored to detect drift”, “data sets may be used to improve an AI/ML model, may be used to select an appropriate AI/ML model, or for other suitable purposes associated with AI/ML model training and validation to avoid drift in AI/ML model processing of data sets” [i.e., a computer-implemented method to monitor performance of an AI/ML neural network model]), comprising:
obtaining input data, wherein the input data comprises multidimensional data (see, e.g., paragraphs 118, “method 1000 begins when an incoming stream of input data 1001 is received by AI/ML model 1002 drift detection component”, 129, “input data is tagged may be derived from many sources … an image data file may have associated characteristics such as "encoding," "dimensions"” and 169, “processor 1601A applies an AI/ML model to incoming data 1602A (e.g., from a camera) to identify 2D objects, and incoming data 1602A is tagged with an indication of the detected 2D objects” [i.e., obtain incoming input data including 2D/multidimensional data]); …
performing a statistical drift detection test (see, e.g., paragraphs 61, “This drift detection process may be performed by statistical analysis of object detection and object recognition rates and performing statistical tests to determine if the model process is in statistically stationary operation.” and 178, “method 1700 detects drift during the operation of an AI/ML model” [i.e., performing a statistical drift detection test]) … ; and
in response to detecting a drift, executing at least one mitigation action, wherein the at least one mitigation action includes automatic model retraining (see, e.g., paragraph 60, “once drift is detected, the data which induced the drift is analyzed to create a set of rich metadata to identify characteristics of the drift-inducing data. This metadata is then compared to the broader data set to identify appropriate data to retrain models experiencing drift.”).
Although Tabet substantially discloses the claimed invention, Tabet is not relied on to explicitly disclose estimating at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data and performing a statistical drift detection test on the at least one high-density region.
In the same field, analogous art Glazer teaches estimating at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data2 (see, e.g., page 2, Sects 1-2, “we use a novel hierarchical minimum-volume [MV] sets estimator to estimate the set of high-density regions directly … We present here a novel method for approximate MV-sets estimation”, “estimating a hierarchical set of MV-sets in input space” and page 3, Sect. 2.1, “estimate high-density regions in the input space while bounding the number of examples in X lying outside these regions … our hierarchical MV-sets estimator … returns a set of decision functions … that satisfy both hierarchy and density requirements.” [i.e., estimating high-density regions having a data volume exceeding a MV/minimum-volume threshold requirement in the input space/data]); and performing a statistical drift detection test on the at least one high-density region (see, e.g., Abstract, “We propose an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data.” and page 6, Sect 4, “We first evaluated our test on concept-drift detection problems in data-stream classification tasks. Concept drifts are associated with distributional changes in data streams that occur due to hidden context … Statistical tests were evaluated” [i.e., perform a statistical drift/data change detection test on the high-density region]).
Tabet and Glazer are analogous art because they are both directed to techniques and systems for detecting and mitigating drift in artificial intelligence/machine learning (AI/ML) models and neural networks (see, e.g., Tabet, Abstract and paragraphs 59-60 and 95, and Glazer, Abstract and page 6, Sect. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Tabet with Glazer to provide “an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data.” and “a novel, hierarchical, minimum-volume sets estimator to represent the distributions to be tested.” that can be applied to “concept-drift detection problems in data-stream classification tasks.” where “Concept drifts are associated with distributional changes in data streams that occur due to hidden context [22] - changes of which the classifier is unaware.” (See, e.g., Glazer Abstract and page 6, Sect. 4). Doing so would have allowed Tabet to use Glazer’s test for detecting distributional change [i.e., drift] in high-dimensional data and Glazer’s hierarchical, minimum-volume sets estimator to “to detect changes in data streams, and the test is especially efficient in this context.” where Glazer’s test has “superiority over existing methods”, the “test is more accurate than density estimation approaches” and “since the estimation of MV-sets is simpler than density estimation, our test can achieve higher accuracy than approaches based on density estimation”, as suggested by Glazer (See, e.g., Glazer, Abstract, page 2, Sect 1 and page 5, Sect 2.2).
Regarding claim 4, as discussed above, Tabet in view of Glazer teaches the method of claim 1.
Tabet further discloses wherein the at least one mitigation action further includes pausing the neural network (see, e.g., FIG. 9 – depicting AI/ML model drift anomaly analysis flowchart with drift detection engine 906 that pauses operation of AI/ML model/neural network after decision by drift decision block 910 to provide time for analyst review 908, and paragraphs 92, “anomaly mitigation 608 may select different AI/ML models for use with the data set associated with the anomaly” and 110, “analyst review 908 may generate a notification to an analyst when AI/ML model drift has been detected, may track the resolution or analysis of drift by the analyst and may perform other suitable operations.” [i.e., mitigation includes pausing the AI/ML model/neural network to select a different AI/ML model and/or to provide time for analyst review after drift/anomaly detection]).
Regarding claim 5, as discussed above, Tabet in view of Glazer teaches the method of claim 1.
Tabet further discloses wherein the at least one mitigation action includes issuing a data drift alert (see, e.g., paragraphs 109-110, “drift detection engine 706 may determine whether drift has occurred that renders an AI/ML model inaccurate, and it may generate an output to drift decision block 910”, “perform analyst review notification … analyst review 908 may generate a notification to an analyst when AI/ML model drift has been detected” [i.e., the drift/anomaly mitigation action includes generating/issuing a drift notification/alert]).
Regarding claim 6, as discussed above, Tabet in view of Glazer teaches the method of claim 1.
Tabet further discloses wherein detecting the drift comprises detecting a data drift (see, e.g., paragraphs 4, “Embodiments of systems and methods for model prediction confidence utilizing drift … AI/ML drift generally occurs when an AI/ML model statistically trained on a specific dataset experiences performance degradation in terms of its ability to inference based on the data objects being presented to the model having changed (data drift)” and 116 “systems and methods described herein may utilize the output of data drift detection” [i.e., detecting drift includes detecting data drift]).
Regarding claim 7, as discussed above, Tabet in view of Glazer teaches the method of claim 1.
Tabet further discloses wherein detecting the drift comprises detecting a concept drift (see, e.g., paragraphs 4, “Embodiments of systems and methods for model prediction confidence utilizing drift … AI/ML drift generally occurs when an AI/ML model statistically trained on a specific dataset experiences performance degradation in terms of its ability to inference based on … the meaning of the data objects having changed (concept drift).”, 32, “Concept drift occurs when the input data remains the same yet its semantic_ meaning changes over time, thus also leading to a failure m inferences, predictions, and/or interpretations.” and 122 “in response to a determination that the AI/ML model is more susceptible to concept drift than data drift, the second weight may be greater than the first weight. Moreover, drift detection component 1004 may also enable a user to change at least one of the first or second weights” [i.e., detecting drift includes detecting concept drift]).
Regarding claim 12, as discussed above, Tabet in view of Glazer teaches the method of claim 1.
Although Tabet substantially discloses the claimed invention and Tabet discloses “AI/ML model 1002 may include, but are not limited to: … Multivariate Adaptive Regression Splines” (see, paragraph 119), Tabet is not relied on to explicitly disclose wherein the input data comprises multivariate data.
In the same field, analogous art Glazer teaches wherein the input data comprises multivariate data (see, e.g., page 2, Sect 1, “the statistical test introduced in this paper traces distributional changes in high dimensional data in general, it is effective in particular for change detection in data streams. Many real-world applications (e.g. process control) work in dynamic environments where streams of multivariate data are collected over time” and page 8, Sect 5, “Our proposed test belongs to a family of nonparametric tests for detecting change in multivariate data” [i.e., input data includes streams of multivariate data]).
Tabet and Glazer are analogous art because they are both directed to techniques and systems for detecting and mitigating drift in artificial intelligence/machine learning (AI/ML) models and neural networks (see, e.g., Tabet, Abstract and paragraphs 59-60 and 95, and Glazer, Abstract and page 6, Sect. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Tabet with Glazer to provide “an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data.” and “a novel, hierarchical, minimum-volume sets estimator to represent the distributions to be tested.” that can be applied to “concept-drift detection problems in data-stream classification tasks.” where “Concept drifts are associated with distributional changes in data streams that occur due to hidden context [22] - changes of which the classifier is unaware.” (See, e.g., Glazer Abstract and page 6, Sect. 4). Doing so would have allowed Tabet to use Glazer’s test for detecting distributional change [i.e., drift] in high-dimensional data and Glazer’s hierarchical, minimum-volume sets estimator to “to detect changes in data streams, and the test is especially efficient in this context.” where Glazer’s test has “superiority over existing methods”, the “test is more accurate than density estimation approaches” and “since the estimation of MV-sets is simpler than density estimation, our test can achieve higher accuracy than approaches based on density estimation”, as suggested by Glazer (See, e.g., Glazer, Abstract, page 2, Sect 1 and page 5, Sect 2.2).
With respect to independent claim 14, claim 14 is substantially similar to claim 1 and therefore is rejected on the same ground as claim 1, discussed above. In particular, claim 14 is a device claim with operations that correspond to the method steps of claim 1.
Tabet further discloses an electronic computation device comprising: a processor; a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to: <perform operations> (see, e.g., paragraphs 5, “an IHS [Information Handling System] may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: identify drift with respect to an AI/ML model” and 12, “a hardware memory device may have program instructions stored thereon that, upon execution, cause an IHS to: calculate drift of an AI/ML model” [i.e., an electronic device including a processor and memory coupled to the processor containing executable instructions to cause the device to perform operations]).
Regarding claims 3 and 16, as discussed above, Tabet in view of Glazer teaches the method of claim 1 and the device of claim 14.
Tabet further discloses wherein the drift detection test comprises a Chi-squared test (see, e.g., paragraphs 60-61, “Systems and methods described herein are directed to autonomic detection and correction of AI/ML model drift.”, “This drift detection process may be performed by statistical analysis of object detection and object recognition rates and performing statistical tests” and 119, “Examples of AI/ML model 1002 may include, but are not limited to: … Chi-squared Automatic Interaction Detection” [i.e., drift detection test includes a Chi-squared detection test]).
With respect to independent claim 18, claim 18 is substantially similar to claim 1 and therefore is rejected on the same ground as claim 1, discussed above. In particular, claim 18 is a computer program product claim with operations that correspond to the method steps of claim 1.
Tabet further discloses a computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to: <perform operations> (see, e.g., paragraphs 5, “an IHS [Information Handling System] may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: identify drift with respect to an AI/ML model”, 12, “a hardware memory device may have program instructions stored thereon that, upon execution, cause an IHS to: calculate drift of an AI/ML model”, 51, “storage device 119 may be implemented using any memory technology allowing IHS 100 to store and retrieve data. For instance, storage device 119 may be a magnetic hard disk storage drive or a solid-state storage drive.” and 185, “a computer-readable storage medium (or "memory") … physical computer-readable storage device that is encompassed by the phrase computer-readable medium or memory.” [i.e., a computer program product for an electronic device including a computer readable storage medium/memory storing processor-executable instructions to cause the device to perform operations]).
Regarding claims 2, 15 and 19, as discussed above, Tabet in view of Glazer teaches the method of claim 1, the device of claim 14, and the computer program product of claim 18.
Although Tabet substantially discloses the claimed invention, Tabet is not relied on to explicitly disclose wherein the drift detection test comprises a Kolmogorov-Smirnov test.
In the same field, analogous art Glazer teaches wherein the drift detection test comprises a Kolmogorov-Smirnov test (see, e.g., Abstract, “We propose an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov [KS] test for detecting distributional change in high-dimensional data.”, pages 5-6, Sect 3, “We now introduce a nonparametric, generalized Kolmogorov-Smirnov (GKS) statistical test for determining whether F ≠ F' in high-dimensional data. … The two-sample KS statistical test is used over T^n,m … to calculate the resulting p-value.” [KS test to detect p-value of change/difference/drift over time T] and page 6, Sect. 4, “Concept drifts are associated with distributional changes in data streams that occur due to hidden context” [i.e., the distributional change/drift detection test includes a Kolmogorov-Smirnov test]).
Tabet and Glazer are analogous art because they are both directed to techniques and systems for detecting and mitigating drift in artificial intelligence/machine learning (AI/ML) models and neural networks (see, e.g., Tabet, Abstract and paragraphs 59-60 and 95, and Glazer, Abstract and page 6, Sect. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Tabet with Glazer to provide “an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data.” and “a novel, hierarchical, minimum-volume sets estimator to represent the distributions to be tested.” that can be applied to “concept-drift detection problems in data-stream classification tasks.” where “Concept drifts are associated with distributional changes in data streams that occur due to hidden context [22] - changes of which the classifier is unaware.” (See, e.g., Glazer Abstract and page 6, Sect. 4). Doing so would have allowed Tabet to use Glazer’s Kolmogorov-Smirnov test for detecting distributional change [i.e., drift] in high-dimensional data and Glazer’s hierarchical, minimum-volume sets estimator to “to detect changes in data streams, and the test is especially efficient in this context.” where Glazer’s Kolmogorov-Smirnov test has “superiority over existing methods”, the “test is more accurate than density estimation approaches” and “since the estimation of MV-sets is simpler than density estimation, our test can achieve higher accuracy than approaches based on density estimation”, as suggested by Glazer (See, e.g., Glazer, Abstract, page 2, Sect 1 and page 5, Sect 2.2).
Regarding claims 8, 17 and 20, as discussed above, Tabet in view of Glazer teaches the method of claim 1, the device of claim 14, and the computer program product of claim 18.
Tabet further discloses wherein estimating the at least one … regions is performed with a second neural network (see, e.g., paragraphs 6, “The AI/ML model may include at least one of: … a Deep Neural Network model”, 141, “to select the AI drift detection model at 1104, method 1100 may provide the input data to a first AI/ML drift detector and to a second AI/ML drift detector … Then, at 1105, method 1100 may: (a) in response to a determination that a first drift confidence score output by the first AI/ML drift detector is smaller than a second drift confidence score output by the second AI/ML drift detector, identify the input data as having the first characteristic” and 158, “inferences made using their data … with respect to physical space or Region-of-Interest” [i.e., estimating regions is performed with a 2nd AI/ML model/neural network]).
Although Tabet substantially discloses the claimed invention, Tabet is not relied on to explicitly disclose estimating the at least one high-density regions.
In the same field, analogous art Glazer teaches estimating at least one high-density regions3 [sic – region] (see, e.g., page 2, Sects 1-2, “we use a novel hierarchical minimum-volume [MV] sets estimator to estimate the set of high-density regions directly … We present here a novel method for approximate MV-sets estimation” [i.e., estimating high-density regions]).
Tabet and Glazer are analogous art because they are both directed to techniques and systems for detecting and mitigating drift in artificial intelligence/machine learning (AI/ML) models and neural networks (see, e.g., Tabet, Abstract and paragraphs 59-60 and 95, and Glazer, Abstract and page 6, Sect. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Tabet with Glazer to provide “an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data.” and “a novel, hierarchical, minimum-volume sets estimator to represent the distributions to be tested.” that can be applied to “concept-drift detection problems in data-stream classification tasks.” where “Concept drifts are associated with distributional changes in data streams that occur due to hidden context [22] - changes of which the classifier is unaware.” (See, e.g., Glazer Abstract and page 6, Sect. 4). Doing so would have allowed Tabet to use Glazer’s test for detecting distributional change [i.e., drift] in high-dimensional data and Glazer’s hierarchical, minimum-volume sets estimator to “to detect changes in data streams, and the test is especially efficient in this context.” where Glazer’s test has “superiority over existing methods”, the “test is more accurate than density estimation approaches” and “since the estimation of MV-sets is simpler than density estimation, our test can achieve higher accuracy than approaches based on density estimation”, as suggested by Glazer (See, e.g., Glazer, Abstract, page 2, Sect 1 and page 5, Sect 2.2).
Regarding claim 9, as discussed above, Tabet in view of Glazer teaches the method of claim 8.
Tabet further discloses wherein the second neural network comprises an autoencoder (see, e.g., paragraph 119, “Examples of AI/ML model 1002 may include, but are not limited to: … Stacked AutoEncoders” [i.e., the 2nd AI/ML model/neural network includes an autoencoder]).
Claims 10, 11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Tabet in view of Glazer as applied to claims 1 and 8 above, and further in view of Moustafa et al. (U.S. Patent Application Pub. No. 2020/0322500 A1, hereinafter “Moustafa”).
Regarding claim 10, as discussed above, Tabet in view of Glazer teaches the method of claim 8.
Although Tabet in view of Glazer substantially teaches the claimed invention, Tabet in view of Glazer is not relied on to teach wherein the second neural network comprises a generative adversarial network.
However, in the same field, analogous art Moustafa teaches wherein the second neural network comprises a generative adversarial network (see, e.g., see, e.g., paragraphs 102, “configuring a Generative Adversarial Network (GAN) that is trained” and 174, “machine learning models (e.g., 256) provided … Such machine learning models 256 may include artificial neural network models, convolutional neural networks” [i.e., the 2nd neural network includes a Generative Adversarial Network/GAN]).
Tabet, Glazer and Moustafa are analogous art because they are each directed to using and monitoring neural networks, artificial intelligence/machine learning (AI/ML) models and classifiers to detect changes in model/classifier performance and drift/changes in data (see, e.g., Tabet, Abstract and paragraphs 59-60 and 95, and Glazer, Abstract and page 6, Sect. 4, and Moustafa paragraphs 174-175, 486 and 979).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tabet in view of Glazer to incorporate the teachings of Moustafa to provide “a system [that] may create synthetic data in order to bolster data sets lacking real data for one or more contexts. [where] In some embodiments, a generative adversarial network (GAN) image generator creates the synthetic data.” (See, e.g., Moustafa paragraph 439). Doing so would have allowed Tabet in view of Glazer to use Moustafa’s generative adversarial network (GAN) “in order to bolster data sets lacking real data for one or more contexts” where “the GAN image generator 5118 may be tuned to provide image data useful for model training” where “Such images may be used to train one or more models (e.g., machine learning models) to be used” as suggested by Moustafa (See, e.g., Moustafa, paragraphs 439 and 450-451).
Regarding claim 11, as discussed above, Tabet in view of Glazer and further in view of Moustafa teaches the method of claim 10.
Although Tabet in view of Glazer substantially teaches the claimed invention, Tabet in view of Glazer is not relied on to teach wherein the second neural network comprises a CycleGAN neural network.
However, in the same field, analogous art Moustafa teaches wherein the second neural network comprises a CycleGAN neural network (see, e.g., paragraphs 102, “configuring a Generative Adversarial Network (GAN) that is trained” and 641, “GAN configuration system 8910 includes GAN model 8920, … Any number of suitable GAN models may be used, including for example, StarGAN, IcGAN, DIAT, or CycleGAN.” [i.e., the 2nd neural network includes a CycleGAN network]).
The motivation to combine Tabet, Glazer and Moustafa is the same as discussed above with respect to claim 10.
Regarding claim 13, as discussed above, Tabet in view of Glazer teaches the method of claim 12.
Although Tabet in view of Glazer substantially teaches the claimed invention, Tabet in view of Glazer is not relied on to teach wherein the input data comprises multimodal data.
In the same field, analogous art Moustafa teaches wherein the input data comprises multimodal data (see, e.g., paragraph 234, “data, may be provided as inputs (e.g., multimodal inputs) to the classification model ( e.g., a trained convolutional neural network)” [i.e., the input data includes multimodal data]).
The motivation to combine Tabet, Glazer and Moustafa is the same as discussed above with respect to claim 10.
Conclusion
The prior art made of record, listed on form PTO-892, and not relied upon, is considered pertinent to applicant's disclosure; and all references generally relate to the detecting and mitigating/remediating drift in data used by machine learning models and neural networks.
For example, non-patent literature Eck, Bradley, et al. ("A monitoring framework for deployed machine learning models with supply chain examples." 2022 IEEE International Conference on Big Data (Big Data). IEEE, 2022., hereinafter “Eck”) discloses “a framework for monitoring machine learning models; and … We use our implementation to study drift in model features, predictions, and performance” where a drift monitoring framework “Evaluate[s] metrics to trigger further actions like model retraining, report generation, or alerting” (see, Abstract and pages 2223-2233, Sect. III).
Also, while the Eck reference has two common co-authors/inventors, Bradley Eck and Duygu Kabakci-Zorlu with the instant application, the 35 USC § 102(b)(1)(A) grace period exception does not apply because the reference was co-authored by Yan Chen, France Savard and Xiaowei Bao, who are not named as inventors of the instant application. See MPEP § 2153.01(a): “If ... the application names fewer joint inventors than a publication (e.g., the application names as joint inventors A and B, and the publication names as authors A, B and C), it would not be readily apparent from the publication that it is by the inventor (i.e., the inventive entity) or a joint inventor and the publication would be treated as prior art under AIA 35 U.S.C. 102(a)(1).” The examiner further notes that Eck was not co-authored by joint, named inventor Amadou Ba.
Also, for example, non-patent literature Porwik et al. ("Detection of data drift in a two-dimensional stream using the Kolmogorov-Smirnov test”, 2022, cited in applicant’s IDS filed 02/27/2023, hereinafter “Porwik”) discloses “that drift detector would send an alert if the model passes some error rate. It is also signal for a classifier to adapt to a new type of data. A drift alarm usually has a statistical guarantee [7]. This provides the Kolmogorov-Smirnov test.” (see, page 170, Sect. 2).
The examiner requests, in response to this office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the reference cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111 (c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RANDY K BALDWIN whose telephone number is (571)270-5222. The examiner can normally be reached on Mon - Fri 9:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached on 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RANDALL K. BALDWIN/Primary Examiner, Art Unit 2125
1 As indicated above in the section 112(b) rejections of these claims, “at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data” has been interpreted as one or more regions of input data having a data volume value, measurement or metric exceeding a minimum data volume value, measurement or metric threshold value.
2 As indicated above in the section 112(b) rejection of this claim, “at least one high-density region that has a data volume that exceeds a minimum data volume threshold within the input data” has been interpreted as one or more regions of input data having a data volume value, measurement or metric exceeding a minimum data volume value, measurement or metric threshold value.
3 As indicated in the objection to claim 8 above, “at least one high-density regions” should read “at least one high-density region”