Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment and remarks filed March 31st, 2026. In the
amendment, claims 1-9, 11-16, and 18-20 were amended and no claims were cancelled or added. As such, claims 1-20 are pending.
Response to Arguments
Applicant’s arguments, see Pages 9-10 and 15-18, filed March 31st, 2026, with respect to the claim objection, 35 U.S.C 112 (f) invocations, the 35 U.S.C 112 (a), and 35 U.S.C 102 rejections have been fully considered and are persuasive. The 35 U.S.C 112 (f) invocation, the 35 U.S.C 112 (a), and 35 U.S.C 102 rejections have been withdrawn.
Applicant’s arguments, see Page 10, filed March 31st 2026, with respect to the 35 U.S.C 112 (b) rejections has been fully considered and are persuasive. Amendments to the claims obviate the rejections of record. The rejections of claims have been withdrawn. See updated rejections below.
Applicant’s arguments, in view of the amendment filed March 31st 2026 with respect to the rejections of claims 1-20 under 35 U.S.C § 101 and 103, are maintained and not persuasive for the following reasons:
35 U.S.C. 101:
Applicant argues that the claims are not directed to mental processes because they recite a "several-step manipulation of data" that cannot practically be performed in the human mind, citing Synopsys (Pages 11-13 of Remarks). The Examiner respectfully disagrees. Applicant's reliance on Synopsys is not persuasive because the Federal Circuit in Synopsys, Inc. v. Mentor Graphics Corp. (Fed. Cir. 2016) actually held the claims at issue to be ineligible as directed to a mental process, finding that the translation steps could be performed mentally or with pencil and paper (See MPEP 2106.04(a)(2)). Furthermore, the claims here recite mental process limitations such as "determine a re-construction loss based at least on the reconstructed first feature value and the first feature value" and "determine, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred," which involve evaluations and judgments that can be performed in the human mind. The claim also recites mathematical concepts such as performing convolution and pooling operations, computing reconstructed values, and weighting losses through multiplication. The recitation of a "processor circuit" and "memory" merely uses generic computer components as a tool to perform the abstract idea, which does not remove the claim from the mental processes or mathematical concepts groupings per MPEP 2106.04(a)(2).
Applicant argues that the "determine... data drift has occurred" and "cause reduction of the data drift" limitations improve systems utilizing machine learning models and reduce inaccurate use of compute resources, and thus integrates the alleged abstract idea into a practical application, citing MPEP § 2106.04(d) (Pages 13-14 of Remarks). The Examiner respectfully disagrees. The "cause reduction of the data drift" limitation is recited at a high level of generality with no specific technical implementation and amounts to merely applying the judicial exception or instructing one to "apply it" with a computer, per MPEP 2106.05(f). The claim does not recite any particular technical mechanism by which the computer itself is improved. Instead, the claim recites a result-oriented outcome ("cause reduction") without specifying how that result is technically achieved, which does not integrate the abstract idea into a practical application under MPEP 2106.05(f). The claim uses a generic processor and memory as a tool to perform the abstract mathematical operations of convolution, pooling, loss computation, and threshold comparison.
Applicant argues that claim 2's "cause the machine learning model to be re-trained" limitation improves computer-related technology like Enfish by reducing data drift in the machine learning model (Pages 14-15 of Remarks). The Examiner respectfully disagrees. Unlike Enfish, where the claims were directed to a specific self-referential table that improved how the computer database itself functioned (see MPEP 2106.05(a)), claim 2 merely recites a high-level result of "caus[ing] the machine learning model to be re-trained" without specifying any particular technical improvement to the computer or to the training process itself. This limitation amounts to mere instructions to apply the judicial exception by invoking a generic re-training step, which under MPEP 2106.05(f) is insufficient to integrate the abstract idea into a practical application or to provide an inventive concept.
35 U.S.C. 103:
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 8, and 15 recite the limitation "…resulting in a compressed knowledge representation of the input feature vector" in limitation 4 of the claim. There is insufficient antecedent basis for this limitation in the claim.
Claim 5 recites the limitation "…weight the re-construction loss of the autoencoder" in line 4 of the claim. There is insufficient antecedent basis for this limitation in the claim.
Claims 5 and 19 recite the limitation “"…weight the re-construction loss of the autoencoder using the plurality of normalized importance values” in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the program code is further structured to cause the at least one processor circuit" in line 3 of the claim. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1: The claim recites a system comprising a processor circuit and memory; therefore, it is directed to the statutory category of a machine.
Step2A Prong 1: The claim recites:
perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector: This limitation recites a mathematical concept because it involves performing convolution and pooling operations on numerical data.
reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value: This limitation recites a mental process because it involves reconstructing a feature value from a decompressed representation, which can be performed in the human mind or with pen and paper.
determine a re-construction loss based at least on the reconstructed first feature value and the first feature value: This limitation recites a mental process because it involves the evaluation/judgement/opinion to determine a re-construction loss based on two observed values.
weight the re-construction loss using the first importance value as a first weight: This limitation recites a mathematical concept because it involves calculating and weighting a reconstruction loss using mathematical operations. See Paragraph 35 which states, “Re-construction loss determiner 106 may then weight each of the determining loss values by its corresponding normalized feature importance value. For example, re- construction loss determiner 106 may multiply the first loss value by the normalized feature importance value…”.
determine, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred…: This limitation is a mental process because it involves the evaluation/judgement/opinion of determining a computed value meeting a threshold.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
A system, comprising: a processor circuit; and memory that stores program code structured to cause by the processor circuit to… and cause reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
…receive a first feature value; receive a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: Data Gather- Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
with respect to a machine learning model trained on the first feature value: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
decompress the compressed knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
A system, comprising: a processor circuit; and memory that stores program code structured to cause by the processor circuit to… and cause reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
…receive a first feature value; receive a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
with respect to a machine learning model trained on the first feature value: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi). Restricting the flow of data through a bottleneck to produce a compressed representation is a conventional autoencoder operation, as seen in Ansari (US 20210304855 A1, March 25th 2021). Paragraph 66 of Ansari describes training a model “that reproduces unlabeled input data using a network architecture that forces information to flow through a bottleneck.” Ansari treats this as a known technique, identifying the architecture as a “conventional autoencoder” as explained in Paragraph 65. Ansari further explains in paragraph 38 that "any bottleneck model that maps input signals back to an original waveform may be used… In other words, any model that includes restrictions causing the model to learn a reduced representation of an input signal…".
decompress the compressed knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi). Decoding/decompressing a compressed representation back to the original dimensionality is a conventional autoencoder operation, as seen in Ansari (US 20210304855 A1, March 25th 2021). Ansari treats this as a known technique, identifying the architecture as a “conventional autoencoder” as explained in Paragraph 65. Ansari further describes in paragraph 48 that "a second network (decoder) may convert the lower dimensional space back to the higher space, i.e., reconstruct the input signal." Ansari uses this decode from compressed representation function as a standard part of the architecture, describing that “[t]he decoding layers 108 may process the embedded features 106. The decoding layers 108 may include layers of an increasing size culminating in the output layer 110 that outputs a reconstructed output signal of equal size to the input signal.” (paragraph 52).
The elements in combination as an ordered whole still do not amount to significantly more than the judicial exception (i.e., the abstract ideas of mental processes and mathematical concepts for performing convolution and pooling operations, reconstructing feature values, calculating and weighting reconstruction loss, and determining data drift based on a threshold). The claim merely describes a process of applying known mathematical techniques (convolution, pooling, reconstruction, loss computation, and threshold comparison) to analyze and evaluate data, compress data through a bottleneck, along with a result-oriented step of reducing data drift. The recitation of a processor, memory, and a machine learning model merely indicates a technological environment in which the abstract ideas are applied, without improving the functioning of a computer or the machine learning model itself.
Therefore, the claim as a whole remains focused on the abstract idea and fails Step 2B of the eligibility analysis.
Claim 2
Step 1: A machine, as above.
Step2A Prong 1: This claim does not recite any abstract ideas, but the claim depends on claim 1, which recites an abstract idea.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
to cause reduction of the data drift, the program code is further structured to cause the processor circuit to: cause the machine learning model to be re-trained: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
to cause reduction of the data drift, the program code is further structured to cause the processor circuit to: cause the machine learning model to be re-trained: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 1: A machine, as above.
Step2A Prong 1: The claim recites:
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
to reconstruct the first feature value, the program code is further structured to cause the processor circuit to: utilize a self-supervised neural network autoencoder: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
to reconstruct the first feature value, the program code is further structured to cause the processor circuit to: utilize a self-supervised neural network autoencoder: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, this additional element represents mere instructions to apply an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 4
Step 1: A machine, as above.
Step2A Prong 1: The claim recites:
sum a plurality of importance values comprising the first importance value and a second importance value corresponding to the second feature value, resulting in a summed value: This limitation recites a mathematical concept because it involves adding two importance values.
and for each importance value of the plurality of importance values, divide the importance value by the summed value, thereby normalizing the importance value: This limitation recites a mathematical concept because it involves dividing the importance value by the result of the importance value.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
to receive the first feature value… receive an input feature vector comprising the first feature value and a second feature value; and to receive the first importance value for the first feature value: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
to receive the first feature value… receive an input feature vector comprising the first feature value and a second feature value; and to receive the first importance value for the first feature value: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 5
Step 1: A machine, as above.
Step2A Prong 1: The claim recites:
wherein to weight the re-construction loss… weight the re-construction loss of the autoencoder using the plurality of normalized importance values as weights: This limitation recites a mathematical concept because it involves multiplying a reconstruction loss by normalized importance values (i.e., performing a mathematical weighting operation on numerical data). See Paragraph 33, which states, “The re-construction error is typically the mean-squared-error… Every layer of autoencoder 200 has an affine transformation (e.g., Wx+b, where x corresponds to a column vector corresponding to a sample from the dataset (e.g., input feature vector(s) 108) that is provided to autoencoder 200, W corresponds to the weight matrix, and b corresponds to a bias vector) followed by a non-linear function.”
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 1: A machine, as above.
Step2A Prong 1: The claim recites:
to determine the re-construction loss… determine a difference between the first feature value and the reconstructed first feature value: These limitations recite a mental process because it involves the evaluation/judgement/opinion to determine the re-construction loss and the difference between two values.
and square the difference to determine the re-construction loss: This limitation recites a mathematical concept because it involves squaring the difference calculated from the previous step.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the program code is further structured to cause the processor circuit to: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 1: A machine, as above.
Step2A Prong 1: The claim recites:
the first importance value is a user-defined importance value: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of a human to pick the value corresponding to the importance value parameter.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, this additional element represents mere instructions to apply
an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 8
Step 1: The claim recites a method; therefore, it is directed to the statutory category of a process.
Step2A Prong 1: The claim recites:
perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector: This limitation recites a mathematical concept because it involves performing convolution and pooling operations on numerical data.
and reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value: This limitation recites a mathematical concept because it involves computing a reconstructed numerical value from transformed data.
determining a re-construction loss based at least on the reconstructed first feature value and the first feature value: This limitation recites a mental process because it involves the evaluation/judgement/opinion to determine a re-construction loss based on two observed values.
weighting the re-construction loss using the first importance value as a first weight: This limitation recites a mathematical concept because it involves calculating and weighting a reconstruction loss using mathematical operations. See Paragraph 35 which states, “Re-construction loss determiner 106 may then weight each of the determining loss values by its corresponding normalized feature importance value. For example, re- construction loss determiner 106 may multiply the first loss value by the normalized feature importance value…”.
determining, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred…: This limitation is a mental process because it involves the evaluation/judgement/opinion of determining a computed value meeting a threshold.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
receiving a first feature value; receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: Data Gather- Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
providing the input feature vector to an autoencoder to cause the autoencoder to: Gather- Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
decompress the knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
with respect to a machine learning model trained on the first feature value; and causing reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
receiving a first feature value; receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
providing the input feature vector to an autoencoder to cause the autoencoder to: The additional element of “providing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of transmitting steps amounts to no more than mere data gathering. This element amounts to transmitting data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
decompress the knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
with respect to a machine learning model trained on the first feature value; and causing reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
the method further comprises generating a notification that indicates that the data drift has been detected; or said causing reduction of the data drift comprises generating a command…: These limitations encompass a mental process because it involves the evaluation/judgement/opinion of generating a message/notification/command based on a detected data draft.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
generating a command that causes the machine learning model to be re-trained or deactivated: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
generating a command that causes the machine learning model to be re-trained or deactivated: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 1: A process, as above.
Step2A Prong 1: This claim does not recite any abstract ideas, but the claim depends on claim 8, which recites an abstract idea.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the autoencoder is a self-supervised neural network: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the autoencoder is a self-supervised neural network: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 11
Step 1: A process, as above.
Step2A Prong 1: The claim recites, inter alia:
summing a plurality of importance values comprising the first importance value and a second importance value corresponding to a second feature value, resulting in a summed value: This limitation encompasses a mathematical concept because it involves adding two different values.
and for each importance value of the plurality of importance values, dividing the importance value by the summed value, thereby normalizing the importance value: This limitation encompasses a mathematical concept because it involves dividing the importance value by the summed value calculated in the previous step.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
said weighting the re-construction loss comprises: weighting the re-construction loss of the autoencoder using the plurality of normalized importance values as weights: This limitation recites a mathematical concept because it involves multiplying a reconstruction loss by normalized importance values (i.e., performing a mathematical weighting operation on numerical data). See Paragraph 33, which states, “The re-construction error is typically the mean-squared-error… Every layer of autoencoder 200 has an affine transformation (e.g., Wx+b, where x corresponds to a column vector corresponding to a sample from the dataset (e.g., input feature vector(s) 108) that is provided to autoencoder 200, W corresponds to the weight matrix, and b corresponds to a bias vector) followed by a non-linear function.”
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
the re-construction loss is determined by: determining a difference between the first feature value and the reconstructed first feature value: These limitations recite a mental process because it involves the evaluation/judgement/opinion to determine the re-construction loss and the difference between two values.
and squaring the difference to determine the re-construction loss: This limitation is a mathematical concept because it involves squaring the difference calculated from the previous step.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 14
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
the plurality of importance values is at least one of: user-defined: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of a human to pick the value corresponding to the importance value parameter.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
or provided as an output from the machine learning model: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
or provided as an output from the machine learning model: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 15
Step 1: The claim recites a computer-readable medium; therefore, it is directed to the statutory
category of an article of manufacture.
Step2A Prong 1: The claim recites, inter alia:
perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector: This limitation recites a mathematical concept because it involves performing convolution and pooling operations on numerical data.
and reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value: This limitation recites a mathematical concept because it involves computing a reconstructed numerical value from transformed data.
determining a re-construction loss based at least on the reconstructed first feature value and the first feature value: This limitation recites a mental process because it involves the evaluation/judgement/opinion to determine a re-construction loss based on two observed values.
weighting the re-construction loss using the first importance value as a first weight: This limitation recites a mathematical concept because it involves calculating and weighting a reconstruction loss using mathematical operations. See Paragraph 35 which states, “Re-construction loss determiner 106 may then weight each of the determining loss values by its corresponding normalized feature importance value. For example, re- construction loss determiner 106 may multiply the first loss value by the normalized feature importance value…”.
determining, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred…: This limitation is a mental process because it involves the evaluation/judgement/opinion of determining a computed value meeting a threshold.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
A computer-readable storage medium having program instructions recorded thereon that, when executed by a processor, perform a method comprising: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
receiving a first feature value; receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: Data Gather- Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
providing the input feature vector to an autoencoder to cause the autoencoder to: Gather- Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
decompress the knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
with respect to a machine learning model trained on the first feature value; and causing reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
A computer-readable storage medium having program instructions recorded thereon that, when executed by a processor, perform a method comprising: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
receiving a first feature value; receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
providing the input feature vector to an autoencoder to cause the autoencoder to: The additional element of “providing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of transmitting steps amounts to no more than mere data gathering. This element amounts to transmitting data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
decompress the knowledge representation: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
with respect to a machine learning model trained on the first feature value; and causing reduction of the data drift: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16 recites similar limitations to claim 9. Therefore, claim 16 is rejected using the same rationale as claim 9.
Claim 17 recites similar limitations to claim 10. Therefore, claim 17 is rejected using the same rationale as claim 10.
Claim 18 recites similar limitations to claim 11. Therefore, claim 18 is rejected using the same rationale as claim 11.
Claim 19 recites similar limitations to claim 12. Therefore, claim 19 is rejected using the same rationale as claim 12.
Claim 20 recites similar limitations to claim 13. Therefore, claim 20 is rejected using the same rationale as claim 13.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 6, 8, 13-15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Allahdadian (US 20230043993 A1) in view of Sarafijanovic-Djukic ("Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder", 2020).
Regarding claim 1,
Allahdadian teaches [a] system, comprising: a processor circuit and memory that stores program code structured to cause the processor circuit to: (Paragraph 124 of Allahdadian, "Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504."
Allahdadian discloses a computer system having a processor coupled to a bus and configured to execute instructions, which corresponds to the claimed processor circuit.).
receive a first feature value (Paragraph 39, "an input tuple is provided as a feature vector ... that contains a respective value for each of all features 121-122"
Allahdadian discloses an input tuple provided as a feature vector containing respective values for each feature, which corresponds to the receiving of a first feature value.).
receive a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification (Paragraph 34, "…computer 100 uses importances 130 of respective features 121-122 ... Importances 130 are numeric weights that indicate the relative natural significance of each feature in any tuple"; Paragraph 36, "…feature importance is more or less a covariance between values of a feature and labels of tuples. A label is a known correct inference for a tuple"
Allahdadian teaches numeric importance values that indicate each feature's significance with respect to the tuple's label (classification).).
decompress the compressed knowledge representation (Paragraph 55, "the autoencoder encodes input into a semantic coding, which the autoencoder further decodes back into a more or less accurate copy of the input."
Allahdadian teaches an autoencoder decoder that decodes the encoded semantic coding back toward the original input. The decoding operation of Allahdadian corresponds to the decompressing of the limitation. Paragraph 32 of the instant specification discloses that “the decoder is configured to decompress the knowledge representations and reconstruct input feature vector(s) 108 back from their encoded form”).
reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value (Paragraph 55, "…the autoencoder contains additional neural layers that are trained to regenerate the original input (i.e. tuple)."
Allahdadian teaches neural layers trained to regenerate the original feature values from the decoded representation. The regenerated feature value of Allahdadian corresponds to the reconstructed first feature value of the limitation.).
determine a re-construction loss based at least on the reconstructed first feature value and the first feature value (Paragraph 56, "A measured difference between the original input and the regenerated input is known as reconstruction loss. Because the original input and the regenerated input are composed of individual features, a difference may be measured between an original feature and a reconstructed feature to calculate a respective reconstruction loss for that feature…"
Allahdadian teaches computing reconstruction loss as the measured difference between an original feature and its reconstructed counterpart.).
weight the re-construction loss using the first importance value as a first weight (Paragraph 64, "…losses of features 121-122 are respectively adjusted by applying importances 130 as respective weighting coefficients"
Allahdadian teaches adjusting per-feature reconstruction losses by applying importance values as weighting coefficients.).
determine, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred with respect to a machine learning model trained on the first feature value (Paragraph 69, "reconstructive model 140 detects an anomaly when tuple loss, such as measured by mean squared error (MSE) or summation, exceeds an anomaly threshold…", Paragraph 85, "…post-weighting step 202 compensates for concept drift by multiplicatively applying feature weights 223 to the original losses to generate a weighted loss for each feature. Weighted losses of the new tuple can be arithmetically combined ... to generate an anomaly score that, when compared to a threshold, indicates whether or not the new tuple is anomalous"
Allahdadian teaches comparing a weighted feature importance adjusted reconstruction loss against a threshold to detect drift in a reconstructive model trained on the feature values.).
and cause reduction of the data drift. (Paragraph 86, "step 203 that (e.g. manually) labels only those new tuples that have the highest (i.e. most anomalous) anomaly scores. Shown as feedback 230, newly labeled tuples are included into labelled traces 222. In that way, labelled traces 222 evolve to reflect concept drift"
Allahdadian discloses a feedback step that labels the most anomalous tuples and incorporates them back into the training labels so that the reconstructive model changes/evolves to compensate for the detected drift, which corresponds to the causing of reduction of the data drift.).
Allahdadian does not teach perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector and restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector.
Sarafijanovic-Djukic, in the same field of endeavor, teaches perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector (Sarafijanovic-Djukic Page 5 Section 3.1, "Inspired by the Inception architecture, we design an Inception-like CAE architecture ... it combines outputs from 1×1, 3×3 and 5×5 convolutions as well as a maximum pooling operation"; Page 6 Table 1, "Inception (8). MaxPooling(2,2). 16 × 16 × 32 / Inception(16). MaxPooling(2,2). 8 × 8 × 64 / Inception(32). MaxPooling(2,2). 4 × 4 × 128", Page 3 Section 2.2, “Deep approaches to anomaly detection for image data often use a convolutional autoencoder (CAE) which include convolutional layers in the AE architecture [24,31].”
Sarafijanovic-Djukic teaches a convolutional autoencoder encoder that performs convolution and max-pooling operations to compress an input from 32×32 down to a 4×4×128 representation. Paragraph 32 of the instant specification describes the encoder as performing “convolutional and pooling operations that compress the input data (i.e., input feature vector(s) 108).”.).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector (Sarafijanovic-Djukic Page 3 Section 2.2, "An AE has a bottleneck layer with a lower dimension than the input layer and hence allows learning a low-dimensional representation (encoding) of the input data."
Sarafijanovic-Djukic discloses an autoencoder bottleneck layer of lower dimension than the input layer that forces the network to learn a compressed low-dimensional encoding of the input, which corresponds to the restriction of flow through a bottleneck resulting in a compressed knowledge representation.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's importance-weighted reconstruction-loss drift detector with Sarafijanovic-Djukic's convolutional autoencoder having convolution, pooling, and bottleneck layers in order to improve the quality of the learned low-dimensional representation of the input data and thereby improve anomaly-detection predictive performance (Sarafijanovic-Djukic, Page 4 Section 3).
Regarding claim 6,
Allahdadian teaches wherein to determine the re-construction loss, the program code is further structured to cause the at least one processor circuit to: determine a difference between the first feature value and the reconstructed first feature value (Paragraph 56, "A measured difference between the original input and the regenerated input is known as reconstruction loss ... a difference may be measured between an original feature and a reconstructed feature to calculate a respective reconstruction loss for that feature"
Allahdadian discloses measuring a difference between an original feature value and its reconstructed feature value as part of computing the reconstruction loss, which corresponds to the determining of a difference between the first feature value and the reconstructed first feature value.).
and square the difference to determine the re-construction loss (Paragraph 69, "reconstructive model 140 detects an anomaly when tuple loss, such as measured by mean squared error (MSE) or summation, exceeds an anomaly threshold"
Allahdadian discloses computing the reconstruction loss as a mean squared error, which requires squaring the difference between each original feature value and its reconstruction.).
Regarding claim 8,
Allahdadian teaches [a] method, comprising: receiving a first feature value (Paragraph 39, "an input tuple is provided as a feature vector ... that contains a respective value for each of all features 121-122"
Allahdadian discloses an input tuple provided as a feature vector containing respective values for each feature, which corresponds to the receiving of a first feature value.).
receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification (Paragraph 34, "Importances 130 are numeric weights that indicate the relative natural significance of each feature in any tuple", Paragraph 36, "feature importance is more or less a covariance between values of a feature and labels of tuples"
Allahdadian teaches numeric importance values that indicate each feature's significance with respect to the tuple's label (classification).).
providing the input feature vector to an autoencoder to cause the autoencoder to: (Paragraph 55, "an autoencoder learns which features should be deemphasized and how to encode retained semantic features. An autoencoder herein further is a reconstructive model"
Allahdadian discloses providing the input tuple feature vector to an autoencoder.).
decompress the knowledge representation (Paragraph 55, "the autoencoder encodes input into a semantic coding, which the autoencoder further decodes back into a more or less accurate copy of the input"
Allahdadian teaches an autoencoder decoder that decodes the encoded semantic coding back toward the original input. The decoding operation of Allahdadian corresponds to the decompressing of the limitation. Paragraph 32 of the instant specification discloses that “the decoder is configured to decompress the knowledge representations and reconstruct input feature vector(s) 108 back from their encoded form.”).
and reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value (Paragraph 55, "…the autoencoder contains additional neural layers that are trained to regenerate the original input (i.e. tuple)."
Allahdadian teaches neural layers trained to regenerate the original feature values from the decoded representation. The regenerated feature value of Allahdadian corresponds to the reconstructed first feature value of the limitation.).
determining a re-construction loss based at least on the reconstructed first feature value and the first feature value (Paragraph 56, "A measured difference between the original input and the regenerated input is known as reconstruction loss ... a difference may be measured between an original feature and a reconstructed feature to calculate a respective reconstruction loss for that feature"
Allahdadian teaches computing reconstruction loss as the measured difference between an original feature and its reconstructed counterpart.).
weighting the re-construction loss using the first importance value as a first weight (Paragraph 64, "losses of features 121-122 are respectively adjusted by applying importances 130 as respective weighting coefficients"
Allahdadian teaches adjusting per-feature reconstruction losses by applying importance values as weighting coefficients.).
determining, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred with respect to a machine learning model trained on the first feature value (Paragraph 69, "reconstructive model 140 detects an anomaly when tuple loss, such as measured by mean squared error (MSE) or summation, exceeds an anomaly threshold", Paragraph 85, "Weighted losses of the new tuple can be arithmetically combined ... to generate an anomaly score that, when compared to a threshold, indicates whether or not the new tuple is anomalous"
Allahdadian teaches comparing a weighted feature importance adjusted reconstruction loss against a threshold to detect drift in a reconstructive model trained on the feature values.).
and causing reduction of the data drift. (Paragraph 86, "Shown as feedback 230, newly labeled tuples are included into labelled traces 222. In that way, labelled traces 222 evolve to reflect concept drift"
Allahdadian discloses a feedback step that labels the most anomalous tuples and incorporates them back into the training labels so that the reconstructive model changes/evolves to compensate for the detected drift, which corresponds to the causing of reduction of the data drift.).
Allahdadian does not teach perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector and restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector.
Sarafijanovic-Djukic, in the same field of endeavor, teaches perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector (Sarafijanovic-Djukic Page 5 Section 3.1, "we design an Inception-like CAE architecture that combines convolutional filters of different kernel sizes ... it combines outputs from 1×1, 3×3 and 5×5 convolutions as well as a maximum pooling operation", Page 6 Table 1, "Inception (8). MaxPooling(2,2). 16 × 16 × 32 / Inception(16). MaxPooling(2,2). 8 × 8 × 64 / Inception(32). MaxPooling(2,2). 4 × 4 × 128", Page 3 Section 2.2, “Deep approaches to anomaly detection for image data often use a convolutional autoencoder (CAE) which include convolutional layers in the AE architecture [24,31].”
Sarafijanovic-Djukic teaches a convolutional autoencoder encoder that performs convolution and max-pooling operations to compress an input from 32×32 down to a 4×4×128 representation. Paragraph 32 of the instant specification describes the encoder as performing “convolutional and pooling operations that compress the input data (i.e., input feature vector(s) 108).”).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector (Sarafijanovic-Djukic Page 3 Section 2.2, "An AE has a bottleneck layer with a lower dimension than the input layer and hence allows learning a low-dimensional representation (encoding) of the input data"
Sarafijanovic-Djukic discloses an autoencoder bottleneck layer of lower dimension than the input layer that forces the network to learn a compressed low-dimensional encoding of the input, which corresponds to the restriction of flow through a bottleneck resulting in a compressed knowledge representation.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's importance-weighted reconstruction-loss drift detector with Sarafijanovic-Djukic's convolutional autoencoder having convolution, pooling, and bottleneck layers in order to improve the quality of the learned low-dimensional representation of the input data and thereby improve anomaly-detection predictive performance (Sarafijanovic-Djukic, Page 4 Section 3).
Claim 13 recites similar limitations to claim 6. Therefore, claim 13 is rejected using the same rationale as claim 6.
Regarding claim 14,
Allahdadian teaches the plurality of importance values is at least one of: user-defined; or provided as an output from the machine learning model (Paragraph 34, "Importances 130 are numeric weights that indicate the relative natural significance of each feature", Paragraph 37, "computer 100 can generate and/or use importances 130 even if reconstructive model 140 is instead hosted on a different computer so long as the different computer sends feature reconstruction loss measurements"
Allahdadian discloses that the importance values may be generated externally to the reconstructive model and supplied to the computer that uses them.)
Regarding claim 15,
Allahdadian teaches [a] computer-readable storage medium having program instructions recorded thereon that, when executed by a processor, perform a method comprising (Paragraph 124, "Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504").
receiving a first feature value (Paragraph 39, "an input tuple is provided as a feature vector ... that contains a respective value for each of all features 121-122"
Allahdadian discloses an input tuple provided as a feature vector containing respective values for each feature, which corresponds to the receiving of a first feature value.).
receiving a first importance value corresponding to the first feature value indicating a level of impact the first feature value has on a classification (Paragraph 34, "Importances 130 are numeric weights that indicate the relative natural significance of each feature in any tuple", Paragraph 36, "feature importance is more or less a covariance between values of a feature and labels of tuples"
Allahdadian teaches numeric importance values that indicate each feature's significance with respect to the tuple's label (classification).).
providing the input feature vector to an autoencoder to cause the autoencoder to: (Paragraph 55, "An autoencoder herein further is a reconstructive model because the autoencoder contains additional neural layers that are trained to regenerate the original input (i.e. tuple)"
Allahdadian discloses providing the input tuple feature vector to an autoencoder, which corresponds to the claimed providing of the input feature vector to an autoencoder.).
decompress the knowledge representation (Paragraph 55, "…the autoencoder encodes input into a semantic coding, which the autoencoder further decodes back into a more or less accurate copy of the input."
Allahdadian teaches an autoencoder decoder that decodes the encoded semantic coding back toward the original input. The decoding operation of Allahdadian corresponds to the decompressing of the limitation. Paragraph 32 of the instant specification discloses that “the decoder is configured to decompress the knowledge representations and reconstruct input feature vector(s) 108 back from their encoded form.”).
and reconstruct the first feature value from the decompressed knowledge representation, resulting in a reconstructed first feature value (Paragraph 55, "…the autoencoder contains additional neural layers that are trained to regenerate the original input (i.e. tuple)."
Allahdadian teaches neural layers trained to regenerate the original feature values from the decoded representation. The regenerated feature value of Allahdadian corresponds to the reconstructed first feature value of the limitation.).
determining a re-construction loss based at least on the reconstructed first feature value and the first feature value (Paragraph 56, "…a difference may be measured between an original feature and a reconstructed feature to calculate a respective reconstruction loss for that feature…"
Allahdadian teaches computing reconstruction loss as the measured difference between an original feature and its reconstructed counterpart.).
weighting the re-construction loss using the first importance value as a first weight (Paragraph 64, "The original losses of features 121-122 are respectively adjusted by applying importances 130 as respective weighting coefficients…"
Allahdadian teaches adjusting per-feature reconstruction losses by applying importance values as weighting coefficients.).
determining, based on the weighted re-construction loss meeting a threshold condition, data drift has occurred with respect to a machine learning model trained on the first feature value (Paragraph 69, "reconstructive model 140 detects an anomaly when tuple loss, such as measured by mean squared error (MSE) or summation, exceeds an anomaly threshold…", Paragraph 85, "Weighted losses of the new tuple can be arithmetically combined ... to generate an anomaly score that, when compared to a threshold, indicates whether or not the new tuple is anomalous"
Allahdadian teaches comparing a weighted feature importance adjusted reconstruction loss against a threshold to detect drift in a reconstructive model trained on the feature values.).
and causing reduction of the data drift (Paragraph 86, "Shown as feedback 230, newly labeled tuples are included into labelled traces 222. In that way, labelled traces 222 evolve to reflect concept drift"
Allahdadian discloses a feedback step that labels the most anomalous tuples and incorporates them back into the training labels so that the reconstructive model changes/evolves to compensate for the detected drift, which corresponds to the causing of reduction of the data drift.).
Allahdadian does not teach perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector and restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector.
Sarafijanovic-Djukic, in the same field of endeavor, teaches perform a convolution and pooling operation on the first feature value, resulting in a compressed input feature vector (Sarafijanovic-Djukic Page 5 Section 3.1, "we design an Inception-like CAE architecture ... it combines outputs from 1×1, 3×3 and 5×5 convolutions as well as a maximum pooling operation", Page 6 Table 1, "Inception (8). MaxPooling(2,2). 16 × 16 × 32 / Inception(16). MaxPooling(2,2). 8 × 8 × 64 / Inception(32). MaxPooling(2,2). 4 × 4 × 128", Page 3 Section 2.2, “Deep approaches to anomaly detection for image data often use a convolutional autoencoder (CAE) which include convolutional layers in the AE architecture [24,31].”
Sarafijanovic-Djukic teaches a convolutional autoencoder encoder that performs convolution and max-pooling operations to compress an input from 32×32 down to a 4×4×128 representation. Paragraph 32 of the instant specification describes the encoder as performing “convolutional and pooling operations that compress the input data (i.e., input feature vector(s) 108).”).
restrict the flow of the compressed input feature vector through a bottleneck, resulting in a compressed knowledge representation of the input feature vector (Sarafijanovic-Djukic Page 3 Section 2.2, "An AE has a bottleneck layer with a lower dimension than the input layer and hence allows learning a low-dimensional representation (encoding) of the input data"
Sarafijanovic-Djukic discloses an autoencoder bottleneck layer of lower dimension than the input layer that forces the network to learn a compressed low-dimensional encoding of the input, which corresponds to the restriction of flow through a bottleneck resulting in a compressed knowledge representation.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's importance-weighted reconstruction-loss drift detector with Sarafijanovic-Djukic's convolutional autoencoder having convolution, pooling, and bottleneck layers in order to improve the quality of the learned low-dimensional representation of the input data and thereby improve anomaly-detection predictive performance (Sarafijanovic-Djukic, Page 4 Section 3).
Claim 20 recites similar limitations to claim 6. Therefore, claim 20 is rejected using the same rationale as claim 6.
Claims 7 is rejected under 35 U.S.C. 103 as being unpatentable over Allahdadian (US 20230043993 A1) in view of Sarafijanovic-Djukic ("Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder", 2020) and in further view of Dalli (US 20220172050 A1).
Regarding claim 7,
Allahdadian teaches the first importance value (Paragraph 34, "Importances 130 are numeric weights that indicate the relative natural significance of each feature in any tuple", Paragraph 36, "feature importance is more or less a covariance between values of a feature and labels of tuples"
Allahdadian teaches numeric importance values that indicate each feature's significance with respect to the tuple's label (classification).)
Allahdadian in view of Sarafijanovic-Djukic does not teach wherein the … value is a user-defined … value.
Dalli, in the same field of endeavor, teaches wherein the… value is a user-defined… value (Paragraph 174, “An exemplary XGAN system may construct explanation scaffolding from the output produced by the explainable architecture of the generator and/or the discriminator, and use it to illustrate the results to the interpreter to assist in understanding how the model arrived at a prediction. An interpreter may be the end-user or a component within the XGAN system”, Claim 16, “…the explainable model is further configured to be trained to learn one or more suggested actions for a given user with a specific context which lead to a change in outcome...”, Paragraph 7, “SemAuto may be able to capture categorical information of the items rated by the user and capture the shared side information of all the items rated by the user.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's autoencoder-based drift detection in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Dalli’s user input value step in the XAED system in order to identify anomalies and instances of data drift to adapt to changes of real-world data distributions (Paragraph 191 of Dalli).
Claims 2, 9, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Allahdadian (US 20230043993 A1) in view of Sarafijanovic-Djukic ("Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder", 2020) and in further view of Kenemer (US 11394732 B1).
Regarding claim 2,
Allahdadian in view of Sarafijanovic-Djukic does not teach to cause reduction of the data drift, the program code is further structured to cause the processor circuit to: cause the machine learning model to be re-trained.
Kenemer, in the same field of endeavor, teaches wherein to cause reduction of the data drift, the program code is further structured to cause the processor circuit to: cause the machine learning model to be re-trained (Kenemer Col. 12 Lines 38-41, "In response to the “possible data drift detected” notification, back-end computing device 704 may send a “set monitoring mode” instruction and/or a neural network model update to retrain the neural network of the respective."
Kenemer discloses sending a model update to retrain the neural network in response to a detected data drift, which corresponds to the causing of the machine learning model to be re-trained.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's autoencoder-based drift detection in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Kenemer's retraining response to detected drift in order to reduce overfitting and improve model performance after drift is detected (Kenemer, Col. 1 Lines 40-45).
Regarding claim 9,
Allahdadian in view of Sarafijanovic-Djukic does not teach explicitly that the method further comprises generating a notification that indicates that the data drift has been detected, or that said causing reduction of the data drift comprises generating a command that causes the machine learning model to be re-trained or deactivated.
Kenemer, in the same field of endeavor, teaches the method further comprises generating a notification that indicates that the data drift has been detected (Kenemer Col. 12 Lines 36-41, "In response to the “possible data drift detected” notification, back-end computing device 704 may send a “set monitoring mode” instruction and/or a neural network model update to retrain the neural network of the respective"
Kenemer teaches generating and transmitting a "possible data drift detected" notification when drift is identified.).
Kenemer further teaches or said causing reduction of the data drift comprises generating a command that causes the machine learning model to be re-trained or deactivated (Kenemer Col. 12 Lines 38-41, "back-end computing device 704 may send ... instruction and/or a neural network model update to retrain the neural network"
Kenemer teaches sending a neural network model update command that causes the neural network to be retrained in response to detected drift.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's autoencoder-based drift detection in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Kenemer's notification-and-retraining response to detected drift in order to reduce overfitting and improve model performance after drift is detected (Kenemer, Col. 1 Lines 40-45).
Claim 16 recites similar limitations to claim 9. Therefore, claim 16 is rejected using the same rationale as claim 9.
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Allahdadian (US 20230043993 A1) in view of Sarafijanovic-Djukic ("Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder", 2020) and in further view of Heitzmann (US 20230409881 A1).
Regarding claim 3,
Allahdadian in view of Sarafijanovic-Djukic does not teach to reconstruct the first feature value, the program code is further structured to cause the processor circuit to: utilize a self-supervised neural network autoencoder.
Heitzmann, in the same field of endeavor, teaches to reconstruct the first feature value, the program code is further structured to cause the processor circuit to: utilize a self-supervised neural network autoencoder (Paragraph 74 of Heitzmann, "…the training of an auto-encoder does not involve the use of labelled training data, as such an ANN is simply trained to replicate, at the output of the network, the values at the input of the network. Thus, an auto-encoder is trained via self-supervised learning", Paragraph 91, “control circuit 504 may be implemented by dedicated hardware, or by a computer program executed by one or more processors”
Heitzmann teaches an autoencoder artificial neural network trained via self-supervised learning by replicating its input at its output without labelled data.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's autoencoder-based drift detection in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Heitzmann's self-supervised autoencoder training in order to enable training of the autoencoder on unlabeled data (Paragraph 74 of Heitzmann).
Claim 10 recites similar limitations to claim 3. Therefore, claim 10 is rejected using the same rationale as claim 3.
Claim 17 recites similar limitations to claim 3. Therefore, claim 17 is rejected using the same rationale as claim 3.
Claims 4-5, 11-12, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Allahdadian (US 20230043993 A1) in view of Sarafijanovic-Djukic ("Fast Distance-based Anomaly Detection in Images Using an Inception-like Autoencoder", 2020) and in further view of Lee (US 20240311531 A1).
Regarding claim 4,
Allahdadian further teaches wherein to receive the first feature value, the program code is further structured to cause the processor circuit to: receive an input feature vector comprising the first feature value and a second feature value; and to receive the first importance value for the first feature value, the program code is further structured to cause the processor circuit to: (Paragraph 39 of Allahdadian, "an input tuple is provided as a feature vector ... that contains a respective value for each of all features 121-122", Paragraph 124, “Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504.”
Allahdadian discloses an input tuple provided as a feature vector that contains a value for each of multiple features, which corresponds to the receiving of an input feature vector comprising the first feature value and a second feature value.).
Allahdadian in view of Sarafijanovic-Djukic does not teach sum a plurality of importance values comprising the first importance value and a second importance value corresponding to the second feature value, resulting in a summed value; and for each importance value of the plurality of importance values, divide the importance value by the summed value, thereby normalizing the importance value.
Lee, in the same field of endeavor, teaches sum a plurality of importance values comprising the first importance value and a second importance value corresponding to the second feature value, resulting in a summed value (Paragraph 47 of Lee, "These can then be normalized to a value between 0 and 1 by dividing by the sum of all feature importance values: "
Lee discloses summing all feature importance values to produce a sum used as the normalization denominator, which corresponds to the claimed summing of a plurality of importance values resulting in a summed value.).
Lee further teaches and for each importance value of the plurality of importance values, divide the importance value by the summed value, thereby normalizing the importance value (Paragraph 47, "These can then be normalized to a value between 0 and 1 by dividing by the sum of all feature importance values", See Equation 4 Corresponding to Paragraph 47,
PNG
media_image1.png
40
203
media_image1.png
Greyscale
Lee discloses dividing each feature importance value by the sum of all feature importance values to produce normalized values between 0 and 1, which corresponds to the dividing of each importance value by the summed value to normalize the importance value.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's importance-weighted reconstruction loss in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Lee's importance normalization technique in order to provide a consistent scale for comparing and evaluating importance values across features (Paragraph 44 of Lee).
Regarding claim 5,
Allahdadian teaches to weight the re-construction loss, the program code is further structured to cause the processor circuit to: weight the re-construction loss of the autoencoder using the plurality of normalized importance values as weights (Paragraph 92 of Allahdadian, “In an autoencoder… step 301 may calculate gradients using backpropagation… can be used with any reconstructive model 140 of any (e.g. opaque) model architecture that provides original (i.e. unweighted) reconstruction loss as value losses and/or tuple loss, step 301 may calculate gradients using partial derivatives based on original reconstruction losses and values of features of labeled tuples.”, Paragraph 64 of Allahdadian, "losses of features 121-122 are respectively adjusted by applying importances 130 as respective weighting coefficients", Paragraph 97, “…calculating importances 130 occurs in two phases that are gradient measurement and weight normalization. In a neural network embodiment such as an autoencoder, gradient measurement may calculate gradients using backpropagation as explained later herein.”
Allahdadian discloses weighting the per-feature reconstruction losses by applying the importance values as weighting coefficients for the autoencoder.).
Regarding claim 11,
Allahdadian teaches receiving the first importance value comprises: (Paragraph 39 of Allahdadian, "an input tuple is provided as a feature vector ... that contains a respective value for each of all features 121-122"
Allahdadian discloses an input tuple provided as a feature vector that contains a value for each of multiple features.):
Allahdadian in view of Sarafijanovic-Djukic does not teach summing a plurality of importance values comprising the first importance value and a second importance value corresponding to a second feature value, resulting in a summed value, and for each importance value of the plurality of importance values, dividing the importance value by the summed value, thereby normalizing the importance value.
Lee, in the same field of endeavor, teaches summing a plurality of importance values comprising the first importance value and a second importance value corresponding to a second feature value, resulting in a summed value (Paragraph 47 of Lee, "These can then be normalized to a value between 0 and 1 by dividing by the sum of all feature importance values"
Lee discloses computing a sum of all feature importance values as the denominator used for normalization, which corresponds to the claimed summing of a plurality of importance values resulting in a summed value.).
Lee further teaches and for each importance value of the plurality of importance values, dividing the importance value by the summed value, thereby normalizing the importance value (Paragraph 47 of Lee, "These can then be normalized to a value between 0 and 1 by dividing by the sum of all feature importance values", See Equation 4 Corresponding to Paragraph 47,
PNG
media_image1.png
40
203
media_image1.png
Greyscale
Lee teaches computing a sum of all feature importance values as the denominator used for normalization.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Allahdadian's importance-weighted reconstruction loss in view of Sarafijanovic-Djukic's convolutional autoencoder architecture with Lee's importance normalization technique in order to provide a consistent scale for comparing and evaluating importance values across features (Paragraph 44 of Lee).
Claim 12 recites similar limitations to claim 5. Therefore, claim 12 is rejected using the same rationale as claim 5.
Claim 18 recites similar limitations to claim 11. Therefore, claim 18 is rejected using the same rationale as claim 11.
Claim 19 recites similar limitations to claim 12. Therefore, claim 19 is rejected using the same rationale as claim 12.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAJD MAHER HADDAD whose telephone number is (571)272-2265. The examiner can normally be reached Mon-Friday 8-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.M.H./Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights