DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 10/16/2025 for application number 17/897,760. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
35 USC 101
On page 8 of the remarks section, Applicant argues that the current amended claims are not directed to an abstract idea. The claims recite a specific, technical solution for extracting and transferring feature representations between models, and for augmenting and training destination models in a way that cannot be performed in the human mind. Key features of the current amended claims that demonstrate they are not abstract: 1) Accessing, at a computing machine, a source model, wherein the source model is an artificial intelligence or a statistical model, configured to compute an output value based on an input value vector comprising values for each of a set of features.
2) Calculating an influence value for one or more features from the set of features, the
influence value of each feature indicating a degree to which the feature affects the output of a model. 3) Determining, using a curve fitting engine and based on the calculated influence values, a curve function mapping the one or more features to the influence value of the one or more features. 4) Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model. 5) Generating feature values for the additional features of the augmented input feature set. 6) Creating a destination model by training a model based on the augmented input feature set and the generated feature values. 7) Utilizing the destination model to make an inference based on input values for the augmented input feature set. These operations require the use of computing machines, AI models, curve fitting engines, and retraining processes that are not capable of being performed mentally or with pen and paper. For example, calculating influence values, curve fitting, and training models involve complex computations, large datasets, and iterative machine learning processes that far exceed human cognitive capabilities. The claims are thus directed to a specific technological process for model interoperability and efficiency, not to an abstract idea.
Examiner respectfully disagrees. The limitations “Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model,” “determining… a curve function mapping the one or more features to the influence value of the one or more features,” “creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model,” and “generating feature values for the additional features of the augmented input feature set” are steps that are recited in a high level of generality and are actions that can be performed mentally with the aid of pen and paper. Therefore, they are construed as mental processes. Given the Broadest Reasonable Interpretation (BRI), the claimed limitations as currently presented merely recite an observation, evaluation, judgment, or opinion (see MPEP § 2106.04(a)(2), subsection III). The limitation “using a curve fitting engine and based on the calculated influence values” was analyzed under Step 2A Prong Two as adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Applicant further argues the claimed invention provides a technical
solution to the problem of transferring feature influence information and improving the
performance of destination models in machine learning systems. The practical application is realized through the following features: “The use of a curve fitting engine to generate a curve function based on calculated influence values, which is then used to augment input features for a destination model,” “The generation of feature values for the augmented input feature set, enabling the destination model to be trained on richer, more informative data,” “The creation and utilization of a destination model that is trained on the augmented input feature set and generated feature values, resulting in improved model performance and resource efficiency,” and “The application of the invention to specific types of models (e.g., GBM, RNN, ANN) and computing environments (e.g., server farms, client devices), demonstrating real-world utility
and technical improvement.” Key features demonstrating practical application: “Accessing and processing models and data using computing machines,” “Calculating influence values and applying curve functions to augment features,” “Generating feature values and training destination models to improve performance,” “Utilizing trained destination models for inference on augmented input feature sets.” The Office's analysis for Step 2A Prong Two evaluated only fragments of the claims for practical application, which is incorrect. To determine if the claim recites a practical application, the Office must look at the claim as a whole, not just a section of the claim. Therefore, the §101
rejection is improper.
The Examiner respectfully disagrees. In regards to the explicit claim limitations “calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model,” “determining… a curve function mapping the one or more features to the influence value of the one or more features,” “creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model,” and “generating feature values for the additional features of the augmented input feature set” are construed to be abstract ideas and therefore under Step 2A Prong 1. The claimed limitations “accessing, at a computing machine, a source model wherein the source model is an artificial intelligence or a statistical model, wherein the source model is configured to compute an output value based on an input value vector, wherein the input value vector comprises values for each of a set of features,” “creating a destination model by training a model based on the augmented input feature set and the generated feature values,” and “utilizing the destination model to make an inference based on input values for the augmented input feature set” are analyzed under Step 2A Prong 2 as additional elements. The said additional elements are recited with a high level of generality, therefore do not integrate the abstract ideas into a practical application. Furthermore, the alleged features “enabling the destination model to be trained on richer, more informative data,” “resource efficiency,” and “the application of the invention to specific types of models (e.g., GBM, RNN, ANN) and computing environments (e.g., server farms, client devices), demonstrating real-world utility and technical improvement” appear to be disclosed invention. However, such disclosed invention as improvements are not explicitly reflected in the claimed invention. If the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. That is, the claim includes the components or steps of the invention that provide the improvement described in the specification. See MPEP 2106.04(d)(1)).
Applicant further argues that the claims are directed to a judicial exception, the claims recite additional elements that amount to significantly more than the exception. The claims require: “Specific technical steps for calculating influence values and curve functions using computing machines and engines,” “Generating feature values and training destination models, which are concrete, technical processes not performed by generic computers,” “Utilizing trained destination models for inference, enabling deployment and use in real-world machine learning systems,” and “Application to specific model architectures and computing environments, demonstrating technical improvement and utility.” These elements are not well-understood, routine, or conventional when considered as an ordered combination. The claims provide a specific, technical solution to a technical problem in the field of machine learning, and thus contain an inventive concept sufficient to transform any alleged abstract idea into patent-eligible subject matter.
Examiner respectfully disagrees. In regards to the argument “specific technical steps for calculating influence values and curve functions using computing machines and engines,” the specific technical steps are not recited in the claims in sufficient detail. In regards to the argument that “generating feature values and training destination models, which are concrete, technical processes not performed by generic computers,” the generating and training steps are written in a high level generality, lacking specific detailed steps of a specialized computer. In regards to the argument “utilizing trained destination models for inference (enabling deployment and use in real-world machine learning systems),” is written with a high level of generality without reciting specific steps of how the trained model is used to make inferences. In regard to the argument stating “Application to specific model architectures and computing environments” is not part of the claim language. If the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. That is, the claim includes the components or steps of the invention that provide the improvement described in the specification. See MPEP 2106.04(d)(1)).
Therefore, the 35 USC 101 rejection is maintained.
35 USC 103
On page 11 of the remarks section, Applicant argues that the cited art does not teach “calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model.” The Applicant further argues that the claimed influence value must specifically indicate the degree to which a feature affects the model output - essentially measuring the feature's impact or importance to the model's decision-making process. Li's contribution values do not indicate or measure the degree to which features affect the model output. Li provides no disclosure of measuring, determining, or indicating how much each feature affects the final model output.
Applicant’s arguments related to the cited limitation have been considered but are moot because the newly amended claim necessitated a new ground of rejection that does not rely on Li alone. Rather, Li in combination with newly applied prior art, Funaya, was used to address the said limitation.
Applicant further argues that the cited art does not teach “determining, using a curve fitting engine and based on the calculated influence values, a curve function mapping the one or more features to the influence value of the one or more features.” In this regard, Applicant further argues that paragraphs [0007] and [0036] of Li discloses no curve fitting engine, no curve function determination, and no mapping of features to influence values.
The Examiner respectfully disagrees. “Curve fitting engine,” “curve function,” and “mapping of features to influence values” are broad terms. These aforementioned terms are not further defined in the claim. Therefore, under BRI, Li does indeed teach the cited limitation. Specifically, Li teaches curve fitting engine (Para 0036, first order distance function), and a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value). In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
The Applicant further argues that the cited art does not teach “creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model.” In this regard, the Applicant further argues that paragraph [0034] states that the hidden layer contains nodes equal to the number of input nodes, with each intermediate node receiving a value from a corresponding input node and applying a transfer function to calculate outputs. This describes the processing of the original input features through the neural network, not the augmentation of the input feature set. The Applicant adds that paragraph [0034] does not disclose creating any augmented input feature set or adding additional features to the original set of features. Instead, Li describes a one-to-one correspondence between input nodes and intermediate nodes, where the same number of features are processed through transfer functions without any augmentation or addition of new features to the input set. The contribution values described are intermediate processing results within the neural network, not additional input features added to augment the original feature set.
The Examiner respectfully disagrees. “Augmented input feature set” and “adding additional features to the original set of features” are broad terms and are not further defined in the claim. Therefore, under the BRI, the newly amended cited limitations are indeed taught by Li in combination with Luo; not Li alone. Specifically, Li teaches creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function), while Luo teaches the additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
The Applicant further argues that the cited art Yoo does not teach generating feature values for the additional features of the augmented input feature set. In this regard, Applicant further argues paragraph [0009] of Yoo does not describe generating feature values for additional features that have been added to augment an input feature set and the processing described operates on already-existing feature maps rather than creating values for newly added augmented features.
Examiner respectfully disagrees. The newly amended cited limitations are taught in combination with Yoo and Luo; not Yoo alone. Specifically, Yoo teaches, generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map), while newly added prior art Luo teaches features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained]. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
The Applicant further argues that the cited art does not teach “creating a destination model by training a model based on the augmented input feature set and the generated feature values.” In this regard, the Applicant further adds that paragraph [0013] does not disclose training a model based on augmented input features and generated feature values, but instead describes architecturally combining two preexisting models by merging their intermediate feature map outputs.
The Examiner respectfully disagrees. Yoo does indeed teach “creating a destination model by training a model based on the augmented input feature set and the generated feature values” para 0013: “The embodiment augments, to form the augmented model configuration (creating a destination model), the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel,” and para 0015: “Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain (training a model based… the generated feature values), wherein the pretrained model is trained to operate on data of a first domain (training a model based on the augmented input feature set).” Additionally, the claim language is broad. Specifically, “training a model based on the augmented input feature set and the generated feature values” does not further define how the model is being trained. Therefore, under BRI, Yoo does indeed teach the limitations in question.
The Applicant further argues that the cited art Datta does not teach “the curve fitting engine mathematically combines the one or more features of the source model into a single feature based on the one or more features having a correlation with one another exceeding a threshold value or being sourced from a similar source.” In this regard, the Applicant adds that Datta's work on quantitative input influence measures describes summing influence values for explanation and reporting, not generating a new input feature for model training. No passage in Datta instructs mathematically merging correlated source features into a single feature for use in retraining a destination model. Thus, Datta does not teach the claimed limitation of combining features based on correlation thresholds or common sourcing to form a new feature input.
Examiner respectfully disagrees. The claim language “mathematically combines” is a broad term. The claim does not further define how the steps of combining are performed mathematically. Therefore, under the BRI, Datta does indeed teach the cited limitation. Specifically in Sect VII (B), para 2: “The figure on the left shows the influence of features on group disparity by Gender in the adult dataset; the figure on the right shows the influence of group disparity by Race in the arrests dataset” and Sect VII (E): “We report runtimes of our prototype for generating transparency reports on the adult dataset. Recall from Section VI that we approximate QII measures by computing (mathematically combines) sums (into a single feature) over samples of the dataset (the one or more features).”
Therefore, the 35 USC 103 rejection is maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 2A Prong 1:
Claim 1 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 2
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the joint influence corresponds to a correlation;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the joint influence corresponds to a correlation;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“storing a table data structure where columns represent the one or more features and rows represent the QII score;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“storing a table data structure where columns represent the one or more features and rows represent the QII score;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 5
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the influence value is computed based on a sum of gradients of an interpolation between a baseline value and the output value of the source model with respect to the one or more features, wherein the source model comprises an artificial neural network (ANN);” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the influence value is computed based on a sum of gradients of an interpolation between a baseline value and the output value of the source model with respect to the one or more features, wherein the source model comprises an artificial neural network (ANN);” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein gradients used in the sum of gradients are computed in parallel using multithreaded processing circuitry;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein gradients used in the sum of gradients are computed in parallel using multithreaded processing circuitry;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the curve function comprises a spline function;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the curve function comprises a spline function;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 8
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 2A Prong 1:
Claim 9 recites:
“[the curve fitting engine] mathematically combines the one or more features of the source model into a single feature based on the one or more features having a correlation with one another exceeding a threshold value or being sourced from a similar source;” Mathematically combining the one or more features of the source model into a single feature based on the one or more features being having a correlation with one another exceeding a threshold value or being sourced from a similar source is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“the curve fitting engine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“the curve fitting engine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored in a memory of the computing machine;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored in a memory of the computing machine;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 11
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored, in a data repository external to the computing machine, in a format that is accessible to the destination model for training the destination model;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored, in a data repository external to the computing machine, in a format that is accessible to the destination model for training the destination model;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model comprises a classifier gradient boosting machine (GBM) model, and wherein the destination model comprises a linear model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model comprises a classifier gradient boosting machine (GBM) model, and wherein the destination model comprises a linear model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model comprises a classifier recurrent neural network (RNN) model, and wherein the destination model comprises a classifier gradient boosting machine (GBM) model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model comprises a classifier recurrent neural network (RNN) model, and wherein the destination model comprises a classifier gradient boosting machine (GBM) model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 14
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the destination model utilizes fewer computing resources than the source model, wherein the computing resources comprise processing circuitry resources or memory resources;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the destination model utilizes fewer computing resources than the source model, wherein the computing resources comprise processing circuitry resources or memory resources;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 15
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model utilizes a server farm having a first amount of memory, and wherein the destination model utilizes a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model utilizes a server farm having a first amount of memory, and wherein the destination model utilizes a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16
Step 2A Prong 1:
Claim 16 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claims 17-19 are system claims that recite identical limitations to method claims 2-4. Therefore, claims 17-19 are rejected using the same rationale as claims 2-4.
Claim 20
Step 2A Prong 1:
Claim 20 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” “using a curve fitting engine and based on the calculated influence values;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 7, 8, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 20040042650 A1), hereinafter Li, in view of Yoo et al. (US 20210073624 A1), hereinafter Yoo, Funaya et al. (WO 2021235177 A1), hereinafter Funaya, and Luo et al. (CN 113362920 A), hereinafter Luo.
Regarding claim 1, Li teaches
A method [Para 0006, a method for determining if an input pattern is a member of an associated class] comprising: accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, wherein features are additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, Para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
The features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Regarding claim 7, Li-Yoo-Funaya-Luo teach all the limitations of claims 1.
Li further teaches,
wherein the curve function comprises a spline function [Para 0035, A number of basis functions are available for use as transfer functions in the claimed classifier; Para 0036, the contribution value is calculated by taking the absolute value of the difference between the feature value and a calculated mean value of this feature from the training set and dividing this result by a calculated standard deviation from the training samples (i.e. |x−μi|/σi)].
Regarding claim 8, Li-Yoo-Funaya-Luo teach all the limitations of claims 1.
Li further teaches,
wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer. [Para 0036, the contribution value is calculated by taking the absolute value of the difference between the feature value and a calculated mean value of this feature from the training set and dividing this result by a calculated standard deviation from the training samples (i.e. |x−μi|/σi)].
Regarding claim 16, Li teaches,
A system (Para 0006, data processing system) comprising: a memory (Para 0045, memory) comprising instructions (Para 0044, computer program); and one or more computer processors (Para 0006, data processing system) [Para 0006, a computer program product operative in a data processing system; Para 0044, FIG. 4 is a flow diagram illustrating the operation of a computer program 100 used to train a pattern recognition classifier via computer software; Para 0045, The actual training process begins at step 104 and proceeds to step 106. At step 106, the program retrieves a pattern sample from memory], wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo-Funaya teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Regarding claim 20, Li teaches,
accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
A tangible machine-readable storage medium including instructions [Para 0127, The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.] that, when executed by a machine, cause the machine to perform operations comprising:
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo-Funaya teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Claim(s) 2-4, 9, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Datta et al. (Algorithmic Transparency via
Quantitative Input Influence:
Theory and Experiments with Learning Systems, published 2016), hereinafter Datta.
Regarding claim 2, Li-Yoo-Funaya-Luo teach all the limitations of claim 1 including the relative influence value.
Li-Yoo-Funaya-Luo do not teach wherein influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features.
Datta teaches,
wherein influence value (Abstract, the degree of influence) is computed (Abstract, measures) based on a quantitative input influence (QII) (Abstract, QII) score computed based on a joint influence (Abstract, joint influence) of a set of one or more features (Abstract, set of inputs (e.g., age and income)) or a difference in outputs with or without the one or more features from the one or more features [Abstract, we introduce a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems… the QII measures also quantify the joint influence of a set of inputs (e.g., age and income) on outcomes (e.g. loan decisions)].
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide Quantitative Input Influence (QII) in order to bolster the calculation of a influence/contribution values by introducing a particular methodology that [Datta, Abstract] has been shown to provide better explanations than standard associative measures. This, in turn, leads to an improvement in machine learning predictions.
Regarding claim 3, Li-Yoo-Funaya-Luo-Datta teach all the limitations of claim 2.
Datta further teaches,
wherein the joint influence corresponds to a correlation [Sect 1, pg. 599, col 1, para 6, we seek measures of joint influence of a set of inputs (e.g., age and income) on a system’s decision; Sect 1, pg. 599, col 2, para 3, These measures (called Unary QII) model the difference in the quantity of interest when the system operates over two related input distributions—the real distribution and a hypothetical (or counterfactual) distribution that is constructed from the real distribution in a specific way to account for correlations among inputs.].
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide Quantitative Input Influence (QII) in order to bolster the calculation of a influence/contribution values by introducing a particular methodology that [Datta, Abstract] has been shown to provide better explanations than standard associative measures. This, in turn, leads to an improvement in machine learning predictions.
Regarding claim 4, Li-Yoo-Funaya-Luo-Datta teach all the limitations of claim 2.
Datta further teaches,
storing a table data structure where columns represent the one or more features and rows represent the QII score [Pg. 609, TABLE II: Comparison of QII with associative measures. For 4 different classifiers, we compute metrics such as Mutual Information (MI), Jaccard Index (JI), Pearson Correlation (corr), Group Disparity (disp) and Average QII between Gender and the outcome of the learned classifier. Each metric is computed in two situations: (A) when Gender is provided as an input to the classifier, and (B) when Gender is not provided as an input to the classifier].
PNG
media_image1.png
288
897
media_image1.png
Greyscale
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide storing a table data structure where columns represent the one or more features and rows represent the QII score in order to provide an intuitive and simple presentation of data to analyze trends.
Regarding claim 9, Li-Yoo-Funaya-Luo teach all the limitations of claims 1 including the curve fitting engine and the curve model.
Li-Yoo-Funaya-Luo do not teach, mathematically combines the one or more features into a single feature based on the one or more features having a correlation with one another, exceeding a threshold value, or being sourced from a similar source.
Datta teaches,
mathematically combines (Sect VII (E), computing) the one or more features (Sect VII (E), samples of the dataset) into a single feature (Sect VII (E), QII measures by computing sums) based on the one or more features having a correlation with one another, exceeding a threshold value, or being sourced from a similar source (Sect VII (E), the dataset). [Sect VII (B), para 2, The figure on the left shows the influence of features on group disparity by Gender in the adult dataset; the figure on the right shows the influence of group disparity by Race in the arrests dataset; Sect VII (E), We report runtimes of our prototype for generating transparency reports on the adult dataset. Recall from Section VI that we approximate QII measures by computing sums over samples of the dataset.]
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide combining multiple features into a single feature [Datta, Abstract] as summation efficiently measures influence in a simplified correlation of multiple inputs.
Claims 17-19 are system claims that recite identical limitations to method claims 2-4. Therefore, claims 17-19 are rejected using the same rationale as claims 2-4.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of
Sattarzadeh et al. (INTEGRATED GRAD-CAM: SENSITIVITY-AWARE VISUAL EXPLANATION OF DEEP
CONVOLUTIONAL NETWORKS VIA INTEGRATED GRADIENT-BASED SCORING, published May 13, 2021), hereinafter Sattarzadeh, and Bach et al. (JP 2020123329 A, see attached translation), hereinafter Bach.
Regarding claim 5, Li-Yoo-Funaya-Luo teach all the limitations of claim 1 including the source model.
Sattarzadeh teaches,
wherein the influence value is computed based on a sum of gradients (Fig. 2, addition result of ReLU) of an interpolation (Fig. 2, annotated box) between a baseline value (Fig. 2, Baseline) and the output value of the source model (Fig. 2, Input) with respect to the one or more features [Figure 2, illustrated below, Schematic of the proposed method considering that the baseline image is set to black and the path connecting the baseline and the input is set as a straight line.]
PNG
media_image2.png
787
1629
media_image2.png
Greyscale
Sattarzadeh is analogous to the claimed invention as they both relate to interpretable ML. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Sattarzadeh and provide calculating influence values based on a sum of gradients as accumulating gradients can provide a more stable update direction for model update parameters.
Li-Yoo-Funaya-Luo-Sattarzadeh do not teach wherein the source model comprises an artificial neural network (ANN).
Bach teaches,
wherein model comprises an artificial neural network (ANN) [Para 0012, It is therefore an object of the present invention to provide a concept for assigning relevance scores to a set of items to which an artificial neural network is applied, which concept is applicable to a broader range of artificial neural networks and/or reduces computational effort.].
Bach is analogous to the claimed invention as they both relate to interpretable ML. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, and Sattarzadeh’s teachings to incorporate the teachings of Bach and provide an ANN in order to analyze complex patterns, especially in non-linear relationships.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, Luo, Sattarzadeh and Bach, and in further view of Wolpert et al. (CIRCUITS FOR A VLSI-BASED STANDALONE BACKPROPAGATION NEURAL NETWORK, published 1992), hereinafter Wolpert.
Regarding claim 6, Li-Yoo-Funaya-Luo-Sattarzadeh-Bach teach all the limitations of claim 5 including the sum of gradients.
Li-Yoo-Funaya-Luo-Sattarzadeh-Bach do no teach wherein gradients are computed in parallel using multithreaded processing circuitry.
Wolpert teaches,
wherein gradients (DISCUSSION, back-propagation) are computed in parallel (Its processing is… parallel) using multithreaded processing circuitry (DISCUSSION, connectivity matrix) [DISCUSSION, The back-propagation neural network, when implemented in dedicated VLSI hardware, offers both aesthetic and tangible advantages over software-based methods. Aesthetically, the process of evaluation of input and weighting coefficients is performed in a parallel and simultaneous manner… Practically, these operations may occur at a higher rate than is possible in software. The connectivity matrix has a dedicated multiplier and summer for each synapse and output node respectively. Its processing is not only rapid, but truly parallel and distributed. The back-propagated error summation process is also achieved simultaneously and rapidly].
Wolpert is analogous to the claimed invention as they both relate to calculating gradients via neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, Sattarzadeh, and Bach’s teachings to incorporate the teachings of Wolpert and provide gradients computed in parallel using multithreaded processing circuitry [Wolpert, DISCUSSION] these operations occur at a higher rate than is possible in software.
Claim(s) 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Vemuri et al. (US 11704535 B1), hereinafter Vemuri.
Regarding claim 10, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the generated feature values for features of the augmented input feature set.
Li-Yoo-Funaya-Luo do not teach, wherein feature values are stored in a memory of the computing machine.
Vemuri teaches,
wherein feature values (Col 4, lines 43-45, input feature maps) are stored (Col 4, lines 43-45, stores) in a memory (Col 4, lines 43-45, memory 106) of the computing machine [Fig. 1, host 102] [Col 4, lines 43-45, In one embodiment, the memory 106 stores… input feature maps].
Vemuri is analogous to the claimed invention as they both relate to neural network processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Vemuri and provide feature values stored in memory for further retraining of a model.
Regarding claim 11, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the generated feature values for features of the augmented input feature set and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein feature values are stored, in a data repository external to the computing machine, in a format that is accessible to model for training.
Vemuri teaches,
wherein feature values (Col 16, lines 2-8, feature-maps) are stored (Col 16, lines 2-8, stores), in a data repository external to the computing machine (Col 16, lines 2-8, external memory), in a format that is accessible to model (Col 4, lines 61-67, neural network) for training (Col 4, lines 61-67, training) [Col 4, lines 61-67 & col 5, lines 1-5, In FIG. 1, the allocated blocks 142 of the memory 140 comprise data of the neural network, including data derived when training the neural network. As with the memory 106 of the host computer 102, the detailed circuitry within the memory 140 is described below, but can include any type of volatile or nonvolatile memory. In one embodiment, the memory 140 includes an array of memory elements.
The DPE array 130 of the reconfigurable IC 120 has any number of DPEs (also referred to as kernel processors), and these DPEs of the DPE array 130 perform operations on the input data (e.g., data points of input feature maps) to generate output data (e.g., data points of output feature maps); Col 16, lines 2-8, The architecture of the DPEs dictates the data organization in the various buffers of the reconfigurable IC. In the exemplary embodiment, external memory stores the feature-maps].
Vemuri is analogous to the claimed invention as they both relate to neural network processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Vemuri and provide feature values stored in an external memory in order to bypass capacity limitations.
Claim(s) 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Montone et al. (US 20230168667 A1), hereinafter Montone.
Regarding claim 12, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the source model and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein model comprises a classifier gradient boosting machine (GBM) model, and wherein model comprises a linear model.
Montone teaches,
wherein model comprises a classifier gradient boosting machine (GBM) model, and wherein model comprises a linear model [Other machine learning algorithms that be leveraged include… Generalized Linear Models, Extreme Gradient Boosting].
Montone is analogous to the claimed invention as they both relate to utilizing a plurality of ML models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Montone and provide utilizing a GBM and a linear model in order to incorporate models that enhance versatility across tasks and flexibility of different data types.
Regarding claim 13, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the source model and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein model comprises a classifier recurrent neural network (RNN) model, and wherein comprises a classifier gradient boosting machine (GBM) model.
Montone teaches,
wherein model comprises a classifier recurrent neural network (RNN) model, and wherein comprises a classifier gradient boosting machine (GBM) model [Other machine learning algorithms that be leveraged include… recurrent neural network (RNN) modeling… Generalized Linear Models].
Montone is analogous to the claimed invention as they both relate to utilizing a plurality of ML models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Montone and provide a GBM for its high predictive accuracy and an RNN for its efficiency in parameter sharing.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Catalin (WO 2019013711 A1), hereinafter Catalin.
Regarding claim 14, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the destination model and the source model.
Li-Yoo-Funaya-Luo do not teach wherein destination model utilize fewer computing resources than source model, wherein the computing resources comprise processing circuitry resources or memory resources.
Catalin teaches,
wherein destination model (Para 0026, devices 20) utilize fewer computing resources (Para 0158, reduce the overall memory footprint) than source model (Para 0026, server 14), wherein the computing resources comprise processing circuitry resources or memory resources (Para 0026, memory footprint) [Para 0026, The backend server 14 may manage training of feature recognition software such as deep learning systems comprising and/or for incorporation into POS software applications… The backend server 14 may also manage distribution of the POS software applications and/or feature recognition software for download by and/or peer-to-peer transfer between a plurality of low-power mobile electronic devices 20; Para 0158, In addition, model compression may be performed in the training pipeline for improved performance of the CNN 100 on low-power mobile device networks...While quantization provides significant (and perhaps, the primary) performance boost, other compression steps for reducing the size of CNN 100 may contribute to viability for use in POS application 24. A smaller size may lower the electronic burden and/or power consumption associated with execution of the object recognition module of the CNN 100 at mobile electronic devices 20, may reduce the overall memory footprint of the POS application 24, and/or may reduce processor cache misses.].
Catalin is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Catalin and provide the destination model utilizing fewer computing resources than source model [Catalin, Para 0158] for improved performance of a neural network on low-power mobile device networks.
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, Luo, and Montone, and in further view of Catalin.
Regarding claim 15, Li-Yoo-Funaya-Luo-Montone teach the limitations of claims 13 including the destination model, the modified destination model, and the source model (see claim 1).
Li-Yoo-Funaya-Luo-Montone do not teach wherein source model utilizes a server farm having a first amount of memory, and wherein destination model utilize a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory.
Catalin teaches,
wherein source model (Para 0026, server 14) utilizes a server farm (Para 214, the processing elements may be located in a single location (e.g…. a server farm) having a first amount of memory (Para 0026, memory footprint), and wherein destination model (Para 0026, devices 20) utilize a client computing device having a second amount of memory (Para 0026, memory footprint), the second amount of memory being less than the first amount of memory (Para 0026, reduce the overall memory footprint) [Para 0026, The backend server 14 may manage training of feature recognition software such as deep learning systems comprising and/or for incorporation into POS software applications… The backend server 14 may also manage distribution of the POS software applications and/or feature recognition software for download by and/or peer-to-peer transfer between a plurality of low-power mobile electronic devices 20; Para 0158, In addition, model compression may be performed in the training pipeline for improved performance of the CNN 100 on low-power mobile device networks...While quantization provides significant (and perhaps, the primary) performance boost, other compression steps for reducing the size of CNN 100 may contribute to viability for use in POS application 24. A smaller size may lower the electronic burden and/or power consumption associated with execution of the object recognition module of the CNN 100 at mobile electronic devices 20, may reduce the overall memory footprint of the POS application 24, and/or may reduce processor cache misses; Para 214, In some example embodiments, the processing elements may be located in a single location (e.g., within a home environment, an office environment or as a server farm].
Catalin is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, and Montone’s teachings to incorporate the teachings of Catalin and provide the destination model utilizing fewer computing resources than source model [Catalin, Para 0158] for improved performance of a neural network on low-power mobile device networks.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED RAYHAN AHMED whose telephone number is (571)270-0286. The examiner can normally be reached Mon-Fri ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED RAYHAN AHMED/Examiner, Art Unit 2126
/VAN C MANG/Primary Examiner, Art Unit 2126