DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 04/06/2026 for application number 17/897,760. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/06/2026 has been entered.
Response to Arguments
35 USC 101
On pages 8 and 9 of the remarks section, Applicant argues that the claim requires "accessing, at a computing machine, a source model, wherein the source model is an artificial intelligence or a statistical model" requires interfacing with complex AI/statistical models that process high-dimensional data structures and perform computations involving potentially millions of parameters-operations that cannot practically be performed by the human mind. Additionally, the claimed limitation "creating a destination model by training a model based on the augmented input feature set and the generated feature values" requires executing complex machine learning training algorithms (e.g., backpropagation in neural networks, gradient boosting optimization) on augmented datasets, involving iterative parameter updates through potentially thousands of epochs-computational operations far beyond human mental
capacity.
The Examiner respectfully disagrees. The claimed limitations "accessing, at a computing machine, a source model, wherein the source model is an artificial intelligence or a statistical model" and "creating a destination model by training a model based on the augmented input feature set and the generated feature values" were not analyzed under Step 2A Prong One as being abstract ideas. Rather the aforementioned claims were analyzed as additional elements under Step 2A Prong Two as being additional elements directed to the judicial exceptions.
On page 8 of the remarks section, Applicant further argues that the limitation "calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model" requires quantitative computational analysis of how individual features affect model predictions. This involves processing potentially thousands of data points through complex model architectures to determine mathematical influence metrics (such as QII scores or Integrated Gradients involving gradient computations). Such calculations require sophisticated computational techniques that cannot practically be performed mentally or with pen and paper, particularly given the scale and complexity of modem machine learning models.
The Examiner respectfully disagrees. After further consideration, the claimed limitation "calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model" was analyzed under Step 2A Prong One as being an abstract idea for reciting a mathematical calculation.
On page 8 of the remarks section, the Applicant further argues that the limitation "determining, using a curve fitting engine and based on the calculated influence values, a curve function mapping the one or more features to the influence value of the one or more features" requires automated computational curve fitting (e.g., spline fitting using algorithms like numpy's polyfit function) on large datasets. This involves iterative optimization algorithms and numerical methods that are computationally intensive and cannot practically be performed in the human mind.
The Examiner respectfully disagrees. The step of "determining… a curve function mapping the one or more features to the influence value of the one or more features" is recited at a high level of generality, lacking the necessary steps and details to fall outside the BRI of an abstract idea. Moreover, the limitation “using a curve fitting engine and based on the calculated influence values” was analyzed under Step 2A Prong Two as an additional element. After further consideration, the Examiner has analyzed the limitation as generically applying the abstract idea ("determining… a curve function mapping the one or more features to the influence value of the one or more features") without sufficient detail as to how the outcome is accomplished. Moreover, the steps and details of the “optimization algorithms and numerical methods that are
computationally intensive” are not sufficiently recited in the claimed limitation.
On page 8 of the remarks section, the Applicant further argues that the limitation "creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model" requires programmatic manipulation of data structures representing feature sets in machine learning systems, involving memory allocation, data structure modification, and systematic feature engineering at scale.
The Examiner respectfully disagrees. The step of "creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model" is recited at a high level of generality, lacking the necessary steps and details to fall outside the BRI of an abstract idea. Although appears to be disclosed invention, the steps and details pertaining to programmatic manipulation of data structures representing feature sets in machine learning systems and systematic feature engineering are explicitly recited in the claimed limitation. Therefore, the step of creating an augmented input feature set was analyzed under Step 2A Prong One as being an action that can be performed in the human mind with the aid of pen and paper.
On page 9 of the remarks section, the Applicant further argues that the claimed limitation "generating feature values for the additional features of the augmented input feature set" requires applying computational transformations (curve functions) to potentially millions of records in training datasets, storing results in appropriate data structures (pandas DataFrames, Tensors, etc.), which cannot practically be performed manually.
The Examiner respectfully disagrees. The claimed limitation "generating feature values for the additional features of the augmented input feature set" is recited at a high level of generality, lacking the necessary steps and details to fall outside the BRI of an abstract idea. Without the necessary steps and details, the action of generating feature values is interpreted as an action that be performed mentally in the human mind. Moreover, although appears to be disclosed invention, the steps of applying computational transformations to millions of records are not explicitly recited in the claimed limitation. If the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. That is, the claim includes the components or steps of the invention that provide the improvement described in the specification.
On pages 10 and 11 of the remarks section, the Applicant argues that the Office's analysis for Step 2A Prong Two evaluated only fragments of the claim for
practical application, which is incorrect. To determine if the claim recites a practical
application, the Office must look at the claim as a whole, not just isolated sections of the claim. Moreover, the practical application to which the claims are directed is the technological improvement of machine learning model performance through a specific method of extracting and transferring feature representations between models of different types. The ordered combination of calculating influence values, determining curve functions, creating augmented feature sets, generating feature values, and training destination models provides a specific technical methodology for transferring learned representations between heterogeneous model types. This solves the technical problem of model selection in machine learning (balancing competing criteria including
evaluative performance metrics, computational resource requirements, and model interpretability) and different model architectures learning features in fundamentally different ways and cannot directly share representations. The specification describes the claimed invention provides a specific technological solution: enabling a destination model (which satisfies desired computational resource and interpretability constraints) to achieve improved evaluative performance by augmenting its input feature set with features extracted from a more complex source model through influence analysis
and curve fitting, as well as concrete benefits including improved evaluation metrics (accuracy, precision, recall, Fl-score, AUC), reduced computational resource requirements, and maintained interpretability-all improvements to the machine learning technology itself.
The Examiner respectfully disagrees. The Office Action sent on 01/08/2026 provides a Step 2A Prong Two analysis that determines that the limitations do not amount to significantly more than the judicial exception itself when considered individually and in combination. The cited additional elements, individually and in combination, do not amount to a practical application as the methods of extracting and transferring feature representations between models of different types are recited at a high level of generality such that the ordered combination of calculating influence values, determining curve functions, creating augmented feature sets, and generating feature values steps were interpreted, under BRI, to be abstract ideas in the form of mental processes and/or mathematical processes. Additionally, the training of destination models is directed to the judicial exception for generally linking transfer learning models to the abstract idea without specific details. The claim’s lack of specific components and/or steps does not suggest a specific technological solution.
On pages 10-12 of the remarks section, the Applicant argues that the practical application to which the claims are directed is the technological improvement
of machine learning model performance through a specific method of extracting and transferring feature representations between models of different types. The claim
recites a specific technical approach using influence values (satisfying mathematical properties such as the efficiency/completeness axiom that the sum of influences equals model score) and curve fitting (to extract feature representations while filtering out feature interactions). This is a particular solution to the technical problem of feature extraction from black-box models, not merely a generic idea of improving models.
This addresses the technical problem of model selection in machine learning, where practitioners must balance competing criteria, including evaluative performance metrics, computational resource requirements, and model interpretability. The specification describes that the claimed invention provides a specific technological solution: enabling a destination model (which satisfies desired computational resource and interpretability constraints) to achieve improved evaluative performance by augmenting its input feature set with features extracted from a more complex source model through influence analysis and curve fitting. This allows, for example, a linear model running on a client device to approach the performance of a complex neural network running on a server farm, while maintaining the computational efficiency and interpretability advantages of the simpler model.
The Examiner respectfully disagrees. Although appears to be disclosed invention, the step of “enabling a destination model” is not explicitly recited in the claimed limitation. Moreover, the claimed limitations that correspond to “augmenting input”, “influence analysis”, and “curve fitting” are described with a high level of generality such that they were analyzed under Step 2A Prong One as reciting abstract ideas. Therefore, in combination and individually, the limitations do not provide a specific technological solution.
On page 11 of the remarks section, Applicant further argues that the claim requires accessing specific types of computing artifacts (AI or statistical models) that are configured to compute outputs based on input vectors. This is not a mere field-of use limitation, but rather specifies the technological substrate upon which the claimed method operates.
The Examiner respectfully disagrees. The usage of “artificial intelligence” and “statistical model” are recited with a high level of generality such that they were analyzed as merely limiting the judicial exception to the field of use or technological environment of machine learning.
In summary of the arguments provided on pages 12 and 13 of the remarks section, the Applicant argues that the complete ordered combination-from accessing source models, through influence calculation and curve fitting, to augmented feature set creation, feature value generation, destination model training, and operational deployment for inference describes a complete technological system for improving machine learning model performance through feature transfer. These improvements are reflected in the claim limitations. For example, "creating a destination model by training a model based on the augmented input feature set and the generated feature values" reflects the improvement of enhanced destination model performance through feature augmentation. "Utilizing the destination model to make an inference based on input values for the augmented input feature set" reflects the practical deployment of the improved model in operational systems. The Examiner's characterization
of "using a curve fitting engine" as merely applying an abstract idea on a computer, without considering how this element contributes to the overall technological improvement to machine learning systems, is therefore improper. The specification clearly sets forth improvements in machine learning technology, including improved model performance, reduced computational resource usage, and enhanced feature representations. When the claims are evaluated as a whole, the ordered combination of limitations-from influence calculation through curve fitting to augmented feature set creation and destination model training-reflects the disclosed improvements to machine learning technology.
The Examiner respectfully disagrees. As mentioned above, even when considered in combination, the steps and components are described at a high level of generality such that the claimed limitations corresponding to influence calculation, creating augmented features, and generating feature values recite judicial exceptions. The claimed limitation corresponding to accessing a source model was determined to be insignificant extra solution activity for reciting well-understood, routine, or conventional activity. Upon further review, the claimed limitation corresponding to training a destination model is also described with a high level generality such that it generally links the abstract idea to transfer learning without providing specific steps and details. More specifically, the claimed limitations "creating a destination model by training a model based on the augmented input feature set and the generated feature values", “utilizing the destination model to make an inference based on input values for the augmented input feature set”, and “using a curve fitting engine” were analyzed under Step 2A Prong Two as generically applying the abstract idea without specific steps and details. Individually or in combination, these limitations cannot provide for practical application.
On pages 13 and 14 of the remarks section, the Applicant further argues that the claim requires implementation using computing machines accessing specific types of
computational models (AI or statistical models), which imposes meaningful technological limitations beyond any abstract concept. The limitation "utilizing the destination model to make an inference based on input values for the augmented input feature set" specifies operational deployment of the improved model in real-world machine learning systems, demonstrating practical technological application.
The Examiner respectfully disagrees. As mentioned above, in combination and individually, the usage of “artificial intelligence” and “statistical model” are recited with a high level of generality such that they were analyzed as merely limiting the judicial exception to the field of use or technological environment of machine learning. This cannot provide for an inventive concept. Additionally, the claimed limitation "utilizing the destination model to make an inference based on input values for the augmented input feature set" was analyzed under Step 2A Prong Two as generically applying the abstract idea without specific steps and details. Individually or in combination, this limitation cannot provide for an inventive concept.
On page 14 of the remarks section, the Applicant further argues that the requirement of "using a curve fitting engine" to determine curve functions based on calculated influence values specifies use of specific computational tools and algorithms (e.g., spline fitting algorithms) that provide a technological solution to the technical problem of extracting generalizable feature representations from complex models.
The Examiner respectfully disagrees. The limitation of "using a curve fitting engine" was analyzed under Step 2A Prong Two as being an additional element that generically applies the abstract idea without specific details. This cannot provide for an inventive concept when considered individually or in combination.
On page 14 of the remarks section, the Applicant further argues that the ordered combination of calculating influence values satisfying mathematical properties (sum of influences equals model output), determining curve functions to map features to influences, and using these curve functions to create augmented feature sets represents an unconventional approach to feature engineering that is not well-understood, routine, or conventional in the field.
The Examiner respectfully disagrees. First, the claimed limitations corresponding to calculating influence values, determining curve functions, and using curve functions are all recited at high level of generality. Therefore, the ordered combination of steps does represent an unconventional approach. Second, as mentioned above, the claimed limitation corresponding to “calculating influence values” was analyzed under Step 2A Prong One as being an abstract idea for reciting a mathematical calculation. The claimed limitation corresponding to “determining curve functions” was analyzed under Step 2A Prong One as being an abstract idea for reciting a mental process. The step of “determining”, under BRI, is an action that can be performed mentally in the human mind. The claimed limitation corresponding to “using these curve functions to create augmented feature sets” was analyzed under Step 2A Prong One as being an abstract idea for reciting a mental process. The “creation” steps are not described in sufficient detail to fall outside the BRI of an action that be performed mentally in the human mind. These limitations cannot provide for an inventive concept when considered individually or in combination.
Lastly, on page 14 of the remarks section, the Applicant further that creating and
training a destination model using augmented input feature sets generated through the claimed process provides concrete technological improvements (improved evaluation metrics, reduced computational resources, maintained interpretability) that go beyond abstract concepts.
The Examiner respectfully disagrees. The claimed limitation corresponding to “creating… a destination model” cannot provide for a practical application that suggests a technological improvement as the limitation was analyzed under Step 2A Prong Two as generically linking the judicial exception to transfer learning without providing sufficient detail. This cannot provide for an inventive concept when considered individually or in combination. Although appears to be disclosed invention, claim 1 does not explicitly recite a limitation “training a destination model”. If the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. That is, the claim includes the components or steps of the invention that provide the improvement described in the specification.
Therefore, the 35 USC 101 rejection is maintained.
35 USC 103
In summary of page 16 of the remarks section, the Applicant argues that the cited art does not teach “calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the feature affects the output of a model”. Further, it is noted that paragraph 0007 does not include the term "influence value." Thus, the Examiner arguably describes a "contribution value" that arguably teaches the claimed influence value. However, since the Examiner agrees that Li, in combination with Yoo, does not teach the claimed influence value, it follows that the described "contribution value" is different from the
claimed influence value. Further, the claimed "influence value" must indicate the degree to which a feature affects the model output, a feature importance or attribution measure. Li's "contribution value" is an intermediate activation produced by applying a transfer function to a feature value during the model's forward computation. The "contribution value" does not measure or indicate how much a feature affects the model's output. The Examiner's mapping requires equating an intermediate computational value with a feature importance metric, which are functionally distinct concepts.
The Examiner respectfully disagrees. The term “influence value” is broad term, so Li’s disclosure of “contribution value” is indeed a valid teaching of claimed “influence value”. Furthermore, Li alone was cited as teaching the claimed “influence value”, but Li alone was not cited as teaching the claimed “influence values of each feature indicating a degree to which the feature affects the output of a model”. The introduction of reference Funaya to teach “indicating a degree to which the feature affects the output of a model” does not render the previous portion of the rejection improper. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On pages 17 and 18 of the remarks section, the Applicant further argues that the cited art does not teach “determining, using a curve fitting engine and based on the calculated influence values, a curve function mapping the one or more features to the influence value of the one or more features”. As discussed above, Li does not teach the claimed influence values; therefore, it is not possible that Li would teach determining, using a curve fitting engine, and based on the calculated influence values, a curve function, because Li does not teach the claimed influence values. Further, Li discloses a neural network classifier that applies a predefined transfer function (specifically, a first-order distance function of the form lx-μl/a) to transform feature values into contribution values during forward propagation. This transfer function is not a curve-fitting engine. A curve-fitting engine is a computational tool that accepts data points as input and derives a function that fits or approximates them. Li's first-order distance function is a predetermined mathematical formula-only its parameters (mean and standard deviation) are computed from training sample statistics. Computing statistical parameters for a known formula is not curve fitting. Furthermore, claim 1 requires "determining a curve function" through curve fitting. Li does not determine its transfer function through any fitting process. The claimed curve function must be a fitted representation of the
feature-to-influence relationship derived from calculated influence values.
The Examiner respectfully disagrees. As mentioned above, the term “influence value” is broad term, so Li’s disclosure of “contribution value” is indeed a valid teaching of claimed “influence value”. Additionally, a curve-fitting engine is a broad term. Although appears to be disclosed invention, a curve-fitting engine being “a computational tool that accepts data points as input and derives a function that fits or approximates them” is not explicitly recited the claimed limitation. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Therefore, under BRI, Li’s “transfer function” is a reasonable teaching of the claimed “curve-fitting engine”. However, the application would be in a better position for allowability if the Applicant chooses to amend the currently recited claim limitation by narrowing the scope of the claimed “curve-fitting engine” in related manner to a curve-fitting engine being “a computational tool that accepts data points as input and derives a function that fits or approximates them”.
On page 18 of the remarks section, the Applicant further argues that the claim requires that the curve fitting process be "based on the calculated influence values," meaning that previously calculated influence values serve as input to the curve fitting engine. The contribution values in Li are outputs produced by applying the transfer function to feature values they do not exist before the transfer function is applied and therefore cannot serve as the basis for determining that function. The causal relationship required by the claim is reversed in Li.
The Examiner respectfully disagrees. Although appears to be disclosed invention, the claimed limitation does not explicitly recite “previously calculated influence values serve as input to the curve fitting engine”. Rather, the claimed limitation suggests that a “curve function mapping” is determined based on “using a curve fitting engine” and “based on the calculated influence values”. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). However, if the Applicant amends the claimed limitation to reflect “previously calculated influence values serve as input to the curve fitting engine”, the 35 USC 103 rejection may be overcome, which would trigger further and consideration.
On page 18 of the remarks section, the Applicant further argues that the claim also requires that the curve function map features to their influence values, where influence values indicate the degree to which features affect model output. Li's transfer function maps feature values to contribution values, which are intermediate activation values in the hidden layer of the neural network, not measures of feature influence on the model's final output.
The Examiner respectfully disagrees. As mentioned above, Li alone was not cited as teaching as teaching the limitation corresponding to “influence values indicate the degree to which features affect model output”. Rather, the introduction of reference Funaya to teach “indicating a degree to which the feature affects the output of a model” does not render the previous portion of the rejection improper. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On page 19 of the remarks section, the Applicant further argues that it can be inferred that Li may create an augmented input feature set, but this augmented input feature set is not used for anything, at least as it relates to claim 1. If an augmented input feature set cannot be used, then the teachings of Li are irrelevant with respect to this feature.
The Examiner respectfully disagrees. Li alone was not cited as teaching the limitation “creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model”. Rather, the combination Li-Yoo-Funaya-Luo teach the aforementioned limitation. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On page 19 of the remarks section, the Applicant further argues that the Office maps the "augmented input feature set" to the "output value, for each of reference" (i.e., contribution values) described in paragraph 0034. However, these contribution values are not an augmented input feature set. They are intermediate computational outputs within the hidden layer of the neural network, produced by applying a transfer function to input feature values during forward propagation. An augmented input feature set would be a set of input features that has been expanded beyond the original set, whereas Li's contribution values are transformed representations of the original features used for internal neural network processing. These contribution values are not input features to any model-they are intermediate activations passed from the hidden layer to the output layer within the same model.
The Examiner respectfully disagrees. While “an augmented input feature set would be a set of input features that has been expanded beyond the original set” appears to be disclosed invention, it is not explicitly recited the claimed limitation. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On page 20 of the remarks section, the Applicant argues that the assertion that Luo describes adding additional features to the feature set is improper. For Luo to add the features to the feature set, Luo would have to add the features as described on claim 1. However, Luo does not teach the claimed influence value or the claimed curve function, so Luo may add some features to some model, but Luo cannot add the features as defined in claim 1 either.
The Examiner respectfully disagrees. Luo individually was not cited as teaching the entirety of claim 1. Rather, the combination Li-Yoo-Funaya-Luo was cited as teaching the entirety of claim 1. Specifically, reference Li was cited as teaching the limitations corresponding to the “influence value” and “claimed curve function”. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On page 21 of the remarks section, Applicant further argues that if Luo adds the features to the feature set, as asserted by the Office, Yoo cannot now teach this feature set that has been created according to Luo. Again, Yoo may add some features to a feature set, but Yoo can not teach adding the claimed features created as described in claim 1.
The Examiner respectfully disagrees. The feature set taught by Yoo is not the same feature set taught by Luo. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On page 21 of the remarks section, Applicant further argues that the claim requires "generating feature values for the additional features of the augmented input feature set." This means (1) generating feature values, (2) for additional features, and (3) of the augmented input feature set. Each component presents a distinct deficiency in the Examiner's mapping to Yoo. Regarding "generating feature values," Yoo does not describe generating new feature values. Paragraph 0014 explicitly states that feature maps are "being output from a layer in the pretrained model" and "being output from a layer in the submodel." These feature maps are produced by the normal forward propagation of data through trained neural network architectures. The pretrained model processes input data through its layers according to its existing training, producing feature maps as intermediate representations. The submodel similarly processes data according to its training on the second domain. These are not newly generated values; they are the natural outputs of trained models processing data.
The Examiner respectfully disagrees. First, reference Yoo alone was not cited as teaching "generating feature values for the additional features of the augmented input feature set." Rather, the combination Li-Yoo-Funaya-Luo teaches the aforementioned claimed limitation. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Second, the currently recited claimed limitation features a very broad methodology of augmentation such that reference Yoo’s methodology of processing data teaches “generating feature values for features of the augmented input feature set”. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “newly generated values”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The application would be in a better position of allowability if the Applicant further amends the claim to narrow the scope of the methodology of augmentation.
On page 22 of the remarks section, the Applicant further argues that, regarding "for the additional features," Yoo's feature maps are not values for "additional features" that were added to an original feature set. Claim 1 requires generating values specifically for features that were newly added to expand an original feature set. Yoo describes extracting and combining existing intermediate representations from two different models-a fundamentally different concept.
The Examiner respectfully disagrees. First, as mentioned above, reference Yoo alone was not cited as teaching "generating feature values for the additional features of the augmented input feature set." Rather, the combination Li-Yoo-Funaya-Luo teaches the aforementioned claimed limitation. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Second, although appears to be disclosed invention, “generating values specifically for features that were newly added to expand an original feature set” is not explicitly recited in the claimed limitations. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On page 22 of the remarks section, the Applicant further argues that, regarding "of the augmented input feature set," Yoo's combined feature map is not an augmented input feature set. The claim requires an augmented input feature set-a dataset or data structure containing input features that has been expanded by adding new features to the original source model features.
The Examiner respectfully disagrees. As mentioned above, the methodology of augmentation recited in the claimed limitations is recited broadly such that reference Yoo, under BRI, particular features of the claimed invention. The application would be in a better position of allowability if the Applicant further amends the claim to narrow the scope of the methodology of augmentation. Furthermore, although appears to be disclosed invention, “an augmented input feature set-a dataset or data structure containing input features that has been expanded by adding new features to the original source model features” is not explicitly recited in the claimed limitation. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On pages 22 and 23 of the remarks section, the Applicant further argues that the claim requires generating feature values specifically "for the additional features" that were added in the prior claim limitation. As demonstrated in the analysis of the previous limitation, neither Li nor Yoo teaches adding additional features to a source model's feature set. Since no additional features were added to augment the original feature set, there can be no generation of values for those non-existent additional features. The claim describes a pipeline: (1) create an augmented input feature set by adding features, then (2) generate values for those added features. Without step (1) being taught, step (2) cannot be satisfied. Yoo describes an architectural pattern for combining two neural networks by coupling their layers and merging intermediate representations during forward propagation. Claim 1 describes a data preparation operation where values are computed for newly created features to populate an augmented dataset. These are different operations with different purposes, different inputs, and different outputs. Therefore, Yoo does not teach "generating feature values for the additional features of the augmented input feature set," as claimed.
The Examiner respectfully disagrees. References Li and Yoo were not cited in isolation in regards to teaching “generating feature values for the additional features of the augmented input feature set”. Rather, the combination Li-Yoo-Funaya-Luo were cited as teaching the aforementioned limitation. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Furthermore, as mentioned above, the data preparation operation recited in the claimed limitations is recited broadly such that reference Yoo’s methodology, under BRI, teaches particular features of the claimed invention.
On page 23 and in summary of page 24 of the remarks section, the Applicant further argues that the cited art does not teach creating a destination model by training a model based on the augmented input feature set and the generated feature values. Claim 1 requires that the destination model be created "by training a model"-meaning the act of training produces the destination model. In Yoo, the destination model (augmented model configuration) is created "by augmenting"-by architecturally coupling two models that have already been trained separately. The training happens first; the creation of the augmented model happens afterward through architectural combination. The claim's causal relationship-training creates the destination model-is not present in Yoo.
The Examiner respectfully disagrees. The claimed limitation explicitly recites “creating a destination model by training a model”. Under BRI, the claimed limitation is taught by any reference that teaches the creation of model by training a model, which Yoo indeed teaches. To clarify, in paragraph 13, Yoo teaches: “The embodiment augments, to form the augmented model configuration (creating a destination model), the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel”. To further clarify, in paragraph 0111, Yoo teaches, “Component 1310 trains (by training a model) submodel 1306 using training data 1304”. The Application would be in a better position of allowability should the Applicant decide to further narrow the “creating” and “training” steps in the claimed limitations.
On page 25 of the remarks section, claim 1 requires training "based on the augmented input feature set and the generated feature values." The Office maps "the pretrained model is trained to operate on data of a first domain" to "augmented input feature set," but this is simply the original training data for the pretrained model. It is not an augmented input feature set, a feature set that has been expanded by adding additional features to a source model's original features (as required by prior claim limitations). Yoo's training data is conventional: the pretrained model was trained on first-domain data, and the submodel is trained on second-domain training data. Neither constitutes an "augmented input feature set" in the claimed sense-a dataset containing original features plus newly added features with generated values.
The Examiner respectfully disagrees. In paragraph 0088, Yoo teaches: “Pretrained model 402 may include an initial convolution stage 402A, a final convolution stage 402B, or both depending on the particular implementation. Similarly, submodel 602 may, but need not necessarily include an initial convolution stage (not shown), final convolution stage 602B, or both depending on the particular implementation of submodel 602”. In paragraph 0090, Yoo further teaches, “augmentation 601 augments submodel 602 with pretrained model 402 by processing and combining one or more features output from one or more layers (some combination of convolution layer 402A, B 1, B2, B3, B4 . . . Bn, and even final convolution layer 402B) with one or more features from a layer in submodel 602 to build an input feature map for another layer in submodel 602”. Therefore, Yoo indeed augments data input into the pretrained model in order to form the augmented model configuration.
On page 25 of the remarks section, Applicant further argues that the Office maps "trains... the submodel using training data corresponding to a second domain" to "generated feature values." But Yoo's "training data corresponding to a second domain" is standard training data for the target application. It is not "the generated feature values"-values that were generated for additional features that were added to augment a source model's feature set. As demonstrated in the analysis of prior limitations, Yoo does not teach adding additional features to a source model's feature set or generating values for such additional features. Since those generated feature values do not exist in Yoo's disclosure, they cannot be used for training.
The Examiner respectfully disagrees. In paragraph 0088, Yoo teaches: “submodel 602 may, but need not necessarily include an initial convolution stage (not shown), final convolution stage 602B, or both depending on the particular implementation of submodel 602”. In paragraph 0090, Yoo further teaches, “augmentation 601 augments submodel 602 with pretrained model 402 by processing and combining one or more features output from one or more layers (some combination of convolution layer 402A, B 1, B2, B3, B4 . . . Bn, and even final convolution layer 402B) with one or more features from a layer in submodel 602 to build an input feature map for another layer in submodel 602”. Therefore, Yoo indeed generates feature values using the submodel in order to form the augmented model configuration. Additionally, Yoo alone was not cited as teaching the limitations corresponding to “additional features that were added to augment a source model's feature set”. Rather, the combination Li-Yoo-Funaya-Luo were cited as teaching the aforementioned limitations. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On page 26 of the remarks section, the Applicant further argues that Datta does not disclose the curve-fitting engine. The claim requires that "the curve fitting engine" (the specific tool from prior limitations) perform the mathematical combination. Without disclosure of the curve-fitting engine, this element cannot be satisfied.
The Examiner respectfully disagrees. Datta alone was not cited as teaching "the curve fitting engine". Rather, the combination Li-Yoo-Funaya-Luo-Datta were cited as teaching the "the curve fitting engine" and “the mathematical combination”. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Additionally, in response to applicant's argument that “Without disclosure of the curve-fitting engine, this element cannot be satisfied”, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
On pages 26 and 27 of the remarks section, the Applicant further argues that the Office incorrectly maps "one or more features" to "samples of the dataset." In machine learning and data analysis terminology, features and samples are distinct concepts. Samples (also called instances, data points, or records) are individual observations in a dataset-the rows. Features (also called attributes, variables, or dimensions) are the measured properties-the columns. Section VII(E) states "we approximate QII measures by computing sums over samples of the dataset"-this explicitly describes iterating over samples (rows), not combining features (columns). The claim requires combining "one or more features of the source model," meaning consolidating feature attributes. Datta describes summing across data samples to compute statistical measures, which operates in the sample space, not the feature space.
The Examiner respectfully disagrees. In Sect VII, paragraph 2, Datta teaches the usage of dataset such as “adult” and “arrests”. In Sect VII, paragraph 2, Datta teaches “This standard machine learning benchmark dataset is a subset of US census data that classifies the income of individuals, and contains factors such as age, race, gender, marital status and other socio-economic parameters”. In same passage, Datta further teaches “From this dataset, we extract the following features: age, gender, race, region, history of drug use, history of smoking, and history of arrests”. Therefore, the cited “samples” are indeed “features”. Moreover, the claimed limitation recites a “combination”. A summation, under BRI, would teach the claimed “combination”.
On page 27 of the remarks section, the Applicant further argues that the Examiner maps "into a single feature" to "QII measures by computing sums." QII (Quantitative Input Influence) measures are quantitative metrics-numerical outputs that represent the degree of influence a feature has on outcomes. QII measures are not "features" in the machine learning sense. A feature is an input variable to a model (e.g., age, income, gender). A QII measure is a derived metric about a feature (e.g., "Gender has QII value 0.036"). Section VII(B) illustrates this: "The figure on the left shows the influence of features on group disparity by Gender"-Gender is a feature, and its influence is measured by QII, producing a numerical metric. Datta computes QII measures for individual features, not consolidated features created by combining multiple input features. The output of Datta's computation is a measurement, not a merged feature that could serve as model input.
The Examiner respectfully disagrees. Under BRI, a derived metric about a feature is equivalent to a feature. Therefore, Datta is not mischaracterized in this regard.
On page 27 of the remarks section, Applicant further argues that claim 9 requires that the mathematical combination be "based on the one or more features having a correlation with one another exceeding a threshold value or being sourced from a similar source." This specifies decision criteria: the curve fitting engine must evaluate whether features are sufficiently correlated (above a threshold) or share similar provenance, and perform a combination only when these criteria are met. Datta provides no such disclosure. The Examiner vaguely maps these criteria to "the dataset," but a dataset is simply a collection of data, not a correlation threshold or a source-similarity evaluation criterion. There is no disclosure of the conditional logic required by the claim
The Examiner respectfully disagrees. First, although appears to be disclosed invention, “the curve fitting engine must evaluate whether features are sufficiently correlated (above a threshold) or share similar provenance, and perform a combination only when these criteria are met” is not explicitly recited the claimed limitations. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Second, a source-similarity evaluation is not explicitly recited in the claimed limitation. However, the application would be in a better position of allowability should the Applicant decide to make amendments narrowing the scope of the “decision criteria”.
On page 27, the Applicant further argues that even if Datta's summation could be characterized as "mathematical combination" (which it cannot, as explained above), the operation occurs in the wrong computational space. The claim requires combining "one or more features ... into a single feature"-a dimensionality reduction operation in feature space where multiple feature columns are merged into one feature column. Datta describes "computing sums over samples of the dataset"-an aggregation operation in sample space where values are summed across data point rows to produce a statistical measure.
The Examiner respectfully disagrees. The term “combining” is much broader than “a dimensionality reduction operation in feature space where multiple feature columns are merged into one feature column”. Therefore, under BRI, Datta’s aggregation teaches the claimed “mathematically combines”. However, the Application would be in a better position of allowability if the claimed limitations were amended to recite the alleged difference.
On page 28 of the remarks section, the Applicant further teaches the Examiner's response misapplies the broadest reasonable interpretation (BRI) standard by conflating fundamental technical concepts and ignoring explicit claim language. The Examiner's interpretation that "samples of the dataset" teach "one or more features" is technically incorrect. Datta's statement "computing sums over samples of the dataset" explicitly describes iteration over data instances (rows), not consolidation of feature attributes (columns). The Office's interpretation mixes row-wise operations with column-wise operations, which is technically incoherent-equivalent to claiming that "summing prices of items" teaches "combining item categories into a single category."
The Examiner respectfully disagrees. As mentioned above, Datta’s “samples” are equivalent to the claimed “features”. In Sect VII, paragraph 2, Datta teaches “This standard machine learning benchmark dataset is a subset of US census data that classifies the income of individuals, and contains factors such as age, race, gender, marital status and other socio-economic parameters”. In same passage, Datta further teaches “From this dataset, we extract the following features: age, gender, race, region, history of drug use, history of smoking, and history of arrests”. Therefore, the cited “samples” are indeed “features”. Additionally, the claimed limitations do not reflect the claimed “mathematically combines” as being “column-wise operations”.
On page 29, the Applicant further argues that the Examiner argues that "mathematically combines" is broad, but the claim explicitly specifies: "mathematically combines the one or more features of the source model into a single feature." BRI must be reasonable. An interpretation that equates "samples" with "features" and "QII measures" with "a single feature" proves to be unreasonable. Such a perspective contradicts established technical terminology, ignores the explicit language of claims, conflates operations that exist in orthogonal dimensions, and ultimately renders the terms within claims meaningless.
The Examiner respectfully disagrees. The claimed “combination” lacks particular steps and details such as the alleged “column-wise operations” or “dimensionality reduction”. Additionally, as shown above, Datta’s “samples” are equivalent to the claimed “features”.
Independent claims 16 and 20 recite similar limitations to claim 1. Therefore, the response to arguments follows the same rationale.
The 35 USC 103 rejection is maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 2A Prong 1:
Claim 1 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 2
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the joint influence corresponds to a correlation;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the joint influence corresponds to a correlation;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“storing a table data structure where columns represent the one or more features and rows represent the QII score;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“storing a table data structure where columns represent the one or more features and rows represent the QII score;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 5
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the influence value is computed based on a sum of gradients of an interpolation between a baseline value and the output value of the source model with respect to the one or more features, wherein the source model comprises an artificial neural network (ANN);” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the influence value is computed based on a sum of gradients of an interpolation between a baseline value and the output value of the source model with respect to the one or more features, wherein the source model comprises an artificial neural network (ANN);” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein gradients used in the sum of gradients are computed in parallel using multithreaded processing circuitry;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein gradients used in the sum of gradients are computed in parallel using multithreaded processing circuitry;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the curve function comprises a spline function;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the curve function comprises a spline function;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 8
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 2A Prong 1:
Claim 9 recites:
“[the curve fitting engine] mathematically combines the one or more features of the source model into a single feature based on the one or more features having a correlation with one another exceeding a threshold value or being sourced from a similar source;” Mathematically combining the one or more features of the source model into a single feature based on the one or more features being having a correlation with one another exceeding a threshold value or being sourced from a similar source is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“the curve fitting engine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“the curve fitting engine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored in a memory of the computing machine;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored in a memory of the computing machine;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 11
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored, in a data repository external to the computing machine, in a format that is accessible to the destination model for training the destination model;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the generated feature values for features of the augmented input feature set are stored, in a data repository external to the computing machine, in a format that is accessible to the destination model for training the destination model;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model comprises a classifier gradient boosting machine (GBM) model, and wherein the destination model comprises a linear model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model comprises a classifier gradient boosting machine (GBM) model, and wherein the destination model comprises a linear model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model comprises a classifier recurrent neural network (RNN) model, and wherein the destination model comprises a classifier gradient boosting machine (GBM) model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model comprises a classifier recurrent neural network (RNN) model, and wherein the destination model comprises a classifier gradient boosting machine (GBM) model;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 14
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the destination model utilizes fewer computing resources than the source model, wherein the computing resources comprise processing circuitry resources or memory resources;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the destination model utilizes fewer computing resources than the source model, wherein the computing resources comprise processing circuitry resources or memory resources;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 15
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the source model utilizes a server farm having a first amount of memory, and wherein the destination model utilizes a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the source model utilizes a server farm having a first amount of memory, and wherein the destination model utilizes a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16
Step 2A Prong 1:
Claim 16 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claims 17-19 are system claims that recite identical limitations to method claims 2-4. Therefore, claims 17-19 are rejected using the same rationale as claims 2-4.
Claim 20
Step 2A Prong 1:
Claim 20 recites:
“calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model;” Calculating an influence value for one or more features from the set of features, the influence value of each feature indicating a degree to which the features affects the output of a model is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
“determining, [using a curve fitting engine and based on the calculated influence values,] a curve function mapping the one or more features to the influence value of the one or more features;” Determining a curve function mapping the one or more features to the influence value of the one or more features is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model;” Creating an augmented input feature set based on the curve function to add additional features to the set of features of the source model is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“generating feature values for the additional features of the augmented input feature set;” Generating feature values for the additional features of the augmented input feature set is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“accessing, at a computing machine, a source model;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “accessing” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“at a computing machine;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“wherein the source model is an artificial intelligence or a statistical model;” “wherein the input value vector comprises values for each of a set of features;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“using a curve fitting engine and based on the calculated influence values;” “wherein the source model is configured to compute an output value based on an input value vector;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“creating a destination model by training a model based on the augmented input feature set and the generated feature values;” “utilizing the destination model to make an inference based on input values for the augmented input feature set;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 7, 8, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 20040042650 A1), hereinafter Li, in view of Yoo et al. (US 20210073624 A1), hereinafter Yoo, Funaya et al. (WO 2021235177 A1), hereinafter Funaya, and Luo et al. (CN 113362920 A), hereinafter Luo.
Regarding claim 1, Li teaches
A method [Para 0006, a method for determining if an input pattern is a member of an associated class] comprising: accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, wherein features are additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, Para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
The features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Regarding claim 7, Li-Yoo-Funaya-Luo teach all the limitations of claims 1.
Li further teaches,
wherein the curve function comprises a spline function [Para 0035, A number of basis functions are available for use as transfer functions in the claimed classifier; Para 0036, the contribution value is calculated by taking the absolute value of the difference between the feature value and a calculated mean value of this feature from the training set and dividing this result by a calculated standard deviation from the training samples (i.e. |x−μi|/σi)].
Regarding claim 8, Li-Yoo-Funaya-Luo teach all the limitations of claims 1.
Li further teaches,
wherein the curve fitting engine maps feature values to feature influences using a n-degree polynomial, wherein n is a positive integer. [Para 0036, the contribution value is calculated by taking the absolute value of the difference between the feature value and a calculated mean value of this feature from the training set and dividing this result by a calculated standard deviation from the training samples (i.e. |x−μi|/σi)].
Regarding claim 16, Li teaches,
A system (Para 0006, data processing system) comprising: a memory (Para 0045, memory) comprising instructions (Para 0044, computer program); and one or more computer processors (Para 0006, data processing system) [Para 0006, a computer program product operative in a data processing system; Para 0044, FIG. 4 is a flow diagram illustrating the operation of a computer program 100 used to train a pattern recognition classifier via computer software; Para 0045, The actual training process begins at step 104 and proceeds to step 106. At step 106, the program retrieves a pattern sample from memory], wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo-Funaya teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Regarding claim 20, Li teaches,
accessing (Para 0013, classifying), at a computing machine [Para 0007, a computer program product operative in a data processing system is disclosed], a source model (Para 0013, binary optimal neural network classification system), wherein the source model is an artificial intelligence (Para 0013, neural network) or a statistical model [Para 0013, In accordance with the present invention, a method for classifying an input pattern via a binary optimal neural network classification system is described], wherein the source model is configured to compute (Para 0007, determine) an output value (Para 0007, determine a classification result) based on an input value vector (Para 0007, input pattern), wherein the input value vector comprises values for each of a set of features (Para 0007, numerical feature value for each feature from the extracted feature data) [Para 0007, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result.];
Calculating an influence value (Para 0007, calculates a contribution value) for one or more features (Para 0007, for each feature value) from set of features (Para 0007, feature from the extracted feature data) [Para 0007, First, a feature extraction stage extracts data from a plurality of preselected features within the input pattern and determines a numerical feature value for each feature from the extracted feature data. Then, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values. Finally, an output layer sums the weighted contribution values from the plurality of features and applies a mathematical function to the sum of the contribution values to determine a classification result];
determining, using a curve fitting engine (Para 0036, first order distance function) and based on the calculated influence values (Para 0036, contribution value), a curve function mapping (Para 0007, transfer function) the one or more features (Para 0007, feature value) to the influence value (Para 0007, contribution value) of the one or more features [Para 0007, a hidden layer calculates a contribution value for each feature value via a common transfer function and applies predetermined weights to each of the contribution values; Para 0035-36, A number of basis functions are available for use as transfer functions in the claimed classifier… A second type of function which can be used in the classifier is a first order distance function. In a first order distance function, the contribution value is calculated by taking the absolute value of the difference between the feature value];
creating an augmented input feature set (Para 0034, output value, for each of reference) based on the curve function (Para 0034, transfer function) [Para 0034, The value received at the intermediate node (e.g., 62B) is subjected to a transfer function to calculate an output to the output layer. This output value, for each of reference, will be referred to as a contribution value];
Li does not teach A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: generating feature values for features of the augmented input feature set, creating a destination model by training a model based on the augmented input feature set and generated features values, the influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to the set of features of the source model, and utilizing the destination model to make an inference based on input values for the augmented input feature set.
Yoo teaches,
A tangible machine-readable storage medium including instructions [Para 0127, The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.] that, when executed by a machine, cause the machine to perform operations comprising:
generating feature values (Para 0014, a second feature map) for features of the augmented input feature set (Para 0014, a first feature map) [Para 0009, A layer accepts as input a feature map provided by a previous layer as output… Alternatively, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations applied to the feature map before inputting to the accepting layer; Para 0014, wherein the pretrained model is trained to operate on data of a first domain… The augmenting includes adjusting an attention value of a channel in a first feature map being output from a layer in the pretrained model, wherein the adjusting causes a first feature matrix of the channel in the first feature map to have a greater weight relative to a second feature matrix of a different channel in the first feature map. The augmenting further includes combining, to form a combined feature map, a first feature matrix of the channel in the first feature map with a second feature map being output from a layer in the submodel; and inputting the combined feature map into a different layer in the submodel.];
creating a destination model (Para 0013, to form the augmented model configuration) by training a model based on the augmented input feature set (Para 0015, the pretrained model is trained to operate on data of a first domain) and generated features values (Para 0015, trains… the submodel using training data corresponding to a second domain) [Para 0009, the accepting layer can be configured such that the layer accepts the feature map output of the providing layer after some processing, such as bias changes, feature reduction, or other operations; Para 0013, The embodiment augments, to form the augmented model configuration, the pretrained model with the submodel. The augmenting includes combining, to form a combined feature map, a first feature map being output from a layer in the pretrained model with a second feature map being output from a layer in the submodel, and inputting the combined feature map into a different layer in the submodel; Para 0015, The embodiment trains… the submodel using training data corresponding to a second domain, wherein the pretrained model is trained to operate on data of a first domain; Para 0042, a pretrained ANN that its trained on some dataset, with a submodel—a different model that is trained on a target dataset… The augmented model can be understood as a combined model in which some layers of the pretrained model are coupled with some layers of the submodel via input/output feature maps.];
Yoo is analogous to the claimed invention as they both relate to transfer learning with neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li’s teachings to incorporate the teachings of Yoo and provide a secondary model performing calculations on an input feature set in order to improve robustness of a machine learning system by utilizing input data in a variety of operations.
Li-Yoo-Funaya teach the above limitations of claim 1 including the set of features (Li, para 0007), the destination model (Yoo, para 0013), the augmented input feature set (Li, para 0034), and the source model (Li, para 0013).
Li-Yoo do not teach influence values of each feature indicating a degree to which the feature affects the output of a model, features being additional features, function to add additional features to set, and utilizing model to make an inference based on input values.
Funaya teaches,
influence values (Para 0087, Feature Importance) of each feature indicating (Para 0087, explanatory variable) a degree to which the feature affects the output of a model (Para 0087, the degree of contribution… to the objective variable in a tree-based learning model) [Para 0087, This embodiment aims to obtain an optimal model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index that represents the degree of contribution of each explanatory variable to the objective variable in a tree-based learning model];
utilizing model to make an inference based on input values [Para 0029, we consider constructing a Hakodate housing price prediction model using the training data of Hakodate housing data, which serves as input data].
Funaya is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide influence values in order to enhance interpretability of machine learning models. Additionally, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li and Yoo’s teachings to incorporate the teachings of Funaya and provide a predictive model in order to improve the output of analytical models.
Li-Yoo-Funaya does not teach features being additional features and function to add additional features to set.
Luo teaches,
features being additional features and function to add additional features to set [Para 0016, According to the prediction performance evaluation function, the feature set S0 is evaluated by the prediction model to obtain the prediction performance after adding feature xi. If the prediction performance is better than before adding feature xi, feature xi is retained].
Luo is analogous to the claimed invention as they both relate to predictive models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, and Funaya’s teachings to incorporate the teachings of Luo and provide adding features in order to [Luo, para 0016] improve prediction performance.
Claim(s) 2-4, 9, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Datta et al. (Algorithmic Transparency via
Quantitative Input Influence:
Theory and Experiments with Learning Systems, published 2016), hereinafter Datta.
Regarding claim 2, Li-Yoo-Funaya-Luo teach all the limitations of claim 1 including the relative influence value.
Li-Yoo-Funaya-Luo do not teach wherein influence value is computed based on a quantitative input influence (QII) score computed based on a joint influence of a set of one or more features or a difference in outputs with or without the one or more features from the one or more features.
Datta teaches,
wherein influence value (Abstract, the degree of influence) is computed (Abstract, measures) based on a quantitative input influence (QII) (Abstract, QII) score computed based on a joint influence (Abstract, joint influence) of a set of one or more features (Abstract, set of inputs (e.g., age and income)) or a difference in outputs with or without the one or more features from the one or more features [Abstract, we introduce a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems… the QII measures also quantify the joint influence of a set of inputs (e.g., age and income) on outcomes (e.g. loan decisions)].
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide Quantitative Input Influence (QII) in order to bolster the calculation of a influence/contribution values by introducing a particular methodology that [Datta, Abstract] has been shown to provide better explanations than standard associative measures. This, in turn, leads to an improvement in machine learning predictions.
Regarding claim 3, Li-Yoo-Funaya-Luo-Datta teach all the limitations of claim 2.
Datta further teaches,
wherein the joint influence corresponds to a correlation [Sect 1, pg. 599, col 1, para 6, we seek measures of joint influence of a set of inputs (e.g., age and income) on a system’s decision; Sect 1, pg. 599, col 2, para 3, These measures (called Unary QII) model the difference in the quantity of interest when the system operates over two related input distributions—the real distribution and a hypothetical (or counterfactual) distribution that is constructed from the real distribution in a specific way to account for correlations among inputs.].
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide Quantitative Input Influence (QII) in order to bolster the calculation of a influence/contribution values by introducing a particular methodology that [Datta, Abstract] has been shown to provide better explanations than standard associative measures. This, in turn, leads to an improvement in machine learning predictions.
Regarding claim 4, Li-Yoo-Funaya-Luo-Datta teach all the limitations of claim 2.
Datta further teaches,
storing a table data structure where columns represent the one or more features and rows represent the QII score [Pg. 609, TABLE II: Comparison of QII with associative measures. For 4 different classifiers, we compute metrics such as Mutual Information (MI), Jaccard Index (JI), Pearson Correlation (corr), Group Disparity (disp) and Average QII between Gender and the outcome of the learned classifier. Each metric is computed in two situations: (A) when Gender is provided as an input to the classifier, and (B) when Gender is not provided as an input to the classifier].
PNG
media_image1.png
288
897
media_image1.png
Greyscale
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide storing a table data structure where columns represent the one or more features and rows represent the QII score in order to provide an intuitive and simple presentation of data to analyze trends.
Regarding claim 9, Li-Yoo-Funaya-Luo teach all the limitations of claims 1 including the curve fitting engine and the curve model.
Li-Yoo-Funaya-Luo do not teach, mathematically combines the one or more features into a single feature based on the one or more features having a correlation with one another, exceeding a threshold value, or being sourced from a similar source.
Datta teaches,
mathematically combines (Sect VII (E), computing) the one or more features (Sect VII (E), samples of the dataset) into a single feature (Sect VII (E), QII measures by computing sums) based on the one or more features having a correlation with one another, exceeding a threshold value, or being sourced from a similar source (Sect VII (E), the dataset). [Sect VII (B), para 2, The figure on the left shows the influence of features on group disparity by Gender in the adult dataset; the figure on the right shows the influence of group disparity by Race in the arrests dataset; Sect VII (E), We report runtimes of our prototype for generating transparency reports on the adult dataset. Recall from Section VI that we approximate QII measures by computing sums over samples of the dataset.]
Datta is analogous to the claimed invention as they both relate to using a particular methodology for computing influence values. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Datta and provide combining multiple features into a single feature [Datta, Abstract] as summation efficiently measures influence in a simplified correlation of multiple inputs.
Claims 17-19 are system claims that recite identical limitations to method claims 2-4. Therefore, claims 17-19 are rejected using the same rationale as claims 2-4.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of
Sattarzadeh et al. (INTEGRATED GRAD-CAM: SENSITIVITY-AWARE VISUAL EXPLANATION OF DEEP
CONVOLUTIONAL NETWORKS VIA INTEGRATED GRADIENT-BASED SCORING, published May 13, 2021), hereinafter Sattarzadeh, and Bach et al. (JP 2020123329 A, see attached translation), hereinafter Bach.
Regarding claim 5, Li-Yoo-Funaya-Luo teach all the limitations of claim 1 including the source model.
Sattarzadeh teaches,
wherein the influence value is computed based on a sum of gradients (Fig. 2, addition result of ReLU) of an interpolation (Fig. 2, annotated box) between a baseline value (Fig. 2, Baseline) and the output value of the source model (Fig. 2, Input) with respect to the one or more features [Figure 2, illustrated below, Schematic of the proposed method considering that the baseline image is set to black and the path connecting the baseline and the input is set as a straight line.]
PNG
media_image2.png
787
1629
media_image2.png
Greyscale
Sattarzadeh is analogous to the claimed invention as they both relate to interpretable ML. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Sattarzadeh and provide calculating influence values based on a sum of gradients as accumulating gradients can provide a more stable update direction for model update parameters.
Li-Yoo-Funaya-Luo-Sattarzadeh do not teach wherein the source model comprises an artificial neural network (ANN).
Bach teaches,
wherein model comprises an artificial neural network (ANN) [Para 0012, It is therefore an object of the present invention to provide a concept for assigning relevance scores to a set of items to which an artificial neural network is applied, which concept is applicable to a broader range of artificial neural networks and/or reduces computational effort.].
Bach is analogous to the claimed invention as they both relate to interpretable ML. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, and Sattarzadeh’s teachings to incorporate the teachings of Bach and provide an ANN in order to analyze complex patterns, especially in non-linear relationships.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, Luo, Sattarzadeh and Bach, and in further view of Wolpert et al. (CIRCUITS FOR A VLSI-BASED STANDALONE BACKPROPAGATION NEURAL NETWORK, published 1992), hereinafter Wolpert.
Regarding claim 6, Li-Yoo-Funaya-Luo-Sattarzadeh-Bach teach all the limitations of claim 5 including the sum of gradients.
Li-Yoo-Funaya-Luo-Sattarzadeh-Bach do no teach wherein gradients are computed in parallel using multithreaded processing circuitry.
Wolpert teaches,
wherein gradients (DISCUSSION, back-propagation) are computed in parallel (Its processing is… parallel) using multithreaded processing circuitry (DISCUSSION, connectivity matrix) [DISCUSSION, The back-propagation neural network, when implemented in dedicated VLSI hardware, offers both aesthetic and tangible advantages over software-based methods. Aesthetically, the process of evaluation of input and weighting coefficients is performed in a parallel and simultaneous manner… Practically, these operations may occur at a higher rate than is possible in software. The connectivity matrix has a dedicated multiplier and summer for each synapse and output node respectively. Its processing is not only rapid, but truly parallel and distributed. The back-propagated error summation process is also achieved simultaneously and rapidly].
Wolpert is analogous to the claimed invention as they both relate to calculating gradients via neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, Sattarzadeh, and Bach’s teachings to incorporate the teachings of Wolpert and provide gradients computed in parallel using multithreaded processing circuitry [Wolpert, DISCUSSION] these operations occur at a higher rate than is possible in software.
Claim(s) 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Vemuri et al. (US 11704535 B1), hereinafter Vemuri.
Regarding claim 10, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the generated feature values for features of the augmented input feature set.
Li-Yoo-Funaya-Luo do not teach, wherein feature values are stored in a memory of the computing machine.
Vemuri teaches,
wherein feature values (Col 4, lines 43-45, input feature maps) are stored (Col 4, lines 43-45, stores) in a memory (Col 4, lines 43-45, memory 106) of the computing machine [Fig. 1, host 102] [Col 4, lines 43-45, In one embodiment, the memory 106 stores… input feature maps].
Vemuri is analogous to the claimed invention as they both relate to neural network processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Vemuri and provide feature values stored in memory for further retraining of a model.
Regarding claim 11, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the generated feature values for features of the augmented input feature set and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein feature values are stored, in a data repository external to the computing machine, in a format that is accessible to model for training.
Vemuri teaches,
wherein feature values (Col 16, lines 2-8, feature-maps) are stored (Col 16, lines 2-8, stores), in a data repository external to the computing machine (Col 16, lines 2-8, external memory), in a format that is accessible to model (Col 4, lines 61-67, neural network) for training (Col 4, lines 61-67, training) [Col 4, lines 61-67 & col 5, lines 1-5, In FIG. 1, the allocated blocks 142 of the memory 140 comprise data of the neural network, including data derived when training the neural network. As with the memory 106 of the host computer 102, the detailed circuitry within the memory 140 is described below, but can include any type of volatile or nonvolatile memory. In one embodiment, the memory 140 includes an array of memory elements.
The DPE array 130 of the reconfigurable IC 120 has any number of DPEs (also referred to as kernel processors), and these DPEs of the DPE array 130 perform operations on the input data (e.g., data points of input feature maps) to generate output data (e.g., data points of output feature maps); Col 16, lines 2-8, The architecture of the DPEs dictates the data organization in the various buffers of the reconfigurable IC. In the exemplary embodiment, external memory stores the feature-maps].
Vemuri is analogous to the claimed invention as they both relate to neural network processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Vemuri and provide feature values stored in an external memory in order to bypass capacity limitations.
Claim(s) 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Montone et al. (US 20230168667 A1), hereinafter Montone.
Regarding claim 12, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the source model and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein model comprises a classifier gradient boosting machine (GBM) model, and wherein model comprises a linear model.
Montone teaches,
wherein model comprises a classifier gradient boosting machine (GBM) model, and wherein model comprises a linear model [Other machine learning algorithms that be leveraged include… Generalized Linear Models, Extreme Gradient Boosting].
Montone is analogous to the claimed invention as they both relate to utilizing a plurality of ML models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Montone and provide utilizing a GBM and a linear model in order to incorporate models that enhance versatility across tasks and flexibility of different data types.
Regarding claim 13, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the source model and the destination model.
Li-Yoo-Funaya-Luo do not teach wherein model comprises a classifier recurrent neural network (RNN) model, and wherein comprises a classifier gradient boosting machine (GBM) model.
Montone teaches,
wherein model comprises a classifier recurrent neural network (RNN) model, and wherein comprises a classifier gradient boosting machine (GBM) model [Other machine learning algorithms that be leveraged include… recurrent neural network (RNN) modeling… Generalized Linear Models].
Montone is analogous to the claimed invention as they both relate to utilizing a plurality of ML models. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Montone and provide a GBM for its high predictive accuracy and an RNN for its efficiency in parameter sharing.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, and Luo, and in further view of Catalin (WO 2019013711 A1), hereinafter Catalin.
Regarding claim 14, Li-Yoo-Funaya-Luo teach the limitations of claim 1 including the destination model and the source model.
Li-Yoo-Funaya-Luo do not teach wherein destination model utilize fewer computing resources than source model, wherein the computing resources comprise processing circuitry resources or memory resources.
Catalin teaches,
wherein destination model (Para 0026, devices 20) utilize fewer computing resources (Para 0158, reduce the overall memory footprint) than source model (Para 0026, server 14), wherein the computing resources comprise processing circuitry resources or memory resources (Para 0026, memory footprint) [Para 0026, The backend server 14 may manage training of feature recognition software such as deep learning systems comprising and/or for incorporation into POS software applications… The backend server 14 may also manage distribution of the POS software applications and/or feature recognition software for download by and/or peer-to-peer transfer between a plurality of low-power mobile electronic devices 20; Para 0158, In addition, model compression may be performed in the training pipeline for improved performance of the CNN 100 on low-power mobile device networks...While quantization provides significant (and perhaps, the primary) performance boost, other compression steps for reducing the size of CNN 100 may contribute to viability for use in POS application 24. A smaller size may lower the electronic burden and/or power consumption associated with execution of the object recognition module of the CNN 100 at mobile electronic devices 20, may reduce the overall memory footprint of the POS application 24, and/or may reduce processor cache misses.].
Catalin is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, and Luo’s teachings to incorporate the teachings of Catalin and provide the destination model utilizing fewer computing resources than source model [Catalin, Para 0158] for improved performance of a neural network on low-power mobile device networks.
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Yoo, Funaya, Luo, and Montone, and in further view of Catalin.
Regarding claim 15, Li-Yoo-Funaya-Luo-Montone teach the limitations of claims 13 including the destination model, the modified destination model, and the source model (see claim 1).
Li-Yoo-Funaya-Luo-Montone do not teach wherein source model utilizes a server farm having a first amount of memory, and wherein destination model utilize a client computing device having a second amount of memory, the second amount of memory being less than the first amount of memory.
Catalin teaches,
wherein source model (Para 0026, server 14) utilizes a server farm (Para 214, the processing elements may be located in a single location (e.g…. a server farm) having a first amount of memory (Para 0026, memory footprint), and wherein destination model (Para 0026, devices 20) utilize a client computing device having a second amount of memory (Para 0026, memory footprint), the second amount of memory being less than the first amount of memory (Para 0026, reduce the overall memory footprint) [Para 0026, The backend server 14 may manage training of feature recognition software such as deep learning systems comprising and/or for incorporation into POS software applications… The backend server 14 may also manage distribution of the POS software applications and/or feature recognition software for download by and/or peer-to-peer transfer between a plurality of low-power mobile electronic devices 20; Para 0158, In addition, model compression may be performed in the training pipeline for improved performance of the CNN 100 on low-power mobile device networks...While quantization provides significant (and perhaps, the primary) performance boost, other compression steps for reducing the size of CNN 100 may contribute to viability for use in POS application 24. A smaller size may lower the electronic burden and/or power consumption associated with execution of the object recognition module of the CNN 100 at mobile electronic devices 20, may reduce the overall memory footprint of the POS application 24, and/or may reduce processor cache misses; Para 214, In some example embodiments, the processing elements may be located in a single location (e.g., within a home environment, an office environment or as a server farm].
Catalin is analogous to the claimed invention as they both relate to transfer learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, Yoo, Funaya, Luo, and Montone’s teachings to incorporate the teachings of Catalin and provide the destination model utilizing fewer computing resources than source model [Catalin, Para 0158] for improved performance of a neural network on low-power mobile device networks.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED RAYHAN AHMED whose telephone number is (571)270-0286. The examiner can normally be reached Mon-Fri ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED RAYHAN AHMED/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights