Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
Claim Rejections – 35 U.S.C. 101
Applicant’s amendments have been fully considered but they are not persuasive.
Applicant argues (pg. 7-9) that the steps of Claim 1 are not a mental process because they “require the use of a computing machine to process large-scale, high-dimensional datasets and to perform complex calculations (e.g., QII computation, normalization, and model adjustment) that are not practical or feasible for a human to perform manually, especially at the scale and speed required for modern AI systems. The claim is directed to a specific, technical solution to the problem of unreliable AI model predictions due to outlier or over-influential features, and not to an abstract idea.”
Examiner respectfully disagrees. Claim 1 recites that the training dataset is a “two-dimensional vector with rows representing datapoints and columns representing features”. This is not a data structure with hundreds or thousands of dimensions, each with numerous data points that make it impossible for a human to feasibly perform calculations at an adequate speed. Furthermore, calculations such as normalization are not necessarily complex, as it involves dividing by a scalar to resize a vector. Therefore, the steps of claim 1 are indeed a mental process. See rejection below for details.
Applicant argues (pg. 9-11) that the steps of Claim 1 are integrated into a practical application. The Applicant states: “The specification describes, in detail, how these steps are performed by a computing machine, and how they result in improved AI model performance and reliability. For example, the specification explains that "outliers and high sensitivity are identified based on feature influence," and that "mitigation can be based on feature influence," including "replacing extreme values with minimum and maximum percentiles" and "restricting the influence of any feature to a selected range." See, e.g., [0082]-[0083], [0107]-[0112]. The claim is thus directed to a practical application of feature influence analysis and adjustment in the context of machine learning, resulting in a tangible improvement to the operation of AI systems. This is not a mere implementation of an abstract idea on a generic computer, but a specific, technical solution to a technical problem in the field of AI.”
Examiner respectfully disagrees. It is not clear how the examples given by the Applicant are examples directly improving the field of AI. Identifying outliers and sensitive information, mitigating based on feature influence, replacing extreme values, and restricting influence of a feature to a range are all steps that, but for a computer, would be mental process. It is not clear how this is a specific improvement to a computer or to the field of artificial intelligence in particular.
The foregoing applies to all independent claims and their dependent claims.
Claim Rejections – 35 U.S.C. 103
Applicant’s prior art arguments have been fully considered and they are persuasive.
Applicant argues (pgs. 11-12) that Wang does not teach the limitation in claim 1 “computing, for each feature, a QII value measuring a degree of influence that the feature exerts on the output value, the QII comprising any combination of one or more of unary QII or marginal QII”.
Applicant argues that Wang does not describe any computation of QII values, nor mention unary/marginal QII, nor any quantitative measurement of feature influence as defined in the claim. Applicant asserts that instead, Wang discusses a qualitative selection process based on classification ability within decision tree nodes. Applicant also asserts that QII measures how much the prediction actually changes when a feature is randomized, implying that Wang does not teach this.
Examiner respectfully disagrees. First, to note, Applicant misquoted the Examiner by stating that Wang, Page 1, Column 2, Paragraph 1 was quoted for this limitation. Instead, the following was quoted: (Wang [Page 3, Column 1, Paragraph 2]: “Next, the normalized area coefficient is regarded as the weight for every feature and an appropriate number of features are selected on the basis of feature weight.” Wang teaches that for each of the feature values, the normalized area coefficient is regarded as the weight for every feature, which measures how influential that feature is on the output.)”
To illustrate the teaching of this limitation, Examiner points to Figure 1 in Wang, page 4. Here, the effective ranges of the feature/class are depicted. The definition of effective range is given by Equation 5, on page 3 in Wang. Since this equation is based on probabilities, the effect of the input of the feature is not like how it works in linear regression, and is analogous to QII, as it works with random probabilities.
Applicant argues (pgs. 13-14) that the cited references do not teach the newly amended limitations that further clarify that the unary QII is based on the difference between outputs from a real distribution of feature values and a hypothetical distribution constructed from the real distribution to account for correlations among input values, as well as that the marginal QII is based on comparing the training dataset with and without the feature value.
Examiner agrees. Accordingly, a new reference, Datta (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) has been added to the rejection, as further detailed below.
The foregoing applies to all independent claims and their dependent claims.
Applicant argues (pg. 14-16) that the cited references do not teach the limitations regarding that upon determining that when the QII value for a given feature in the input vector is not within the predefined range, adjusting the training dataset or the machine learning engine based on the QII value for the given feature in the input vector being outside the predefined range.
Examiner agrees. Accordingly, a new reference, Datta (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) has been added to the rejection, as further detailed below.
The foregoing applies to all independent claims and their dependent claims.
Applicant argues (pg. 16-17) that the cited references do not teach the limitations regarding adjusting the given feature in the training set to place the QII value into the predefined range.
Examiner agrees. Accordingly, a new reference, Datta (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) has been added to the rejection, as further detailed below.
The foregoing applies to all independent claims and their dependent claims.
Applicant argues (pg. 17-18) that the cited references do not teach the limitations regarding reducing an influence on a predicted output value of the given feature in the input vector when the QII value is not within the predefined range.
Examiner agrees. Accordingly, a new reference, Datta (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) has been added to the rejection, as further detailed below.
The foregoing applies to all independent claims and their dependent claims.
Applicant argues (pg. 18-19) that the cited references do not teach the limitations regarding computing a normalized QII value, and if this value exceeds a threshold, readjusting the training dataset or the machine learning engine to reduce the normalized QII value.
Examiner agrees. Accordingly, a new reference, Datta (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) has been added to the rejection, as further detailed below.
The foregoing applies to all independent claims and their dependent claims.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-13 are method claims. Claims 14-20 are machine/system/product claims. Therefore, claims 1-20 are directed to either a process, machine, manufacture or composition of matter.
With respect to claim 1:
Step 2A – Prong 1:
…
… a training dataset, the training dataset comprising a plurality of datapoints, each datapoint having an input vector of feature values and an output value, … (mental process – a person can recognize that the training dataset comprises a plurality of datapoints, each datapoint having an input vector of feature values and an output value.)
…
…
computing, for each feature value, a QII (quantitative input influence) value measuring a degree of influence that the feature exerts on the output value, the QII comprising any combination of one or more of unary QII or marginal QII, the unary QII based on a difference between outputs from a real distribution of feature values and a hypothetical distribution constructed from the real distribution to account for correlations among input values, the marginal QII based on comparing the training dataset with and without the feature value; (mental process – a person can manually compute, for each feature value, a QII value measuring a degree of influence that the feature exerts on the output value with the assistance of a pen/paper.)
for each datapoint from at least a subset of the plurality of datapoints: determining whether the QII value for each feature value in the input vector is within a predefined range, wherein the predefined range comprises an upper bound and a lower bound, in the column corresponding to the feature; (mental process – a person can manually determine whether the QII value for each feature value in the input vector is within a predefined range with the assistance of a pen/paper.)
and upon determining that the QII value for a given feature in the input vector is not within the predefined range, adjusting the training dataset or the machine learning engine based on the QII value for the given feature in the input vector being outside the predefined range; (mental process – a person can manually adjust the training dataset or the machine learning engine based on the QII value for the given feature value in the input vector being not within the predefined range with the assistance of a pen/paper.)
…
making … and inference based on a received input value. (mental process – a person can manually make an inference based on a received input value with the assistance of a pen/paper.)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
A method implemented at a computing machine comprising processing circuitry and memory, the method comprising: accessing, at the processing circuitry of the computing machine, … (mere instructions to apply the exception using a generic computer component – computer applies exception.)
…
… wherein the training dataset is for training a machine learning engine to predict the output value based on the input vector of feature values, wherein each feature value corresponds to a feature; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to predict the output value based on the input vector of feature values.);
storing, in the memory, the training dataset as a two-dimensional vector with rows representing datapoints and columns representing features; (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
…
…
…
training the machine learning engine to predict the output value based on the adjusted training dataset. (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to predict the output value based on the adjusted training dataset.);
… by the trained machine learning engine … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to make an inference based on a received input value.);
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
A method implemented at a computing machine comprising processing circuitry and memory, the method comprising: accessing, at the processing circuitry of the computing machine, … (mere instructions to apply the exception using a generic computer component – computer applies exception.)
…
… wherein the training dataset is for training a machine learning engine to predict the output value based on the input vector of feature values, wherein each feature value corresponds to a feature; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to predict the output value based on the input vector of feature values.);
storing, in the memory, the training dataset as a two-dimensional vector with rows representing datapoints and columns representing features; (MPEP 2106.05(d)(II) indicate that merely “Storing and retrieving information in memory” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the training dataset as a two-dimensional vector with rows representing datapoints and columns representing features is merely stored in memory). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
…
…
…
training the machine learning engine to predict the output value based on the adjusted training dataset. (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to predict the output value based on the adjusted training dataset.);
… by the trained machine learning engine … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to make an inference based on a received input value.);
With respect to claim 2:
Step 2A – Prong 1:
The method of claim 1, wherein adjusting the training dataset or the machine learning engine comprises: adjusting the given feature value in the training set to place the QII value into the predefined range. (mental process – a person can manually adjust the given feature value in the input vector to place the QII value into the predefined range with the assistance of a pen/paper.)
With respect to claim 3:
Step 2A – Prong 1:
The method of claim 1, wherein adjusting the training dataset or the machine learning engine comprises: reducing, in the machine learning engine, an influence on a predicted output value of the given feature in the input vector when the QII value is not within the predefined range. (mental process – a person can manually reduce, in the machine learning engine, an influence, on a predicted output value, of the given feature value in the input vector when the QII value is not within the predefined range with the assistance of a pen/paper.)
With respect to claim 4:
Step 2A – Prong 1:
The method of claim 1, further comprising: computing, for a plurality of feature values in the input vector, including the given feature, a normalized QII value; (mental process – a person can manually compute, for a plurality of feature values in the input vector, including the given feature value, a normalized QII value with the assistance of a pen/paper.)
and if the normalized QII value exceeds a threshold, readjusting the training dataset or the machine learning engine to reduce the normalized QII value. (mental process – a person can manually readjust the training dataset or the machine learning engine to reduce the normalized QII value with the assistance of a pen/paper.)
With respect to claim 5:
Step 2A – Prong 1:
The method of claim 4, wherein the normalized QII value is computed as a square root of a sum of the squares of the QII values for each of the plurality of feature values in the input vector. (mental process – a person can recognize that the normalized QII value is computed as a square root of a sum of the squares of the QII values for each of the plurality of feature values in the input vector.)
With respect to claim 7:
Step 2A – Prong 1:
The method of claim 1, (mental process from claim 1)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
wherein training the machine learning engine comprises supervised learning, unsupervised learning, or reinforcement learning. (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine using supervised learning, unsupervised learning or reinforcement learning.);
With respect to claim 9:
Step 2A – Prong 1:
The method of claim 1, wherein the unary QII takes into account a joint influence of a plurality of input values. (mental process – a person can recognize that the unary QII takes into account a joint influence of a plurality of input values.)
With respect to claim 11:
Step 2A – Prong 1:
The method of claim 1, further comprising: detecting an outlier datapoint having an outlier input vector of feature values relative to the training dataset; (mental process – a person can manually detect an outlier datapoint having an outlier input vector of feature values relative to the training dataset with the assistance of a pen/paper.)
and removing the outlier datapoint from the training dataset. (mental process – a person can manually remove the outlier datapoint from the training dataset with the assistance of a pen/paper.)
With respect to claim 12:
Step 2A – Prong 1:
The method of claim 1, wherein the predefined range is between a first percentile of QII values in the training dataset and a second percentile of QII values in the training dataset. (mental process – a person can recognize that the predefined range is between a first percentile of QII values in the training dataset and a second percentile of QII values in the training dataset.)
Claim 13 is substantially similar to claim 1, but has the following additional elements:
With respect to claim 13:
Step 2A – Prong 1:
computing, for a plurality of feature values in the input vector, a normalized QII value; (mental process – a person can manually compute, for a plurality of feature values in the input vector, a normalized QII value with the assistance of a pen/paper.)
and if the normalized QII value exceeds a threshold: adjusting the training dataset or the machine learning engine to reduce the normalized QII value; (mental process – a person can manually adjust the training dataset or the machine learning engine to reduce the normalized QII value with the assistance of a pen/paper.)
Claim 14 is substantially similar to claim 1, but has the following additional elements:
With respect to claim 14:
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: (mere instructions to apply the exception using a generic computer component – machine-readable storage medium applies exception.)
Claims 15, 16, 17 are rejected on the same grounds under 35 U.S.C. 101 as claims 2, 4, 5, as they are substantially similar, respectively. Mutatis mutandis.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, 9, 11-16 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (“An Improved Feature Selection Based on Effective Range for Classification”) hereinafter known as Wang in view of Amzal et al. (US 20220058169 A1) hereinafter known as Amzal in view of Datta et al. (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) hereinafter known as Datta.
Regarding independent claim 1, Wang teaches:
A method implemented at a computing machine comprising processing circuitry and memory, the method comprising: accessing, at the processing circuitry of the computing machine, a training dataset, the training dataset comprising a plurality of datapoints, each datapoint having an input vector of feature values and an output value, wherein the training dataset is for training a machine learning engine to predict the output value based on the input vector of feature values, wherein each feature value corresponds to a feature; (Wang [Page 1, Column 1, Paragraph 1]: “In real-world applications, the huge dataset usually has a large number of features which contains much irrelevant or redundant information” Wang teaches that the dataset has many features, that may be represented as a vector with the features being represented in the different dimensions. Wang [Page 1, Column 2, Paragraph 1]: In these algorithms, the features with the strongest ability of classification are selected in the nodes of the tree, and then the selected features are utilized to conduct a subspace to perform the learning tasks. Wang teaches that the dataset with the features is used for classification, which is the prediction of the output class based on the input features.)
…
…
for each datapoint from at least a subset of the plurality of datapoints: determining whether the QII value for each feature in the input vector is within a predefined range, wherein the predefined range comprises an upper bound and a lower bound, in the column corresponding to the feature; (Wang [Page 2, Column 2, Paragraph 5]: “Effective range (𝑅𝑖𝑗) of 𝑗th class 𝑌𝑗 for 𝑖th feature 𝐹𝑖 is defined by … here 𝑟 − 𝑖𝑗 and 𝑟 + 𝑖𝑗 are the lower and upper bounds of the effective range, respectively.” Wang teaches that the effective range for a feature is defined by an upper bound and a lower bound. This effective range is based on the features, which is equivalent to the columns in the data structure that stores the dataset.)
…
and training the machine learning engine to predict the output value based on the adjusted training dataset; (Wang [Page 5, Column 1, Paragraph 1]: “Therefore, we can select the features according to their weights and choose features with larger weights to form the selected feature subset.” Wang teaches that once the algorithm chooses the features that are most influential, then they are given the larger weights. Then this selection is transmitted to form the selected feature subset.)
making, by the trained machine learning engine, and inference based on a received input value. (Wang [Page 5, Column 2, Algorithm 1]: Wang teaches an algorithm that selects for the best k features based on an input data matrix. This is an inference on what the k best features are.)
Wang does not explicitly teach:
storing, in the memory, the training dataset as a two-dimensional vector with rows representing datapoints and columns representing features;
However, Amzal teaches:
storing, in the memory, the training dataset as a two-dimensional vector with rows representing datapoints and columns representing features; (Amzal [¶ 0023]: “a feature may be a column with one or more rows (e.g., a plurality of rows) representing a plurality of values of the measure. The query engine 208 may store the results in a training data set 210 which may be a table, a file, a document, etc., within a data storage device.” Amzal teaches that the training data that comprises of features is a column with many rows. This shows that each feature corresponds to the features and the rows are the various datapoints.)
Wang and Amzal are in the same field of endeavor as the present invention, as the
references are directed to the analysis of features in machine learning and the storing of features in memory, respectively. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine determining the features that influence the output the most as taught in Wang with storing the dataset as a data structure with features as columns as taught in Amzal. Amzal provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Wang to include teachings of Amzal because the combination would allow for the features in the various data points in the dataset to be organized into columns as they are being analyzed. This has the potential benefit of speeding up the determination of the influence of features, as a particular column can be analyzed across different data points to zone in on a particular feature.
Wang and Amzal do not explicitly teach:
computing, for each feature, a QII (quantitative input influence) value measuring a degree of influence that the feature exerts on the output value, the QII comprising any combination of one or more unary QII or marginal QII, the unary QII based on a difference between outputs from a real distribution of feature values and a hypothetical distribution constructed from the real distribution to account for correlations among input values, the marginal QII based on comparing the training dataset with and without the feature value;
…
and upon determining that the QII value for a given feature in the input vector is not within the predefined range: adjusting the training dataset or the machine learning engine based on the QII value for the given feature in the input vector being outside the predefined range;
However, Datta teaches:
computing, for each feature, a QII (quantitative input influence) value measuring a degree of influence that the feature exerts on the output value, the QII comprising any combination of one or more unary QII or marginal QII, the unary QII based on a difference between outputs from a real distribution of feature values and a hypothetical distribution constructed from the real distribution to account for correlations among input values, the marginal QII based on comparing the training dataset with and without the feature value; (Datta [Page 2, Column 2, Paragraph 3]: “These measures (called Unary QII) model the difference in the quantity of interest when the system operates over two related input distributions—the real distribution and a hypothetical (or counterfactual) distribution that is constructed from the real distribution in a specific way to account for correlations among inputs.” Datta teaches that the unary QII is based on difference of outputs between a real and hypothetical random distribution. Datta [Page 2, Column 2, Paragraph 4]: “Marginal QII measures that model the difference on the quantity of interest as we consider sets with and without the specific input whose marginal influence we want to measure. … For example, we could fix a set of inputs and ask about the marginal influence of any given input in that set on the quantity of interest.” Datta teaches that the marginal QII is based on the influence on a given input/feature value.)
…
and upon determining that the QII value for a given feature in the input vector is not within the predefined range: adjusting the training dataset or the machine learning engine based on the QII value for the given feature in the input vector being outside the predefined range; (Datta [Page 10, Column 1, Paragraph 3]: “Finally, we observe that a change in a single element x’ of D will cause a change of at most 1 / abs(D AND Y) if x’ in D AND Y, or of at most 1 / abs(D NOT Y) if x’ in D NOT Y.” Datta teaches that given a dataset D, an change in the element given the stipulation that the element follows a set characteristic. Since this set is effectively a range (anything not in the set is outside of the range), this shows that the element is changed with a measurable effect on the outcome.)
Datta is in the same field as the present invention, since it is directed to analyzing the influence of the input data / features on the output in machine learning using QII. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine determining the features that influence the output the most as taught in Wang as modified by Amzal with using unary and marginal QII to do so as taught in Datta. Datta provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Wang as modified by Amzal to include teachings of Datta because the combination would allow for the analysis of the most important elements / features to be done in both distribution wise (unary) and element wise (marginal). This has the potential benefit of determining which features/data is most influential so that key decisions can be made in choosing where to increase/decrease efforts.
Regarding dependent claim 2, Wang and Amzal teach:
The method of claim 1,
Datta teaches:
wherein adjusting the training dataset or the machine learning engine comprises: adjusting the given feature in the training set to place the QII value into the predefined range. (Datta [Page 10, Column 1, Paragraph 3]: “Finally, we observe that a change in a single element x’ of D will cause a change of at most 1 / abs(D AND Y) if x’ in D AND Y, or of at most 1 / abs(D NOT Y) if x’ in D NOT Y.” Datta teaches that given a dataset D, an change in the element given the stipulation that the element follows a set characteristic. Since this set is effectively a range (anything not in the set is outside of the range), this shows that the element is changed with a measurable effect on the outcome. Since the change to the variable is clamp, it follows that the feature is adjusted to place the QII value into a range that is clamped.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 3, Wang and Amzal teach:
The method of claim 1,
Datta teaches:
wherein adjusting the training dataset or the machine learning engine comprises: reducing, in the machine learning engine, an influence on a predicted output value of the given feature in the input vector when the QII value is not within the predefined range. (Datta [Page 10, Column 1, Paragraph 3]: “Finally, we observe that a change in a single element x’ of D will cause a change of at most 1 / abs(D AND Y) if x’ in D AND Y, or of at most 1 / abs(D NOT Y) if x’ in D NOT Y.” Datta teaches that given a dataset D, an change in the element given the stipulation that the element follows a set characteristic. Since this set is effectively a range (anything not in the set is outside of the range), this shows that the element is changed with a measurable effect on the outcome. Since the change to the variable is clamp, it follows that the feature is adjusted to place the QII value into a range that is clamped.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 4, Wang and Amzal teach:
The method of claim 1,
Wang teaches:
further comprising: computing, for a plurality of feature values in the input vector, including the given feature, a normalized QII value; (Datta [Page 10, Column 1, Paragraph 6]: “Shapley and Banzhaf indices are normalized sums of the differences of the set influence functions” Datta teaches that the QII values, which are the Shapley values, are normalized.)
and if the normalized QII value exceeds a threshold, readjusting the training dataset or the machine learning engine to reduce the normalized QII value. (Datta [Page 10, Column 1, Paragraph 6]: “Shapley and Banzhaf indices are normalized sums of the differences of the set influence functions” Datta teaches that the QII values, which are the Shapley values, are normalized. This shows that the following, which established above deals with limits/ranges, is also applicable to normalized QII values: “Datta [Page 10, Column 1, Paragraph 3]: “Finally, we observe that a change in a single element x’ of D will cause a change of at most 1 / abs(D AND Y) if x’ in D AND Y, or of at most 1 / abs(D NOT Y) if x’ in D NOT Y.”)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 7, Wang and Amzal teach:
The method of claim 1,
Wang teaches:
wherein training the machine learning engine comprises supervised learning, unsupervised learning, or reinforcement learning. (Wang [Page 1, Column 2, Paragraph 1]: In these algorithms, the features with the strongest ability of classification are selected in the nodes of the tree, and then the selected features are utilized to conduct a subspace to perform the learning tasks. Wang teaches that the dataset with the features is used for classification, which is the prediction of the output class based on the input features. This is an example of supervised learning, as the classes are predetermined and it is the task of the machine learning to categorize.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 9, Wang and Amzal teach:
The method of claim 1,
Wang teaches:
wherein the unary QII takes into account a joint influence of a plurality of input values. (Wang [Page 3, Column 1, Paragraph 2]: “Then, the overlapping area of the effective ranges is calculated according to (3), and the area coefficient is computed for each feature.” Wang teaches that the overlapping area of the effective ranges represent a joint influence, where the shared area can be the part of the range that two or more input values affect the output in.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 11, Wang and Amzal teach:
The method of claim 1,
Wang teaches:
further comprising: detecting an outlier datapoint having an outlier input vector of feature values relative to the training dataset; (Wang [Page 3, Column 1, Paragraph 5]: “In Figure 2(a), the number of samples belonging to the overlapping area is small but the number of samples belonging to the overlapping area in Figure 2(b) is relatively large. Thus, it is obvious that feature 1 is more. important than feature 2 since more samples can be correctly classified. In other words, the weight assigned to feature1 should be greater than that assigned to feature 2.” Wang teaches that based on the effective range, when it is determined that a particular feature is not in the range, which is an outlier, it is adjusted to be less weighted than a more important feature. This can be to be weighted to zero, and thus removed.)
and removing the outlier datapoint from the training dataset. (Wang [Page 3, Column 1, Paragraph 5]: “In Figure 2(a), the number of samples belonging to the overlapping area is small but the number of samples belonging to the overlapping area in Figure 2(b) is relatively large. Thus, it is obvious that feature 1 is more. important than feature 2 since more samples can be correctly classified. In other words, the weight assigned to feature1 should be greater than that assigned to feature 2.” Wang teaches that based on the effective range, when it is determined that a particular feature is not in the range, which is an outlier, it is adjusted to be less weighted than a more important feature. This can be to be weighted to zero, and thus removed.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 12, Wang and Amzal teach:
The method of claim 1,
Wang teaches:
wherein the predefined range is between a first percentile of QII values in the training dataset and a second percentile of QII values in the training dataset. (Wang [Page 2, Column 2, Paragraph 5]: “Effective range (𝑅𝑖𝑗) of 𝑗th class 𝑌𝑗 for 𝑖th feature 𝐹𝑖 is defined by … here 𝑟 − 𝑖𝑗 and 𝑟 + 𝑖𝑗 are the lower and upper bounds of the effective range, respectively.” Wang teaches that the effective range for a feature is defined by an upper bound and a lower bound. This effective range is based on the features, which is equivalent to the columns in the data structure that stores the dataset. Furthermore, the upper bound and the lower bound both correspond to some percentile values of the training dataset, as they are interpolated.)
The reasons to combine are substantially similar to those of claim 1.
Claim 13 is rejected on the same grounds under 35 U.S.C. 103 as claims 1 and 4 as they are substantially similar. Mutatis mutandis.
Claim 14 is substantially similar to claim 1, but has the following additional elements:
Regarding independent claim 14, Wang and Amzal teach:
A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: (Amzal [¶ 0042]: “The storage 540 may store software modules or other instructions which can be executed by the processor 520 to perform the method” Amzal teaches a storage that can store instructions that can be executed by the processor.)
The reasons to combine are substantially similar to those of claim 1.
Claims 15, 16 are rejected on the same grounds under 35 U.S.C. 103 as claims 2, 4 as they are substantially similar, respectively. Mutatis mutandis.
Claims 5, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Amzal in view of Patel et al. (“Euclidean distance based feature ranking and subset selection for bearing fault diagnosis”) hereinafter known as Patel.
Regarding dependent claim 5, Wang and Amzal teach:
The method of claim 4,
Wang and Amzal do not teach:
wherein the normalized QII value is computed as a square root of a sum of the squares of the QII values for each of the plurality of feature values in the input vector.
However, Patel teaches:
wherein the normalized QII value is computed as a square root of a sum of the squares of the QII values for each of the plurality of feature values in the input vector. (Patel [Page 3, Column 2, Equation 3]: Patel teaches the Euclidean distance used for feature selection, where the distance between the classes is calculated based on the square root of the sum of square of each of the distances between features of two different classes.)
Patel is in the same field as the present invention, since it is directed to feature selection using a metric involving the square root of a sum of the squares of values. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine determining the features that influence the output the most as taught in Wang as modified by Amzal with using a metric involving the square root of a sum of the squares of values as taught in Patel. Patel provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Wang as modified by Amzal to include teachings of Patel because the combination would allow for the calculation of the importance of a feature to factor in the Euclidean distance from the feature to other features. This has the potential benefit of the importance of a feature to take into account how similar it is to other features, so that the metric can be normalized.
Claim 17 is rejected on the same grounds under 35 U.S.C. 103 as claim 5 as they are substantially similar. Mutatis mutandis.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYU HYUNG HAN whose telephone number is (703) 756-5529. The examiner can normally be reached on MF 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kyu Hyung Han/
Examiner
Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123