DETAILED ACTION
This action is in response to the application filed on 01/29/2024. Claims 1-20 are pending in the application and have been examined.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3, 10 and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 3, 10 and 17,
Claims 3, 10 and 17 recite the limitation “wherein the feature attribution is determined based on LIME, LRP, DeepLIFT … Grad-CAM”. The acronyms regarding LIME, LRP, DeepLIFT and Grad-CAM are not clearly defined and thus fail to clearly define the scope of the invention.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1,
(Step 1): Claim 1 recites A system comprising: a processor configured to execute instructions; a non-transitory computer-readable medium containing instructions executable by the processor for:, thus a machine, one of the four statutory categories of patentable subject matter.
(Step 2A Prong 1): However, Claim 1 further recites generating a feature attribution with respect to an output of a computer model relative to input features for each data sample of a group of data samples which constitutes the evaluation of the computer model’s output relative to the inputted features for each data sample of a group of data samples to determine which input features are attributable to parts of the output, thus corresponding to a mental process which can be done mentally or by pen and paper
clustering the group of data samples into a plurality of subgroups based on the respective feature attribution of each data sample, which constitutes the evaluation of the groups of data samples and their respectively determined feature attributions to determine clustered subgroups, thus corresponding to a mental process which can be done mentally or by pen and paper;
generating a feature region description in feature space with respect to input features for a subgroup of the plurality of subgroups; which constitutes the evaluation of the input features in the clustered subgroups to determine a description, thus corresponding to a mental process which can be done mentally or by pen and paper
Thus, Claim 1 recites an abstract idea.
(Step 2A Prong 2): The claim does not recite any additional elements which integrate the abstract idea into a practical application because the additional elements consist of:
a processor configured to execute instructions … a non-transitory computer readable medium containing instructions executable by the processor …, which is implementing an abstract idea on generic computing components (MPEP 2106.05(f))
and thus, the claim is directed to the abstract idea of determining feature attribution relationships between input features and the computer model output in order to cluster data samples and determine feature region descriptions.
(Step 2B): The additional elements, taken alone or in combination, cannot provide significantly more than the abstract idea itself because element a) (via MPEP 2106.05(f), “apply it on a computer”) cannot provide an inventive concept. Thus, Claim 1 is subject-matter ineligible.
Regarding Dependent Claims 2-7,
Claims 2-5 merely recite the particular technological environment or field of use in which the abstract idea is to be performed and thus (via MPEP 2106.05(h)) cannot integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, Claims 2-5 are subject-matter ineligible.
Claim 6 recites additional steps of the abstract idea (mental processes) but merely recites insignificant extra-solution activity of data outputting (MPEP 2106.05(g))) and thus (via MPEP 2106.05(g)) cannot integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, Claim 6 is subject-matter ineligible.
Claim 7 merely recites insignificant extra-solution activity of data outputting (MPEP 2106.05(g))) and thus (via MPEP 2106.05(g)) cannot integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, Claim 7 is subject-matter ineligible.
Claims 8-14 recite the exact method executed by the system of Claims 1-7 respectively. As performance of an abstract idea on generic computing components cannot integrate an abstract idea into a practical application nor provide significantly more than the abstract idea itself (see MPEP 2106.05(f)), Claims 8-14 is rejected for reasons set forth in the rejection of Claim 1-7 respectively.
Claims 15-20 recite a non-transitory computer readable medium comprising instructions to execute the exact method performed by the system of Claims 1-6 respectively. As performance of an abstract idea on generic computing components cannot integrate an abstract idea into a practical application nor provide significantly more than the abstract idea itself (see MPEP 2106.05(f)), Claims 15-20 are rejected for reasons set forth in the rejection of Claim 1-6 respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable by Cheng et al. (US 20220405623 A1, hereinafter “Cheng”) in view of Leung et al. (US 12436967 B2, hereinafter “Leung”).
Regarding Claim 1,
Cheng discloses A system comprising: a processor configured to execute instructions; a non-transitory computer-readable medium containing instructions executable by the processor for: generating a feature attribution with respect to an output of a computer model relative to input features for each data sample of a group of data samples; (Cheng [Abstract]; “The disclosure is directed to a query-driven machine learning platform for generating feature attributions and other data for interpreting the relationship between inputs and outputs of a machine learning model.”)
generating a feature region description in feature space with respect to input features …; (Cheng [0044]; “For decision trees, in some examples the explanation engine 130 can generate feature attributions based on measures for how each feature contributed to the construction of boosted decision trees within the model. The more a feature is used to make key decisions in the tree, the higher the explanation engine 130 can rate the importance of that feature. The explanation engine 130 can compute the feature attribution explicitly for each feature in a dataset, and output those attributions ordered according to value, e.g., highest to lowest. The feature attribution for a single decision tree can be calculated by the amount that each feature split point improves the performance measure of the decision tree, weighted by the number of observations the node is responsible for” wherein the generated feature importance rankings determined through the explanation engine thus reads on a feature region description in feature space (feature importance rankings being a description of the features within the region) with respect to input features)
Cheng doesn’t disclose but Leung discloses clustering the group of data samples into a plurality of subgroups based on the respective feature attribution of each data sample; (Leung [Column 2 Paragraph 2]; “The user may further interact with the visualization to further explore the data. First, the system may analyze the clusters to identify one or more additional features (i.e., different than the selected feature) that correlate with or explain the different outputs of the different clusters. To determine the additional feature(s), a shallow decision tree may be trained on the data with the cluster membership as a label to be learned, such that the decision tree learns a feature (different from the selected feature) and a respective value that most successfully predicts the cluster membership. The other feature and its value may be displayed along with information showing how the feature distinguishes between the clusters, helping the user to understand relationships between the clustering (describing different data instance behavior with respect to model outputs) and characteristics of the underlying data instances”)
It would have been obvious to perform Leung’s clustering on Cheng’s data samples based on determined feature attributions of the samples before generating feature region descriptions upon the data samples. One would have been motivated to do so because “In complex data sets, this can more readily enable a user to explore how different regions of the data feature space behave with respect to real data samples” (Leung [Column 1 “Summary” Section Paragraph 2]).
Cheng discloses generating a feature region description in feature space with respect to input features …. Cheng does not disclose but Leung discloses input features for a subgroup of the plurality of subgroups. By using the Leung’s input features associated with a subgroup of the plurality of subgroups as replacement for Cheng’s inputted features in Cheng’s feature region description generation, the combination thus discloses generating a feature region description in feature space with respect to input features for a subgroup of the plurality of subgroups.
Regarding Claim 2,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung already discloses wherein the group of data samples is a subset of a set of data samples and the feature region description is determined with respect to the subgroup relative to the set of data samples (Leung [Column 2 Paragraph 2]; “The user may further interact with the visualization to further explore the data. First, the system may analyze the clusters to identify one or more additional features (i.e., different than the selected feature) that correlate with or explain the different outputs of the different clusters.” wherein a cluster is a subset of the set of clusters and an analysis of the feature region description is determined with respect to the cluster relative to the plurality of clusters)
Regarding Claim 3,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung already discloses wherein the feature attribution is determined based on LIME, LRP, DeepLIFT, Integrated Gradients, Shapley Values, Grad-CAM, or Deep Taylor Decomposition (Cheng [0045]; “The explanation engine 130 can also process input data and machine learning models according to one or more model-agnostic approaches, in which the architecture of the model does not matter to the model explainability approach applied. Example approaches include permutation feature importance, partial dependence plots, Shapley values, SHAP (Shapley Additive Explanations), KernelSHAP, TreeSHAP, and integrated gradients.” wherein the feature attribution is based on Shapley Values)
Regarding Claim 4,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung already discloses wherein the feature region description described one or more rules with respect to one or more input features (Cheng [0044]; “For decision trees, in some examples the explanation engine 130 can generate feature attributions based on measures for how each feature contributed to the construction of boosted decision trees within the model. The more a feature is used to make key decisions in the tree, the higher the explanation engine 130 can rate the importance of that feature. The explanation engine 130 can compute the feature attribution explicitly for each feature in a dataset, and output those attributions ordered according to value, e.g., highest to lowest. The feature attribution for a single decision tree can be calculated by the amount that each feature split point improves the performance measure of the decision tree, weighted by the number of observations the node is responsible for” wherein the feature importance rankings interpretable as the feature region description inherently describes some rule with respect to one or more input features (since relative ranking of features is inherently rule-based in its determination of the rankings according to some rule-based comparison of feature strength))
Regarding Claim 5,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung already discloses wherein the feature region description is determined by training a decision tree with respect to membership in the subgroup (Cheng [0044]; “For decision trees, in some examples the explanation engine 130 can generate feature attributions based on measures for how each feature contributed to the construction of boosted decision trees within the model. The more a feature is used to make key decisions in the tree, the higher the explanation engine 130 can rate the importance of that feature. The explanation engine 130 can compute the feature attribution explicitly for each feature in a dataset, and output those attributions ordered according to value, e.g., highest to lowest. The feature attribution for a single decision tree can be calculated by the amount that each feature split point improves the performance measure of the decision tree, weighted by the number of observations the node is responsible for”
Cheng [0045]; “The explanation engine 130 can also process input data and machine learning models according to one or more model-agnostic approaches, in which the architecture of the model does not matter to the model explainability approach applied. Example approaches include permutation feature importance, partial dependence plots, Shapley values, SHAP (Shapley Additive Explanations), KernelSHAP, TreeSHAP, and integrated gradients.” wherein TreeSHAP thus reads on such determined feature attributions being utilized in determining the feature region description through a trained decision tree)
Regarding Claim 6,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung already discloses wherein the instructions are further executable for: determining that a data sample is a member of a subgroup based on the feature region description;
identifying an action associated with the subgroup;
performing the action for the data sample; (Leung [Column 2 Paragraph 2]; “The user may further interact with the visualization to further explore the data. First, the system may analyze the clusters to identify one or more additional features (i.e., different than the selected feature) that correlate with or explain the different outputs of the different clusters.”
Leung [0029]; “FIG. 4C shows example interfaces for exploring characteristics of the different clusters. The user may view the clusters generated for a data set as shown in FIG. 4A and navigate to interfaces to explore characteristics of individual clusters. A local cluster view 440 may be presented, such as cluster views 440A for cluster 0 and cluster view 440B for cluster 1. The cluster view 440 may present the instance-feature variation plot for only the data instances belonging to cluster. In addition, the feature interpretation of the clusters may also be displayed with a visual display of a feature that describes the difference between the clusters. As discussed above, a decision tree (or another approach) may be used to identify another feature (a second feature) other than the selected feature that is associated with cluster membership” wherein the decision tree associated with the feature region description determining cluster membership is interpreted as determining that a data sample is a member of a subgroup based on the feature region description; wherein visualization and description of differences between the clusters in interpretable ways to users for data exploration thus reads on identification of some action associated with the cluster (visualization or difference description) and performance of said action for the cluster identified by user (performing the action for the data sample))
Regarding Claim 7,
Cheng/Leung teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Cheng/Leung further discloses providing a visualization for display to a user device, the visualization showing the feature region description relative to the group of data samples; (Leung [Column 2 Section “Detailed Description” Paragraph 1]; “FIG. 1 is an example environment for a model visualization system 100, according to one embodiment. The model visualization system 100 provides visualization information to a client device 170 for presentation to a user of the client device 170 via a network 160. The network 160 provides a communication channel between the model visualization system 100 and the client device 170. The model visualization system 100 includes a trained computer model 140 which may be a computer model the provides an output based on a multi-dimensional (e.g., multi-feature) input.”)
It would have been obvious to modify the Cheng/Leung combination’s feature region description to additionally incorporate Leung’s method comprising the visualization of the determined feature region description. One would have been motivated to do so because “A model visualization system provides a way to visualize and understand behavior of complex, “black box” computer models by analyzing the effects of modifying individual data instances with respect to a selected feature” (Leung [Column 1 Section “Summary” Paragraph 1]).
Claims 8-14 recite the exact method executed by the system of Claims 1-7 respectively. Thus, Claims 8-14 is rejected for reasons set forth in the rejection of Claim 1-7 respectively.
Claims 15-20 recite a non-transitory computer readable medium comprising instructions to execute the exact method performed by the system of Claims 1-6 respectively. Thus, Claims 15-20 are rejected for reasons set forth in the rejection of Claim 1-6 respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Generating Locally Invariant Explanations for Machine Learning” (US 20240135239 A1) which discloses clustering of data samples into a plurality of local neighborhoods and computation of feature similarities in black-box ML models
“Machine Learning and Reject Inference Techniques Utilizing Attributes of Unlabeled Data Samples” (US 20220318654 A1) which discloses feature affiliation scores determined between model output and unlabeled data samples.
“Root Cause Analysis in Multivariate Unsupervised Anomaly Detection” (US 20210136098 A1) which discloses determination of feature affiliation between subgroups of data samples.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN J KIM whose telephone number is (571)272-0523. The examiner can normally be reached 8-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN J KIM/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights