Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 2/2/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7 10-14, 16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nia (US 20220129791 A1), in view of Carreira (US 20250103856 A1).
Regarding claim 1, Nia discloses “generating, by a processor, a plurality of perturbed instances, wherein each of the plurality of perturbed instances is generated by perturbing one or more encoded features of a[n] AI model instance;” (See [0025]; data samples (perturbed instances) are generated by perturbing the feature values of a target data sample)
“determining, by the processor for each of the plurality of perturbed instances, a distance between one or more encoded features of each perturbed instance and a corresponding one or more of the encoded features of the multimodal AI instance;” (See [0064]; LIME is used to weight the data samples (perturbed instances) to determine a distance between one or more encoded features)
“converting each distance to a weight using a kernel function;” (See [0091]; weights are calculated based on the distances of each target data sample (perturbed instance))
“determining for each weight a modality-specific Shapley value corresponding to a modality associated with each weight” (See [0013], [0023]; LIME is a model explainer used to determine weights for data samples, and Shapley values for each weight can be determined using Kernel SHAP, an expansion of LIME)
“outputting an interpretable surrogate model based on the final weights.” (See [0061], [0095]; surrogate ML models are generated from the training data set, and they are based on the final generated weights for the data samples).
Nia fails to explicitly disclose, “AI model instance” is a “multimodal AI model instance”. Nia fails to further explicitly disclose, “and post-weighting each weight with the modality-specific Shapley value associated with the weight to obtain a plurality of final weights;”. As noted above however, Nia teaches "Shapley values" used in a similar manner. See [0023]; Shapley values for each weight can be determined using Kernel SHAP, an extension of LIME.
Carreira teaches “a multimodal AI model instance” (See [0060]; the AI instance can be multimodal, featuring a combination of different types of data). Carreira further teaches “and post-weighting each weight with the modality-specific … value associated with the weight to obtain a plurality of final weights;” (See [0065]; a set of embeddings (weights) is updated by generating a new weight for each embedding so that each embedding is replaced by the new weights).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Carreira before them to modify Nia to be a multimodal AI model instance and to further modify by post-weighting each weight to obtain final weights. One would be motivated to modify Nia into a multimodal AI model instance in order to give an AI model a more comprehensive understanding of a data set, as being able to take multiple types of input, see e.g., [0060], where Carreira describes multiple examples of input data types, such as image, video, and audio data, would broaden an AI model’s context of a given task, as opposed to being offered only a single type of data, such as text, to accomplish a task. One would also be motivated to modify Nia to post-weight each weight to obtain final weights to be able to update the set of weights in the model to prepare it for use in other models or algorithms.
Regarding claim 2, Nia discloses “generating independently perturbs the one or more features of each perturbed instance according to the modality of each of the one or more features.” (See [0025]; Kernel SHAP generates a data sample by perturbing the features of each perturbed instance).
Regarding claim 3, Nia fails to explicitly disclose, “generating is performed using at least one of a pretrained autoencoder or a generative adversarial network.”.
Carreira teaches “generating is performed using at least one of a pretrained autoencoder or a generative adversarial network.” (See [0104], an autoencoder is shown being used for generating a data set).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Carreira before them to modify Nia to generate an instance using an autoencoder. One would be motivated to do so in order to process the data set of a multimodal AI instance by perturbing the features of an instance to generate a perturbed instance, see e.g., [0104], where Carreira describes that an auto encoder is used to process the data set to generate a corresponding data set.
Regarding claim 4, Nia discloses “determining a distance uses a modality-specific distance metric corresponding to a modality of encoded features for which the distance is determined.” (See [0064]; a distance is determined by determining a radius of a hypersphere, where the hypersphere is the plurality of known data samples and the radius is the distance between one or more instances).
Regarding claim 5, Nia discloses “the kernel function for converting each distance” (See [0091]; a kernel function is used to convert the distance of a data sample to a weight).
Nia fails to explicitly disclose, “a modality-specific kernel function corresponding to a modality of encoded features for which the distance is determined”.
Carreira teaches “a modality-specific kernel function corresponding to a modality of encoded features for which the distance is determined” (See [0060]; if the entity is multimodal, each type or domain of data can also be associated with one or more modality-specific features).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Carreira before them to modify Nia to specify using a modality-specific kernel function when converting distances. One would be motivated to do so because the AI model being used makes use of a multimodal AI model instance, so it would be advantageous to use a kernel function that considers the multimodal nature of the model.
Regarding claim 7, Nia discloses “iteratively tuning hyperparameters of the interpretable surrogate model by comparing true explanations with outputs generated by the interpretable surrogate model.” (See [0061], [0166]; hyperparameters of the model are iteratively tuned when a determination to generate a new set of hyperparameter specifications is made. By comparing the output of a black-box model to the output of a trained surrogate model, Nia compares true explanations with outputs generated by the surrogate model).
Regarding claim 10 and 19, these claims are similar in scope to claim 1.
Regarding claim 11, this claim is similar in scope to claim 2.
Regarding claim 12, this claim is similar in scope to claim 3.
Regarding claim 13, this claim is similar in scope to claim 4.
Regarding claim 14, this claim is similar in scope to claim 5.
Regarding claim 16 and 20, these claims are similar in scope to claim 7.
Claim Rejections - 35 USC § 103
Claims 6, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Nia (US 20220129791 A1), in view of Carreira (US 20250103856 A1), and further view of Ribeiro (LIME: Local Interpretable Model-agnostic Explanations).
Regarding claim 6, Nia fails to explicitly disclose, “the interpretable surrogate model is a sparse linear model comprising weights corresponding to feature importance values”.
However, Ribeiro discloses “the interpretable surrogate model is a sparse linear model comprising weights corresponding to feature importance values”. (See [Page 1, Paragraph 1]; LIME is an interpretable surrogate model that is a sparse linear model. Because LIME is interpretable, its weights are meaningful, which indicates that they can be used to indicate values that are important (feature importance values)).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Ribeiro before them to modify Nia to use a sparse linear model for the surrogate model. One would be motivated to do so to improve the AI model with an explanation algorithm that does not require a complex model to function, see e.g., [Page 1, Paragraph 1], where Ribeiro describes using a simpler model to help with explaining the AI model’s outputs.
Regarding claim 15, this claim is similar in scope to claim 15.
Claim Rejections - 35 USC § 103
Claim(s) 8, 9, 17, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nia (US 20220129791 A1), in view of Carreira (US 20250103856 A1), and further view of Sweeney (US 20230374589 A1).
Regarding claim 8, Nia fails to explicitly disclose, “true explanations comprise true feature attribution weights generated using a logistic regression model”.
Sweeney teaches “true explanations comprise true feature attribution weights generated using a logistic regression model” (See [0040]; hyperparameter optimization is accomplished with logistic regressions by showing how a risk score (true feature attribution weight) is determined by using a parameter-based method to generate a risk score, which includes using logistic regression).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Sweeney before them to modify Nia to incorporate logistic regressions to accomplish optimizing the hyperparameters to generate true feature attribution weights. One would be motivated to do so for the purpose of verifying if the model is outputting correct data, see e.g., [0040] where Sweeney describes how to calculate a risk score by using linear regression, a parameter-based method for understanding how variables and hyperparameters are influenced by different inputs.
Regarding claim 9, Nia fails to explicitly disclose, “values for the hyperparameters are determined based upon a Pearson correlation coefficient or a normalized discounted cumulative gain.”.
Sweeney teaches “values for the hyperparameters are determined based upon a Pearson correlation coefficient or a normalized discounted cumulative gain.” (See [0186]; values for the hyperparameters are determined based on a Pearson correlation by sorting the absolute value of a group of genes' Pearson correlation and then using a classifier model to rank the genes to determine values for the hyperparameters).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Nia and Sweeney before them to modify Nia to determine hyperparameter values using a Pearson correlation coefficient. One would be motivated to do so to find hyperparameters that have a higher correlation with the desired results for the purpose of iteratively tuning the hyperparameters of the model, see e.g., [0186], where Sweeney describes using the results of the Pearson correlation with class labels to determine which values would be used for the hyperparameters.
Regarding claim 17, this claim is similar in scope to claim 8.
Regarding claim 18, this claim is similar in scope to claim 9.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID KIM whose telephone number is (571)272-4331. The examiner can normally be reached 7:30 AM - 4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.K./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141