Notice of Pre-AIA or AIA Status
The present application 18/414,076, filed on 1/16/2024 (or after March 16, 2013), is being examined under the first inventor to file provisions of the AIA (First Inventor to File).
In the event the determination of the status of the application as subject to AIA 35
U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
DETAILED ACTION
Claims 1-20 are pending in this application.
Drawings
The Drawings filed on 3/14/2024 are acceptable for examination purpose.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3, 5-8,10,14,18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to Claim 3, it is unclear what is meant by “wherein the one or more instructions to update the trained model comprise an indication to a user that the trained model has failed robustness criteria, an indication of improvements to make to the trained model, or an indication to automatically generate training data for improving the trained model. The term improvements , improving is a relative term that makes the claim indefinite
As to Claim 5-8 , “for the plurality of data objects assigned by the baseline classification data to a first class of the plurality of classes, wherein the plurality of data objects comprises a set of one or more images,
define a plurality of spatial regions in the one or more images;
calculate, for each spatial region, a respective perturbation importance score based on perturbations applied to the spatial region that caused misclassification;
display a visual representation of an example image from the first class; and
display a visual overlay over the example image indicating the perturbation “importance score” for one or more of the plurality of spatial regions”. The term “importance score” is a relative term that makes the claim indefinite.
Further, as to claim 8, it is unclear what is meant by determining a “minimum level” of perturbation intensity that caused misclassification, and “minimum level” is a relative term that makes the claim indefinite
As to Claim 10, it is unclear what is meant by “wherein displaying the visual representation indicating average class confidence levels comprises displaying a first indication of a “first average class confidence level” by which the trained model classified the perturbed data objects into the first class. The term “average class confidence level” is a relative term(s) that makes the claim indefinite
As to Claim 14,18, it is unclear what is meant by wherein the visual representation comprises a first line graph indicating average class confidence levels generated by the trained model at “various levels of perturbation intensity”, without defining “confidence levels”, “various levels”, as such “confidence levels”, “various levels” is a relative term(s) that makes the claim indefinite
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application.
Claim 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The eligibility analysis in support of these findings is provided below, in accordance with the 2019 Revised Patent Subject Matter Eligibility Guidance, Federal Register (84 FR 50) on January 7, 2019 hereinafter 2019 PEG
Step 1. In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), it is noted that the method of claim 1,19-20 directed to one of the eligible categories of subject matter and therefore satisfy Step 1.
Step 2A. In accordance with Step 2A prong one of the 2019 PEG, the limitations reciting the abstract idea are highlighted, and the limitations directed to additional elements are highlighted, as set forth in exemplary claim 1
Claim 1,19-20
.” A system for evaluating trained models, the system comprising one or more processors configured to cause the system to:
receive a trained model;
receive a set of test data comprising a plurality of data objects;
receive baseline classification data that assigns each data object to a class of a plurality of classes;
apply one or more perturbation operations to the test data to generate, for each of the data objects in the set of test data, a respective plurality of perturbed data objects;
apply the trained model to each of the perturbed data objects to generate, for each of the perturbed data objects, respective post-perturbation classification data, wherein the respective post-perturbation classification data indicates classification of the respective perturbed data object into at least one of the plurality of classes and an associated confidence level of the trained model with respect to the classification;
determine, for each of the perturbed data objects, whether the respective post-perturbation classification data indicates a misclassification as compared to the baseline classification data; and
generate and display a visualization based on the determination of whether the post-perturbation classification data indicates a misclassification”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. For example, in the context of this claim, this limitation encompasses the user thinking of data collection such as classification data
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas set forth in the 2019 PEG. Accordingly, the claim recites an abstract idea.
With respect to Step 2A prong two of the 2019 PEG, the judicial exception is not integrated into a practical application. The additional elements are directed to method steps, however, these elements fail to integrate the abstract idea into a practical application because they fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular data structure of receive a trained model, receive baseline classification data, confidence level of the trained model, generate and display a visualization based, to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment.
Furthermore, although these elements have been fully considered, they are directed to the use of generic computing elements (fig 1, fig 7, 0047,0049, 0070, 0129- 0135 of the instant specification make it clear that the disclosed functionality is implemented on well-known computing systems and general purpose computing devices) to perform the abstract idea, which is not sufficient to amount to a practical application (as noted in the 2019 PEG) and is amount to simply saying "apply it" using a general purpose computer, which merely serves to tie the abstract idea to a particular technological environment computer based operating environment) by using the computer as a tool to perform the abstract idea.
Since the analysis of Step 2A prong one and prong two results in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception
Step 2B. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional method limitations are directed to a generic computer, at a very high level of generality and without imposing meaningful limitations on the scope of the claim. In addition fig 1, fig 7, 0047,0049, 0070, 0129- 0135 of the instant specification describe generic off-the-shelf computer-based elements for implementing the claimed invention which does not amount to significantly more than the abstract idea and is not enough to transform an abstract idea into eligible subject matter. Such generic, high-level, and nominal involvement of a computer or computer-based elements for carrying out the invention merely serves to tie the abstract idea to a particular technological environment, which is not enough to render the claims patent-eligible, as noted at pg. 74624 of Federal Register/Vol. 79, No. 241, citing Alice, which in turn cites Mayo. Further, See, e.g., Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 134 S. Ct. 2347, 2359-60, 110 USPQ2d 1976, 1984 (2014). See also OIP Techs. v. Amazon.com, 788 F.3d 1359, 1364, 115 USPQ2d 1090, 1093-94 (Fed. Cir. 2015) ("Just as Diehr could not save the claims in Alice, which were directed to 'implement[ing] the abstract idea of intermediated settlement on a generic computer', it cannot save O/P's claims directed to implementing the abstract idea of price optimization on a generic computer.") (citations omitted). See also, Affinity Labs of Texas LLC v. DirecTV LLC, 838 F.3d 1253, 1257-1258 (Fed. Cir. 2016) (mere recitation of a GUI does not make a claim patent-eligible); Intellectual Ventures I LLC v. Capital One Bank, 792 F.3d 1363, 1370 (Fed. Cir. 2015) ("the interactive interface limitation is a generic computer element".)
The additional elements are broadly applied to the abstract idea at a high level of generality ("similar to how the recitation of the computer in the claims in Alice amounted to mere instructions to apply the abstract idea of intermediated settlement on a generic computer,") as explained in MPEP § 2106.05(f)) and they operate in a well-understood, routine, and conventional manner.
MPEP § 2106.05 (d)(II) sets forth the following:
The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g. at a high level of generality) as insignificant extra-solution activity.
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec...; TLI Communications LLC v. AV Auto. LLC...; OIP Techs., Inc., v. Amazon.com, Inc... ; buySAFE, Inc. v. Google, Inc...;
Performing repetitive calculations, Flook ... ; Bancorp Services v. Sun Life...;
Electronic recordkeeping, Alice Corp...; Ultramercial... ;
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc...;
Electronically scanning or extracting data from a physical document, Content Extraction and Transmission, LLC v. Wells Fargo Bank...; and
A web browser's back and forward button functionality, Internet Patent Corp. v. Active Network, Inc.
Courts have held computer-implemented processes not to be significantly more than an abstract idea (and thus ineligible) where the claim as a whole amounts to nothing more than generic computer functions merely used to implement an abstract idea, such as an idea that could be done by a human analog (i.e., by hand or by merely thinking).
Claim 2, further elaborates “wherein the one or more processors are configured to cause the system to generate, based on the determination of whether the post-perturbation classification data indicates a misclassification, one or more instructions to update the trained model”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 3, further elaborates “wherein the one or more instructions to update the trained model comprise an indication to a user that the trained model has failed robustness criteria, an indication of improvements to make to the trained model, or an indication to automatically generate training data for improving the trained model”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. .
Claim 4, further elaborates “wherein the one or more processors are configured to cause the system to execute the one or more instructions to update the trained model:, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. .
Claim 5, further elaborates on wherein the one or more processors are configured to cause the system to:
for the plurality of data objects assigned by the baseline classification data to a first class of the plurality of classes, wherein the plurality of data objects comprises a set of one or more images,
define a plurality of spatial regions in the one or more images;
calculate, for each spatial region, a respective perturbation importance score based on perturbations applied to the spatial region that caused misclassification;
display a visual representation of an example image from the first class; and
display a visual overlay over the example image indicating the perturbation importance score for one or more of the pluralities of spatial regions”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer component(s) such as by the processor. That is, other than reciting “by a processor”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “by a processor” calculate, for each spatial region, If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim 5 recites an abstract idea, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 6, further elaborates “wherein calculating, for each spatial region, a respective perturbation importance score comprises:
applying a Gaussian blur to the respective spatial region;
minimizing L2 Norm and total variational noise of perturbations applied to the respective spatial region; and
determining a minimum level of perturbation intensity that caused misclassification”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer component(s) such as by the processor. That is, other than reciting “by a processor”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “by a processor” calculate, for each spatial region, If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim 6 recites an abstract idea, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 7, further elaborates “wherein the one or more processors are configured to cause the system to:
calculate, for each image in the set of one or more images, one or more respective feature importance scores based on perturbations applied to the image that changed one or more features of the image that caused misclassification; and
generate and display a histogram indicating, for each image in the first class, the one or more feature importance scores”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer component(s) such as by the processor. That is, other than reciting “by a processor”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “by a processor” calculate, for each image…….., If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim 7 recites an abstract idea, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 8, further elaborates “wherein calculating, for each image in the set of one or more images, one or more respective feature importance scores comprises:
generating an image pixel mask for the respective image;
generating a salience pixel mask for the respective image;
combining the image pixel mask and the salience pixel mask; and
calculating one or more feature importance scores based on the combined image pixel mask and salience pixel mask”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer component(s) such as by the processor. That is, other than reciting “by a processor”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “by a processor” calculating one or more feature …….., If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim 8 recites an abstract idea, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 9, further elaborates “wherein the one or more processors are configured to cause the system to:
select a subset of post-perturbation classification data, wherein the subset of post-perturbation classification data corresponds to data objects assigned to a first class by the baseline classification data; and
generate and display a visual representation indicating average class confidence levels generated by the trained model for the selected subset of data at various levels of perturbation intensity”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 10, further elaborates “wherein displaying the visual representation indicating average class confidence levels comprises displaying a first indication of a first average class confidence level by which the trained model classified the perturbed data objects into the first class”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 11, further elaborates “wherein displaying the first indication comprises:
displaying a first region of the first indication at which average class confidence levels for the first class are highest compared to other classes; and
simultaneously displaying a second region of the first indication at which average class confidence levels for the first class are not highest compared to other classes”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 12, further elaborates “wherein displaying the visual representation indicating average class confidence levels comprises displaying a second indication of a second average class confidence level by which the trained model classified the perturbed data objects into a second class different from the first class”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 13, further elaborates “wherein the second indication is displayed for levels of perturbation intensity at which class confidence level for the second class is higher than for any other class”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 14, further elaborates “wherein the visual representation comprises a first line graph indicating average class confidence levels generated by the trained model at various levels of perturbation intensity”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 15, further elaborates “wherein the first line graph comprises one or more lines corresponding to one or more classes of the plurality of classes”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 16, further elaborates “wherein the one or more processors are configured to cause the system to:
detect a user input comprising a selection of a first region visually indicating a first option to add to the first line graph one or more lines corresponding to one or more classes of the plurality of classes; and
in response to detecting the user input, add the one or more lines to the first line graph”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 17, further elaborates “wherein the one or more processors are configured to cause the system to:
detect a user input comprising a selection of a second region visually indicating a second option to remove from the first line graph one or more lines corresponding to one or more classes of the plurality of classes; and
in response to detecting the user input, remove the one or more lines from the first line graph”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 18, further elaborates “wherein the one or more processors are configured to cause the system to:
detect a user input comprising a selection of a region visually indicating a name of a class of the plurality of classes; and
in response to detecting the user input, generate and display a second line graph indicating average class confidence levels of the trained model at various levels of perturbation intensity for the class; and
generate and display a third line graph indicating associated confidence levels of the trained model at various levels of perturbation intensity for at least one data object in the class”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mattyus et al., (hereafter Mattyus), US Pub. No. 2019/0147320 published May, 2019 in view of Goodsitt et al., (hereafter Goodsitt), US Pub. No. 2020/0272944 published Aug, 2020
As to Claim 1,19-20, Mattyus teaches a system which including “a system for evaluating trained models, the system comprising one or more processors configured to cause the system to (Mattyus: Abstract)
“receive a trained model” (Mattyus: fig 4, 0111,0140 – Mattyus teaches receiving training data from the training model supporting the adversarial network model particularly MatAn data from generator trained)
PNG
media_image1.png
189
196
media_image1.png
Greyscale
“receive a set of test data comprising a plurality of data objects” (Mattyus: fig 1, 0139 – Mattyus teaches one or more image data object sets generated from the generator network system including images as input data objects to uniquely have data identification representing the object sets);
“receive baseline classification data that assigns each data object to a class of a plurality of classes” (Mattyus: 0013, 0023,0074 – Mattyus teaches image objects including classification data particularly objects within plurality of predetermined classification);
“apply one or more perturbation operations to the test data to generate, for each of the data objects in the set of test data, a respective plurality of perturbed data object” (Mattyus: 0019, 0072,0116 – Mattyus teaches applying perturbation to the training data objects of the generator MatAn particularly training data including image objects for example fig 5A-5B describes applying perturbation applied on to the image dataset objects)
PNG
media_image2.png
159
240
media_image2.png
Greyscale
PNG
media_image3.png
227
327
media_image3.png
Greyscale
“apply the trained model to each of the perturbed data objects to generate, for each of the perturbed data objects, respective post-perturbation classification data, wherein the respective post-perturbation classification data indicates classification of the respective perturbed data object into at least one of the plurality of classes and an associated confidence level of the trained model with respect to the classification” (Mattyus: 0023, 0068,0112, 0116-0121,0125 - Mattyus; teaches generative adversarial network that train generative models using generative samples that including both negative, positive samples as detailed in fig 5A-5B, applied to post-perturbation operations based on classification of image objects within predetermined classification including identification of objects of the trained model, and fig 6C shows typical pertubations applied on the image objects);
PNG
media_image4.png
162
252
media_image4.png
Greyscale
“determine, for each of the perturbed data objects, whether the respective post-perturbation classification data indicates as compared to the baseline classification data” (Mattyus: 0120-0121,0147 -Mattyus teaches examples of training data objects including perturbed data objects within predetermined classification) ; and
“generate and display a visualization based on the determination of whether the post-perturbation classification data” (Mattyus : fig 6, fig 8-9, 0148,0151-0152 – Mattyus teaches displaying and/or visualizing image objects of perturbed training data)
It is however, noted that Mattyus does not disclose “classification data indicates a misclassification”, although Mattyus teaches baseline classification data for each of the perturbed data objects (Mattyus: fig 5-6, 0120-0125). On the other hand, Googsitt disclosed “classification data indicates a misclassification (Goodsitt: Abstract, fig 1A-1B, 0034-0035,0039 - Googsitt teaches generating datasets and determining the misclassification of the data where generation model generates not only continuously training of the training model, but also identifying misclassified data
PNG
media_image5.png
189
182
media_image5.png
Greyscale
PNG
media_image6.png
182
178
media_image6.png
Greyscale
It would have been obvious to a person of ordinary skill in the art at the time of filing the claimed invention machine lerning accuracy by synthetic data generator particularly determining the data types of the misclassified datasets of Googsitt et al., into training data using generatic network processing of classified data of Mattyus et al., because both Mattyus, Goodsitt teaches trained model where training data sets are used in predeiction of the data objects (Mattyus: Abstract; Goodsitt: fig 1A-1B), and they both are from the same field of endeavor. Because both Mattyus, Goodsitt teaches training model(s), it would have been obvious to one skill ed in the art to substitute and/or modify one method for the other particularly determining the misclassified data from the training model that including training data sets assigned scores, while misclassification may be determined based on the assigned classification scores (Goodsitt: 0008-0009), thereby improving the overall training model and classification of data, implementing procedures iterated until the misclassification is no longer determined during the training of the model(s) (Goodsitt: 0010)
As to Claim 2, the combination of Mattyus, Goodsitt disclosed “ wherein the one or more processors are configured to cause the system to generate, based on the determination of whether the post-perturbation classification data indicate, is a misclassification, one or more instructions to update the trained model (Mattyus: fig 1-2, 0074-0075). On the other hand, Goodsitt disclosed “classification data indicate, is a misclassification, one or more instructions to update the trained model” (Goodsitt: fig 1A-1B, Abstract,0034-0035)
As to Claim 3, he combination of Mattyus, Goodsitt disclosed “wherein the one or more instructions to update the trained model comprise an indication to a user that the trained model “ (Mattyus:0068-0069, 0074-0075) On the other hand, Goodsitt disclosed “trained model has failed robustness criteria, an indication of improvements to make to the trained model, or an indication to automatically generate training data for improving the trained model“ (Goodsitt: 0035-0038, fig 1A-1B)
As to Claim 4, the combination of Mattyus, Goodsitt disclosed “wherein the one or more processors are configured to cause the system to execute the one or more instructions to update the trained model” (Mattyus: 0107-0108, fig 3).
As to Claim 5, the combination of Mattyus, Goodsitt disclosed
for the plurality of data objects assigned by the baseline classification data to a first class of the plurality of classes, wherein the plurality of data objects comprises a set of one or more images,
“define a plurality of spatial regions in the one or more images” (Mattyus: 0077,0112);
“calculate, for each spatial region, a respective perturbation based on perturbations applied to the spatial region” (Mattyus: fig 5A-5B, 0113-0115);
“display a visual representation of an example image from the first class” (Mattyus: fig 5A-5B); and
“display a visual overlay over the example image indicating the perturbation for one or more of the plurality of spatial regions” (Mattyus: fig 6A-6C, 0117-0118).
On the other hand, Goodsitt disclosed misclassification,(Goodsitt: Abstract, fig 1A-1B) importance score” (Goodsitt: 0059, fig 6A)
PNG
media_image7.png
187
779
media_image7.png
Greyscale
As to Claim 6, the combination of Mattyus, Goodsitt disclosed
applying a Gaussian blur to the respective spatial region (Mattyus: 0068-0070 – Mattyus teaches improve processing particularly surrounding pixels and noise vector calculation);
“minimizing L2 Norm and total variational noise of perturbations applied to the respective spatial region” (Mattyus:0068-0070, 0116-0117) ; and
“determining a minimum level of perturbation intensity” (Mattyus: 0114,0117-0118) . On the other hand, Goodsitt disclosed “misclassification” (Goodsitt: fig 1A-1B, Abstract).
As to Claim 7, the combination of Mattyus, Goodsitt disclosed
“calculate, for each image in the set of one or more images, one or more respective feature based on perturbations applied to the image that changed one or more features of the image” (Mattyus: fig 6A-6C, 0117-0120), and
generate and display a histogram indicating, for each image in the first class, the one or more feature” (Mattyus: fig 7-8, 0128-0129). On the other hand, Goodsitt disclosed “feature importance scores(Goodsitt: fig 6, 0011,0059, fig 6A)
As to Claim 8, the combination of Mattyus, Goodsitt disclosed
“generating an image pixel mask for the respective image” (Mattyus: fig 5A-5B,0155);
“generating a salience pixel mask for the respective image” (Mattyus: 0114-0115, 0155);
“combining the image pixel mask and the salience pixel mask” (Mattyus: 0154-0155); and
“calculating one or more feature based on the combined image pixel mask and salience pixel mask” (Mattyus: 0154-0156). On the other hand, Goodsitt disclosed “feature importance scores(Goodsitt: fig 6, 0011,0059, fig 6A)
As to Claim 9, the combination of Mattyus, Goodsitt disclosed
“select a subset of post-perturbation classification data, wherein the subset of post-perturbation classification data corresponds to data objects assigned to a first class by the baseline classification data” (Mattyus: 0013, 0023,0074,0112); and
“generate and display a visual representation indicating average class confidence levels generated by the trained model for the selected subset of data at various levels of perturbation intensity” (Mattyus: fig 5A-5B,fig 6A-6C, 0119-0125).
As to Claim 10, the combination of Mattyus, Goodsitt disclosed “ wherein displaying the visual representation indicating average class confidence levels comprises displaying a first indication of a first average class confidence level by which the trained model classified the perturbed data objects into the first class” (Mattyus: 0067-0068, 0071-0072).
Claim 11, the combination of Mattyus, Goodsitt disclosed:
“displaying a first region of the first indication at which average class confidence levels for the first class are highest compared to other classes” (Mattyus: fig 5A-5B); and
“simultaneously displaying a second region of the first indication at which average class confidence levels for the first class are not highest compared to other classes” (Mattyus: fig 6A-6C, 0116-0117).
As to Claim 12, the combination of Mattyus, Goodsitt disclosed “ wherein displaying the visual representation indicating average class confidence levels comprises displaying a second indication of a second average class confidence level by which the trained model classified the perturbed data objects into a second class different from the first class” (Mattyus:0067-0069, 0071-0072)
As to Claim 13, the combination of Mattyus, Goodsitt disclosed “wherein the second indication is displayed for levels of perturbation intensity at which class confidence level for the second class is higher than for any other class” (Mattyus: 0068-0069).
As to Claim 14, the combination of Mattyus, Goodsitt disclosed “wherein the visual representation comprises a first line graph indicating average class confidence levels generated by the trained model at various levels of perturbation intensity” (Mattyus: fig 5A-5B, fig 6A-6C, 0119-0125). .
As to Claim 15, the combination of Mattyus, Goodsitt disclosed “wherein the first line graph comprises one or more lines corresponding to one or more classes of the plurality of classes” (Mattyus: fig 7B-7D, 0146-0148).
As to Claim 16, the combination of Mattyus, Goodsitt disclosed
“detect a user input comprising a selection of a first region visually indicating a first option to add to the first line graph one or more lines corresponding to one or more classes of the plurality of classes” (Mattyus: 0079-0080,0158); and
“in response to detecting the user input, add the one or more lines to the first line graph” (Mattyus: fig 7B-7D).
As to Claim 17, the combination of Mattyus, Goodsitt disclosed
“detect a user input comprising a selection of a second region visually indicating a second option to remove from the first line graph one or more lines corresponding to one or more classes of the plurality of classes” (Mattyus: fig 5A-5B, 6A-6C, 0114-0117); and
“in response to detecting the user input, remove the one or more lines from the first line graph” (Mattyus:. Mattyus: fig 7B-7D).
As to Claim 18, the combination of Mattyus, Goodsitt disclosed
“detect a user input comprising a selection of a region visually indicating a name of a class of the plurality of classes” (Mattyus: fig 7B-7D, 0146-0148).; and
“in response to detecting the user input, generate and display a second line graph indicating average class confidence levels of the trained model at various levels of perturbation intensity for the class” (Mattyus:0067-0069, 0071-0072); and
“generate and display a third line graph indicating associated confidence levels of the trained model at various levels of perturbation intensity for at least one data object in the class” (Mattyus: fig 5A-5B, fig 6A-6C, 0119-0125). .
.
Conclusion
The prior art made of record
a. US Pub. No. 2019/0147320
b. US Pub. No. 2020/0272944
Examiner's Note: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
SEE MPEP 2141.02 [R-5] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert. denied, 469 U.S. 851 (1984) In re Fulton, 391 F.3d 1195, 1201,73 USPQ2d 1141, 1146 (Fed. Cir. 2004). >See also MPEP §2123.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
The prior art made of record, listed on form PTO-892, and not relied upon, if any, is considered pertinent to applicant's disclosure
Authorization for Internet Communications
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax (not Examiner's Fax), Regular postal mail, or EFS Web using PTO/SB/439.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Srirama Channavajjala whose telephone number is 571-272-4108. The examiner can normally be reached on Monday-Friday from 8:00 AM to 5:30 PM Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gorney, Boris, can be reached on (571) 270- 5626. The fax phone numbers for the organization where the application or proceeding is assigned is 571-273-8300 Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)
/Srirama Channavajjala/Primary Examiner, Art Unit 2154
Prosecution Insights