Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because figures 1 and 4 merely point out empty boxes with number. The boxes should have text explaining what is going on in the flowchart. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12, and 16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without integration into a practical application or recitation of significantly more.
In the analysis below, the method of independent claim 1 is considered representative of independent claims 1, 11 and 16 since all of the independent claims recite identical steps despite being directed to different statutory matter. Furthermore, each of independent claims 1, 5, 6 and 10 are directed to one of the four statutory categories of eligible subject matter; thus, the claims pass Step 1 of the Subject Matter Eligibility Test (See flowchart in MPEP 2106).
Step 2A, prong 1: Yes
The independent claims are directed to
PNG
media_image1.png
306
638
media_image1.png
Greyscale
When viewed under the broadest most reasonable interpretation, the instant claims are directed to Judicial Exception – an abstract idea belong to the group of mental process. Particularly, steps 1, 2, 3, 4, and 5 can be performed mentally. For example a person can receive an image acquired by a medical scanner, look at the image and identify an artifact, segment the artifact to decide where to perform inpainting, perform inpainting using on the image, and move the inpainted image to a training set for a machine learning model.
Reference may be made to the July 2024 PEG and those various limitations drawn to the mental processes grouping(s), to include those of Example 47 claim 2. The claims/limitations in question are recited at a high level of generality and lack any specifics precluding such ‘performing’, ‘determining’, ‘implementing’, ‘executing’, etc., from being interpreted under the mental processes grouping practically performed in the mind. As identified in the most recent PEG, even a form of automating that broadly/generically involves the use of a machine learning model, would fail to preclude the limitations in question from being drawn to the mental processes grouping (see guidance with respect to ‘apply it’ consideration of MPEP 2106.05(f)). Hence, the limitations 1,2,3,4, and 5 are interpreted as mental steps. Dependent claims similarly analyzed, further limit said ‘executing’ second action, but not in such a manner so as to preclude an interpretation directed to the identified exception.
Additional elements
The additional elements recited in each of the independent claims are the preamble recites that this is a computer-implemented method
Step 2A, prong 2: No
The above-identified additional elements do not integrate the judicial exception into a practical application.
The steps of a) receiving a notification and storing the association by a network interface and b) storing the association by a database amount to data gather which is insignificant pre-solution activity which does not integrate the claimed mental process into a practical application (See MPEP 2106.05(g)). Furthermore, the limitations of “wherein the new signage comprises signage that has recently been placed in an area; and wherein the notification comprises an image of the new signage that has been detected” are also considered to be insignificant extra solution activities that are merely added to the judicial exception (See MPEP 2106.05(g)).
The use of a computer implemented method amounts to merely using a generic computer as a tool to perform the claimed mental process. Implementing an abstract idea on a computer does not integrate a judicial exception into a practical application (See MPEP 2106.05(f)).
Moreover, the additional elements of the claims do not recite an improvement in the functioning of a computer or other technology or technical field, the claimed steps are not performed using a particular machine, the claimed steps do not effect a transformation, and the claims do not apply the judicial exception in any meaningful way beyond generically linking the use of the judicial exception to a particular technological environment (See MPEP 2106.04(d)). Therefore, the analysis under prong two of step 2A of the Subject Matter Eligibility Test does not result in a conclusion of eligibility (See flowchart in MPEP 2106).
Step 2B: No
The pending claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As explained above in Prong Two, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer. Each of the additional elements are generic computer features which perform generic computer functions that are well-understood, routine, and conventional and do not amount to more than implementing the abstract idea with a computerized system.
Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation, and mere implementation on a generic computer does not add significantly more to the claims. Accordingly, the analysis under step 2B of the Subject Matter Eligibility Test does not result in a conclusion of eligibility (See flowchart in MPEP 2106).
Regarding claim 2, a human looking at the inpainted image could extract features and label the features.
Regarding claim 3, a human looking at the image could identify an artifact including motion artifacts, text marker, or an artifact induced by an object imaged with the patient.
Regarding claim 4, the human could receive an image generated by a chest xray, mri, etc.
Regarding claim 5, the additional limitation of applying a machine-learning model to classify image features is generic as it specifies no details of the machine learning model. The Examiner interprets this as mere instructions to apply an exception by implanting a classifying using a generic machine learning model, see 2106.05(f).
Regarding claim 6, a human could segment the artifact based on a class activation map.
Regarding claims 7-8, similar to claim 5 a deep generative machine learnt model is generic specifying no particulars of the model thus this claim is interpreted under 2106.05(f) as mere instructions to apply an exception.
Regarding claim 9 a human could train the machine learning model (supervised learning).
Regarding claim 10, a human could display the inpainted image (share it on a screen or hold up a physical image).
Claims 11-12 are similarly analyzed to claims 1-2.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-11, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Kirkerød et al. Unsupervised preprocessing to improve generalization for medical image classification (hereinafter “Kirkerød”, cited in the IDS) in view of Mironica et al. US 2022/0392025.
Regarding claim 1, Kirkerød discloses
A computer-implemented method for training a machine- learning model for classifying image features in an image acquired by a medical scanner (see the abstract, multiple approaches to inpaint problematic areas in the images to improve anomaly classification)
PNG
media_image2.png
262
420
media_image2.png
Greyscale
, comprising: a} receiving the image acquired by the medical scanner (see figure 1 and the abstract, preprocessing the input data [such as seen in figure 1] to inpaint problematic areas)
PNG
media_image3.png
326
390
media_image3.png
Greyscale
b} identifying an artefact in the image (see figure 1 with the unwanted overlay identified, also see section III A. Preprocessing, the Kvasir dataset has some unwanted artefacts that present in a good portion of the data)
PNG
media_image4.png
300
410
media_image4.png
Greyscale
; c}
PNG
media_image5.png
136
406
media_image5.png
Greyscale
PNG
media_image6.png
124
406
media_image6.png
Greyscale
; and e} incorporating the inpainted image into a training set for the machine-learning model (see the introduction, deep learning uses the Kvasir images as training, and the training images are first preprocessed to inpaint the artefacts)
PNG
media_image7.png
503
409
media_image7.png
Greyscale
Kirkerød discloses that only parts corresponding to a mask are inpainted, but does not explicitly disclose segmenting the artefact to obtain a segmentation mask although it would appear obvious based on figure 2 that a segmentation mask of the artefact is obtained.
However for expedited examination a further reference will be provided to teach segmenting artifacts to obtain a segmentation mask to be inpainted.
Mironica discloses an image restoration system that inpaints pixels of local defects that are indicated by a segmentation mask (see paragraph 0063).
PNG
media_image8.png
228
350
media_image8.png
Greyscale
Kirkerød and Mironica are analogous art because they are from the same field of endeavor of inpainting to enhance images.
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine Kirkerød and Mironica to segment the artefact to obtain a segmentation mask of a region, on which to perform inpainting. The motivation would be to find the proper places to inpaint without damaging parts with no artefacts.
Regarding claim 3, as seen in figure 1, Kirkerød discloses where the artefact comprises a text marker added onto the image.
Regarding claim 4, although Kirkerød discloses using endoscopy images, the images used would be a matter of design choice and the general principle of inpainting artefacts would apply to be equally effective in other types of medical imaging.
Regarding claim 5, Kirkerød discloses that it is well known to analyze a classification result to determine whether a class exhibits an error exceeding a threshold and determining a co-occurrence of the artefact in the image (see section II related works).
PNG
media_image9.png
224
400
media_image9.png
Greyscale
Regarding claims 6-7, Mironica discloses segmenting the artefact with a deep generative machine-learnt model (see paragraph 0007, figure 3 uses a neural network).
PNG
media_image10.png
46
252
media_image10.png
Greyscale
Regarding claim 8, Kirkerød discloses a generative adversarial network (see section V.
PNG
media_image11.png
204
410
media_image11.png
Greyscale
Regarding claim 9, Kirkerød as discussed above in the introduction discloses training the model on the training set.
Regarding claim 10, Kirkerød discloses displaying the inpainted image (see figure 2).
Claim 11 is similarly analyzed to claim 1.
Claim 16 is similarly analyzed to claim 1.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kirkerød in view of Mironica and further in view of Jiang et al. US 2019/0377979 (hereinafter “Jiang”).
Regarding claim 2, as discussed above the combination of Kirkerød and Mironica disclose the limitations of claim 1.
Kirkerød discloses using models trained on enhanced datasets to improve classification scores of models, but does not go into specific detail regarding the classification models as it is generalized to any classification model. Thus Kirkerød does not explicitly disclose extracting features from the inpainted image and incorporating data that labels features into the training set. However it is well known with classification models to extract features and label them as shown by Jiang.
Jiang discloses encoding a training image through feature extraction and generating a label of the feature (see step 201 of figure 2)
PNG
media_image12.png
320
406
media_image12.png
Greyscale
Kirkerød and Jiang are analogous art because they are from the same field of endeavor of image classification.
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine Kirkerød, Mironica and Jiang to use the classification model of Kirkerød to extract features and label the features. The motivation would be to help diagnose diseases based on the images.
Claim 12 is similarly analyzed to claim 2.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see attached 892 notice of references cited.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN B STREGE whose telephone number is (571)272-7457. The examiner can normally be reached M-F 9-5 (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN B STREGE/ Primary Examiner, Art Unit 2669
Prosecution Insights