DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed December 11, 2025 have been fully considered and are persuasive with regard to the rejection of the claims under 35 U.S.C. 101, but they are not persuasive with regard to the rejection of the claims under 35 U.S.C. 103.
Regarding the rejection of claims 1-11 under 35 U.S.C. 101, Applicant argues, in part, that claims 1 and 11 are directed to a technical improvement in the field of model training and image segmentation. In particular, Applicant argues that the claims recite a specific improvement in the efficient generation of high-quality synthetic segmentation masks, which improves the training process of machine-learning models, and thereby advances the functionality of the trained model itself. The examiner agrees that the claims as amended recite such a purported improvement and that the specification provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as pertaining to a purported improvement in training machine learning models (paras. [0003] and [0009]-[0036] of the present specification). For this reason, the claims as amended pass Step 2A, Prong Two of the subject matter eligibility flowchart corresponding to the Alice/Mayo test. Accordingly, the rejection of the claims under 35 U.S.C. 101 is withdrawn.
Regarding the rejection of independent claims 1 and 11 under 35 U.S.C. 103 as being unpatentable over Fisher in view of Nikolenko, Applicant argues that the examiner’s asserted motivation to combine their teachings, namely, expanding the training dataset to improve training performance, would in result in technical incompatibility. Specifically, Applicant argues that the model of Fisher would fail to operate if a normal-object image was input to the model 102 because it allegedly has “no capability to simulate the initial formation of a tumor (oncogenesis) from a completely normal tissue image. Fisher teaches tumor growth, not tumor generation.”
The examiner disagrees. First of all, the term “normal object” is not defined in the present specification. In the medical field, tissue or organs that are considered to be normal because they appear normal can have abnormalities, such as precancerous cells or pre-malignant gene mutations. Secondly, Fisher discloses that model 102 can be used with images that contain abnormalities, but appear normal, because Fisher states that “computational model 102 is used to capture capturing [sic] the genetic makeup, mutations, and/or biomarkers of normal, precancerous, and/or cancerous tissue(s)” (para. [0040]).
Therefore, the input scan image of the object that is input to the computational model 102 can be that of a normally-appearing object with, for example, precancerous tissue. In that case, the synthesized scan image generated by image adaptation model 106 would be an image that contains a tumor in a region of interest (ROI) corresponding to the precancerous tissue region that was in the start scan image, but that had not yet become a tumor. Therefore, Fisher does disclose that the model(s) has the capability to simulate the formation of a tumor from a scan image that does not contain a tumor.
With regard to the rejection of claim 6, the subject matter of which has been incorporated into independent claims 1 and 11 by the amendment, Applicant argues that “the system architecture of Fisher differs fundamentally from that of Soni, such that the Examiner's proposed combination lacks a sound technical rationale.” Specifically, Applicant argues that the process of Soni of augmenting source images with elements/features from a catalog is not a prediction, but rather is a synthetic compositing process that would not be compatible with the predictive process used in Fisher to predict tumor growth over time.
The examiner disagrees. Fisher discusses an embodiment in which the model 106 is implemented as a generative adversarial network (GAN), which is a predictive model, but makes clear that it does not have to be a GAN or any particular configuration of a model: “[i]n certain aspects, image adaptation model 106 is implemented as software, a combination of software and hardware, or by hardware.” (para. [0050]).
Therefore, the image adaptation model 106 could easily be implemented to retrieve images of tumors having shapes and sizes corresponding to different growth periods from a catalog as taught by Soni and combine them with the images output from the computational model 102 of Fisher to create the realistic synthetic images corresponding to second mask recited in claims 1 and 11. The catalog could be, for example, stored in storage module 104, which Fisher discloses can be used to “capture, register and store model output images” of tumor growth that are generated by the computational model 102 (para. [0045] and Fig. 1A). These stored model output images would correspond to the catalog of Soni.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. BRIs for particular claim terms are provided herein. Should Applicant believe that these interpretations are inaccurate, Applicant should point to the portions of the specification that provide a basis for different interpretations.
BRIs for particular claim limitations are as follows:
the limitation “acquiring a region of the normal object” recited in claim 1: Figs. 8 and 9, paras.[0092]-[0093], extracting a region in an image that includes a target object that is normal, i.e., the object is not abnormal; for example, the region of the image may include a healthy kidney without lesions;
“first mask” recited in claim 1: Figs. 8 and 9, paras.[0092]-[0093], the region of the image containing the normal object;
the limitation “generate a second mask by changing a state of the first mask” recited in claim 1: Fig. 11, para. [0097] the first mask is changed to generate the second mask;
“third mask” recited in claim 11: paras. [0117]-[0118], an extracted portion of an image that contains an object;
the limitation “region deviating from a region of the normal object” recited in claim 11: para. [0120], a region of the third mask that is different from the normal region of the third mask;
the limitation “a function of estimating” in claim 11: paras. [0120]-[0121], processing the third mask to estimate the normal region or to estimate the abnormal region.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-5 and 7-9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publ. Appl. No. 2023/0126877 A1 to Fisher et al. (hereinafter referred to as “Fisher”) in view of U.S. Publ. Appl. No. 2020/0320345 A1 to Nikolenko et al. (hereinafter referred to as “Nikolenko”) and further in view of U.S. Publ. Appl. No. 2022/0058437 A1 to Soni et al. (hereinafter referred to as “Soni”).
Regarding claim 1, Fisher discloses a manufacturing method of a learning model that estimates a region-of-interest or that estimates a normal region, for an object included in an image, based on a state change with respect to a normal object having a defined state (Fisher discloses training a machine learning model to estimate regions of interest (ROIs) as corresponding to tumor growth over time (para. [0009]), the manufacturing method of a learning model comprising:
causing a computer (Figs. 1A and 1B, system 100 for synthetic data generation and tumor detection, para. [0036]) to:
acquire a region of the normal object included in a processing target image as a first mask, wherein a medical image is applied to the processing target image, and an anatomical structure is applied as the object (Fig. 1A, the input “scan image” is the acquired region of the object; the input scan image constitutes the first mask; Fisher does not explicitly disclose that the object is a “normal” object because the example provided in Fisher indicates that the acquired image includes a tumorous region; however, as indicated above, the acquired scan image can be a normal-appearing image in which the ROI contains, for example, a precancerous region; the anatomical structure applied as the object can be, for example, the human brain, Fig. 1A; the image acquisition is discussed in para. [0040] and paras. [0027] and [0038]: “computerized tomography (CT) scans, magnetic resonance imaging (MM) scans, functional MRI (fMRI) scans, positron emission tomography (PET) scans, three-dimensional (3D) models, and/or the like”);
generate a second mask by combining an abnormality simulated shape simulating an abnormality with the first mask to deform a shape of the first mask (Fig. 1A, paras. [0036]-[0051], the first and second machine learning models 102 and 106, respectively, transform the input scan image corresponding to the first mask of claim 1 into a modified synthetic image representing growth of the tumor at a later point in time and then transform the modified image into a more realistic modified synthetic image; the more realistic modified synthetic image constitutes the second mask of claim 1; Fisher does not explicitly disclose that the second mask is generated by combining an abnormality simulated shape simulating an abnormality with the first mask to deform a shape of the first mask); and
perform training to estimate a difference between the second mask and the first mask as the region-of-interest or perform training to estimate the first mask from the second mask, using the first mask and the second mask as learning data (Figs. 2A and 2B, steps 206 and 224-228, the similarity between the realistic synthesized image (second mask) and the input scan image (first mask) is determined at step 224 and used to train the machine learning model at step 226, which constitutes estimating a difference between the first and second masks as the ROI).
As indicated above, Fisher does not explicitly disclose that the input scan image contains a region of a “normal” object. Nikolenko, in the same field of endeavor, discloses generating a synthetic image for training a machine learning model by acquiring an input image of a normal object, such as a person’s face, and then modifying the input image, e.g., by removing the nose, to generate a synthetic image that is used in combination with the input image to train a machine learning model (Figs. 1-2B, paras. [0020]-[0041] and [0050]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the machine learning system and algorithms of Fisher based on the teachings of Nikolenko to acquire input scan images of normal regions such as input images that do not include a tumor (e.g., precancerous regions that appear normal, as discussed above in the response to Applicant’s arguments) and then generate synthetic images by modifying the input scan images to show tumor development/growth over time. A person of ordinary skill in the art would have been motivated to make the modification to enlarge the training data set to improve training, prevent overfitting and improve detection accuracy. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying the software used to implement model 102 of Fig. 1A of Fisher to synthesize tumors in ROIs of input scan images that do not already contain a tumor, such as ROIs that have been identified as containing precancerous cells).
The combined teachings of Fisher and Nikolenko do not teach generating the second mask by combining a shape simulating an abnormality with the first mask to deform the shape of the first mask. Soni, in the same field of endeavor, discloses generating a data set of synthetic medical images for training a machine learning model to detect features in medical images such as brain lesions (para. [0031]). To generate the data set, the shape of a first mask corresponding to a source image is deformed by using an element augmentation component 112 that combines the source image with one or more mask images (e.g., brain lesions, occluded arteries) retrieved from an element catalog 202 to generate the synthetic image comprising the second mask (paras. [0033] and [0079]). The shape of the image elements/features in the catalog can be varied by varying the spatial dimensions of the elements/features (para. [0081]), which corresponds to deforming the shape simulating the abnormality.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the machine learning system and methods of Fisher based on the teachings of Soni to configure the image adaptation model 106 of Fisher to combine elements/features from a catalog corresponding to abnormalities of different shapes and sizes with images received from the model 102 of Fisher to generate the synthesized images having tumor regions. A person of ordinary skill in the art would have been motivated to make the modification to generate a large data set of synthetic images that can be used to train machine learning models, thereby reducing costs, improving training, preventing overfitting and improving the accuracy with which the models can detect tumors. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (storing the elements features in a catalog in storage model 104 of Fisher and modifying the software executed by the model 106 shown in Fig. 1A of Fisher to retrieve the elements/features from the catalog).
Regarding claim 3, Fisher discloses that the synthetic images indicate tumor growth over time. Since a tumor is a specific type of lesion and since a tumor that has grown in size has a different shape than the shape that it had at an earlier point in time, the process performed by the model 102 and/or by the model 106 combines a shape simulating a lesion to deform the shape of the first mask (paras. [0041]-[0042]).
Regarding claim 4, Fisher does not explicitly disclose that the second mask is generated as an omission of the anatomical structure from the first mask. As indicated above, Nikolenko discloses generating a synthetic image for training a machine learning model by acquiring an input image of a normal object, such as a person’s face, and then modifying the input image by, for example, removing the nose, to generate a synthetic image that is used in combination with the input image to train a machine learning model to recognize the omission of the nose (Figs. 1-2B, paras. [0020]-[0041] and [0050]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the machine learning system and algorithms of Fisher based on the teachings of Nikolenko to synthesize images with regions omitted and to train the machine learning models in step 206 of Fisher to detect simulated tumors even in cases where one or more regions are omitted from the synthesized image as taught by Nikolenko. A person of ordinary skill in the art would have been motivated to make the modification to provide yet another approach for enlarging the training data set to improve training, prevent overfitting and improve the accuracy with which tumors can be detected. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying the software used to implement models 102, 106 and/or 108 shown in Fig. 1A of Fisher).
With regard to claim 5, Fisher discloses that the second mask, i.e., the synthesized image, is generated by expanding the shape of the first mask by growing the tumor to a larger size in the synthesized image (para. [0009]).
Regarding claim 7, the combined teachings of Fisher and Nikolenko do not teach that a plurality of the abnormality simulated shapes can be combined with the first mask source image to generate the second mask synthesized image. Soni discloses that a plurality of the abnormality simulated shapes stored in the catalog 202 can be combined with the first mask source image to generate the second mask synthesized image (para. [0083]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to further modify the machine learning system and methods of Fisher as modified based on the teachings of Nikolenko to combine a plurality of the elements/features from the catalog of Soni corresponding to abnormalities with the source image used in Fisher to synthesize images with abnormal tumor regions. A person of ordinary skill in the art would have been motivated to make the modification to provide yet another approach for enlarging the training data set to improve training, prevent overfitting and improve the accuracy with which tumors can be detected by the detection system of Fisher. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (modifying the software used to implement models 102, 106 and/or 108 shown in Fig. 1A of Fisher) to yield predictable results.
Regarding claim 8, the combined teachings of Fisher and Nikolenko do not teach that the second mask is generated by combining a rotated abnormality simulated shape with the first mask. Soni discloses that the abnormal elements/features from the catalog can be rotated and combined with the first mask source image to yield different synthetic training images (para. [0066]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to further modify the machine learning system and methods of Fisher as modified based on the teachings of Nikolenko to combine a rotated abnormal element/feature from the catalog of Soni with the source image used in Fisher to synthesize images with different types of abnormal tumor regions. A person of ordinary skill in the art would have been motivated to make the modification to provide yet another approach for enlarging the training data set to improve training, prevent overfitting and improve the accuracy with which tumors can be detected by the detection system of Fisher. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (modifying the software used to implement models 102, 106 and/or 108 shown in Fig. 1A of Fisher) to yield predictable results.
With regard to claim 9, Fisher does not explicitly disclose that the second mask, i.e., the realistic synthesized image, is generated by deforming the first mask comprising the input scan image by omitting the tumor from the first mask. Nikolenko discloses generating a synthetic image for training a machine learning model by acquiring an input image of a normal object, such as a person’s face, and then deforming the input image by, for example, removing the nose, to generate a synthetic image that is used in combination with the input image to train a machine learning model to recognize faces even in images where the nose is omitted (Figs. 1-2B, paras. [0020]-[0041] and [0050]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the machine learning system and algorithms of Fisher based on the teachings of Nikolenko to synthesize images with abnormal tumor regions omitted and to train the machine learning models in step 206 of Fisher to detect simulated tumors even in cases where the tumor region is omitted from the synthesized image as taught by Nikolenko (para. [0035] discloses training the models to detect the “beginning stages of a tumor's life”). A person of ordinary skill in the art would have been motivated to make the modification to provide yet another approach for enlarging the training data set to improve training, prevent overfitting and improve the accuracy with which tumors can be detected, such as in cases where a pre-cancerous region is present that has not yet developed into a tumor. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (modifying the software used to implement models 102, 106 and/or 108 shown in Fig. 1A of Fisher) to yield predictable results.
Regarding claim 11, to the extent that claim 11 recites limitations that are recited in claim 1, the rejection of claim 1 applies mutatis mutandis to claim 11.
Fisher discloses a non-transitory, computer-readable medium having a program thereon (para. [0012]) for estimating a region-of-interest or for estimating a normal region, for an object included in an image, based on a state change with respect to a normal object having a defined state, the program causing, when read by a computer, the computer to realize:
a function of extracting a region of an object included in a processing target image as a third mask by applying a learning model, wherein a medical image is applied to the processing target image, and an anatomical structure is applied as the object, wherein the learning model is trained by using a first mask and a second mask as learning data to estimate a difference between the second mask and the first mask as the region-of-interest or to estimate the first mask from the second mask, the first mask is generated by acquiring a region of the normal object included in a processing target image, and the second mask is generated by combining an abnormality simulated shape simulating an abnormality with the first mask to deform a shape of the first mask (see the rejection of claim 1 regarding these limitations as they relate to generation of the first and second mask limitations; the first and second masks corresponding to the scan image input to model 102 and the more realistic modified synthetic image output from model 106, respectively, are used to train the prediction model 108, Fig. 1A, paras. [0052]-[0055], Fig. 2A, block 206 and para. [0069]; once the model 108 has been trained and deployed, the model 108 can be used to detect tumors, para. [0127], which would involve a function of extracting a region of an object included in a processing target image as a third mask by applying model 108 to an image inputted to model 108, paras [0052]-[0054]), and
a function of estimating, from the third mask, a region deviating from a region of the normal object as the region-of-interest or of estimating the normal region from the third mask (paras. [0065]-[0066] discuss model 108 estimating a region in a synthetically-generated image corresponding to the aged tumor deviating from a region of the object in the input scan image).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached on (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL J. SANTOS/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights