DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to independent claim(s) 1, 10, and 14 (and their respective deponent claims), have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Further, with respect to Muller reference Applicant argues that “the claims require a second trained machine learning model that takes as input data representative of the image feature vector and generates natural language text. Muller does not disclose such a model. Rather, Muller's downstream processing is a classifier, not a natural language generation model. There is no teaching or suggestion in Muller of: transforming a feature vector into natural language output, or employing a second model specifically configured for text generation” (please see Remarks, page 12, first paragraph).
Examiner respectfully disagrees, as Muller, Fig. 3B, Fig. 11, and paragraphs 50 and 70, discloses the feature vectors are input into a trained machine learning classifier, wherein they are classified as benign or as malignant, i.e., resulting class corresponds to the natural language text as claimed. Hence Muller machine learning classifier provide as an output a natural language text. Examiner suggests Applicant to further elaborate on how the text/natural language is generated in order to overcome the cited references.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 10, and 14-19, is/are rejected under 35 U.S.C. 103 as being unpatentable over Muller (US PGPUB 2023/0041804 A1) and further in view of Zheng (CN 111681730 A, English Translation of the CN is being attached herewith and used for citation below).
As per claim 1, Muller discloses a computer-implemented method for generating natural language text describing a change between first medical imaging data and second medical imaging data (Muller, Fig. 3B, and paragraphs 38-40), the method comprising:
obtaining imaging data representative of the first medical imaging data and the second medical imaging data, or of a difference between the first medical imaging data and the second medical imaging data (Muller, Fig. 4, Fig. 5 and paragraphs 57 and 58);
inputting the imaging data into a (first trained machine learning) model to generate an image feature vector representative of the difference between the first medical imaging data and the second medical imaging data (Muller, Fig. 3B, and paragraphs 42, 49 and 58); and
inputting data representative of the image feature vector into a second trained machine learning model to generate the natural language text (Muller, Fig. 3B, Fig. 11, and paragraphs 50 and 70, discloses the feature vectors are input into a trained machine learning classifier, wherein they are classified as benign or as malignant; please note this resulting class corresponds to the natural language text as claimed).
Muller discloses generation of image feature vector utilizing a different mathematical model not generated by a first trained machine leaning model as claimed, further Muller also discloses inputting data representative of the image feature vector into a second trained machine learning model to generate the natural language text as explained above however, said difference is well known in the art for instance Zhang discloses inputting the imaging data into a first trained machine learning model to generate an image feature vector (Zheng, Fig. 6:11, and related text, specifically please see pages 6-7);
inputting data representative of the image feature vector into a second trained machine learning model to generate the natural language text including at least one sentence describing the change between the first medical imaging data and the second medical imaging data (Zhang, Fig. 6:13, and pages 7-8, discloses the analysis module 13 is specifically used for the sentence pair of the first mark sentence and the second mark sentence to match the character string, determining the second mark sentence corresponding to the difference character of the first mark sentence; and determining the second medical image report relative to the modified type of the first medical image report according to the semantic meaning of the difference character);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Muller teachings by implementing a machine learning model to the system, as taught by Zheng.
The motivation would be to improve the accuracy of the subsequent comparison analysis result of the medical image report (page 5), as taught by Zheng.
As per claim 2, Muller in view of Zheng further discloses the method of claim 1, comprising: generating a second medical text report associated with the second medical imaging data based on a first medical text report associated with the first medical imaging data and the natural language text describing the change (Zheng, pages 6-7)
As per claim 3, Muller in view of Zheng further discloses the method of claim 1, wherein the first medical imaging data is for a patient and has been captured at a first time and the second medical imaging data is for the patient and has been captured at a second, later, time (Muller, paragraphs 58 and 70).
As per claim 4, Muller in view of Zheng further discloses the method of claim 1, wherein the change represents occurrence, progression, regression, or disappearance of a medical abnormality (Muller, paragraphs 42, and 70).
As per claim 5, Muller in view of Zheng further discloses the method of claim 1, comprising: obtaining the first medical imaging data and the second medical imaging data, the first medical imaging data comprising a plurality of first intensity values and the second medical imaging data comprising a plurality of corresponding, second, intensity values (Muller, paragraph 41); and
for each of the plurality of first intensity values, comparing the first intensity value with the corresponding second intensity value, to obtain a differential intensity value, wherein the imaging data comprises the obtained differential intensity values (Muller, paragraph 41).
As per claim 6, Muller in view of Zheng further discloses the method of claim 5, wherein comparing the first intensity value with the second intensity value comprises performing a subtraction operation using the first intensity value and the second intensity value (Muller, paragraph 41, discloses a difference in pixel intensity).
As per claim 10, Muller, discloses a computer implemented training method of training a machine learning model for generating natural language text describing a change between first medical imaging data and second medical imaging data (Muller, Fig. 3B, Fig. 5, Fig. 11, and paragraphs 50 and 70), the method comprising:
providing the machine learning model, the machine learning model comprising:
(a) a first (machine learning) model configured to generate, based on an input of given imaging data representative of given first medical imaging data and given second medical imaging data, or of a difference between the given first medical imaging data and the given second medical imaging data, an image feature vector (Muller, Fig. 4, Fig. 5 and paragraphs 50, 57 and 58), and
(b) a second machine learning model configured to generate, based on an input of data representative of the image feature vector, natural language text (Muller, Fig. 3B, Fig. 11, and paragraphs 50 and 70, discloses the feature vectors are input into a trained machine learning classifier, wherein they are classified as benign or as malignant; please note this resulting class corresponds to the natural language text describing the change as claimed);
providing training data comprising a plurality of sets of training imaging data, each set of training imaging data representative of first training medical imaging data and second training medical imaging data, or of a difference between the first training medical imaging data and the second training medical imaging data (Muller, Fig. 3B, Fig. 11, and paragraphs 50 and 70), the training data further comprising, for each set of training imaging data, ground truth natural language text describing a change between the first training medical imaging data and the second training medical imaging data (Muller, paragraphs 32, 40 and 43-45); and
training the machine learning model based on the training data so as to minimize a loss function between the natural language text generated for the sets of training imaging data by the machine learning model and the corresponding ground truth natural language text for the sets of training imaging data (Muller, paragraphs 65 and 74).
Muller discloses generation of image feature vector utilizing a different mathematical model not generated by a first trained machine leaning model as claimed, Further Muller also discloses (b) a second machine learning model configured to generate, based on an input of data representative of the image feature vector, natural language text as being explained above, however, Muller does not explicitly disclose inputting the imaging data into a (first trained machine learning) model to generate an image feature vector; (b) a second machine learning model configured to generate, based on an input of data representative of the image feature vector, natural language text describing a change between the given first medical imaging data and the given second medical imaging data; Zheng discloses inputting the imaging data into a first trained machine learning model to generate an image feature vector (Zheng, Fig. 6:11, and related text, specifically please see pages 6-7);
(b) a second machine learning model configured to generate, based on an input of data representative of the image feature vector, natural language text describing a change between the given first medical imaging data and the given second medical imaging data (Zhang, Fig. 6:13, and pages 7-8, discloses the analysis module 13 is specifically used for the sentence pair of the first mark sentence and the second mark sentence to match the character string, determining the second mark sentence corresponding to the difference character of the first mark sentence; and determining the second medical image report relative to the modified type of the first medical image report according to the semantic meaning of the difference character);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Muller teachings by implementing a machine learning model to the system, as taught by Zheng.
The motivation would be to improve the accuracy of the subsequent comparison analysis result of the medical image report (page 5), as taught by Zheng.
As per claim 14, Muller discloses a system for generating natural language text describing a change between first medical imaging data and second medical imaging data (Muller, Fig. 3B, Fig. 5, Fig. 11, and paragraphs 50 and 70):
a non-transitory memory device for storing computer readable program code (Muller, Fig. 1:28, and paragraph 28); and
a processor in communication with the non-transitory memory device (Muller, Fig. 1:22:28), the processor being operative with the computer readable program code to perform steps (Muller, paragraph 28)
For rest of claim limitations please see the analysis of claim 1.
As per claim 15, please see the analysis of claim 2.
As per claim 16, please see the analysis of claim 3.
As per claim 17, please see the analysis of claim 4.
As per claim 18, please see the analysis of claim 5.
As per claim 19, please see the analysis of claim 6.
Allowable Subject Matter
Claims 7-9, 11-13, and 20, are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED Z HAIDER whose telephone number is (571)270-5169. The examiner can normally be reached MONDAY-FRIDAY 9-5:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SAM K Ahn can be reached at 571-272-3044. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED HAIDER/Primary Examiner, Art Unit 2633
Prosecution Insights