DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-11, 13-14, 16, 24, 27 and 27 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Automated Generation of Accurate & Fluent Medical X-Ray Reports” by Nguyen et al. (hereinafter ‘Nguyen’).
In regards to claim 1, Nguyen teaches a method comprising, by one or more computing systems: accessing a plurality of radiographic images of an animal, wherein one or more first radiographic images of the plurality of radiographic images depict the animal from one or more views, respectively, and wherein one or more second radiographic images of the plurality of radiographic images depict one or more body parts of the animal, respectively; (See Nguyen Section 4.1.1, Nguyen teaches a dataset of multi-view x-ray images.)
determining one or more disease classifications associated with the animal based on analyzing the plurality of radiographic images by a machine learning model; (See Nguyen Figure 1 and Section 3.1.3, Nguyen teaches disease classification using a multi-view image encoder.)
generating, based on the machine learning model, a diagnostic report associated with the animal, wherein the diagnostic report comprises the one or more disease classifications and a natural-language textual radiology report; and sending, to a user device, instructions for presenting the diagnostic report. (See Nguyen Figure 1 and Section 3.2, Nguyen teaches using a transformer encoder to create a diagnostic report.)
In regards to claim 2, Nguyen teaches wherein each of the plurality of radiographic images is formatted as a Digital Imaging and Communications in Medicine ("DICOM") image. (See Nguyen Section 4.1.1)
In regards to claim 3, Nguyen teaches wherein the machine learning model is based on at least one first neural network and at least one second neural network, the at least one first neural network and the at least one second neural network being coupled with each other. (See Nguyen Figure 1).
In regards to claim 4, Nguyen teaches wherein generating the diagnostic report comprises: accessing a plurality of reference reports; encoding the plurality of reference reports into a feature space; encoding the plurality of radiographic images into the feature space; and determining the diagnostic report based on similarity search in the feature space. (See Nguyen Figure 2 and Sections 3.2, 3.3 and 4.1.2, Nguyen teaches reports database that is used along with enriched disease embedding in generating diagnostic reports. )
In regards to claim 5, Nguyen teaches wherein one of the one or more disease classifications indicates an abnormal tissue. (See Nguyen section 3.1.3).
In regards to claim 6, Nguyen teaches further comprising: identifying the abnormal tissue as at least one of cardiovascular, pulmonary structure, mediastinal structure, pleural space, or extra thoracic. (See Nguyen Figure 1.)
In regards to claim 7, Nguyen teaches further comprising: accessing a plurality of training radiographic images, wherein the plurality of training radiographic images are associated with a plurality of training radiology reports, respectively; and training the machine learning model based on the accessed training radiograph images and their respective training radiology reports. (See Nguyen Section 4.1.1, Nguyen teaches a dataset of images and corresponding medical reports.)
In regards to claim 8, Nguyen teaches further comprising: preprocessing each of plurality of training radiographic images, wherein the preprocessing comprises one or more of padding, random augmentation, random flip, Gaussian blur, or normalization. (See Nguyen Section 4.1.2, Nguyen teaches applying preprocessing steps to the dataset.)
In regards to claim 9, Nguyen teaches further comprising: applying long document encoding to each of the plurality of training radiology reports. (See Nguyen Figure 1, Nguyen teaches a text encoder.)
In regards to claim 10, Nguyen teaches further comprising: preprocessing each of plurality of training radiology reports, wherein the preprocessing comprises one or more of tokenization, padding, adding a classification token, or applying an attention mask. (See Nguyen Section 3.2, Nguyen teaches self-attention.)
In regards to claim 11, Nguyen teaches wherein the machine learning model comprises an image encoder, a multi-image encoder, a text decoder, and a multimodal decoder. (See Nguyen Figure 1).
In regards to claim 13, Nguyen teaches wherein the diagnostic report further comprises one or more of the plurality of radiologic images. (See Nguyen Section 3.2 and 3.3).
Claims 14, 16 and 24 recite limitations that are similar to that of claims 1, 3 and 11, respectively. Therefore, claims 14, 16 and 24 are rejected similarly as claims 1, 3 and 11, respectively.
Claims 27 and 37 recite limitations that are similar to that of claims 1 and 3, respectively. Therefore, claims 27 and 27 are rejected similarly as claims 1 and 3, respectively.
Allowable Subject Matter
Claims 12, 25 and 38 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
In regards to claims 12, 25 and 28, the applied art does not teach or suggest “further comprising: generating, by the image encoder, a feature map based on the plurality of radiologic images; generating, by the multi-image encoder based on the feature map, one or more multi- image keys and values; and generating, by the multimodal decoder based on the one or more multi-image keys and values and a start of sentence token, the natural-language textual radiology report.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UTPAL D SHAH whose telephone number is (571)272-5729. The examiner can normally be reached M-F: 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UTPAL D SHAH/Primary Examiner, Art Unit 2668
Prosecution Insights