Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The response received on 5/18/2026 has been placed in the file and was considered by the examiner. An action on the merit follows.
Response to Amendment
The amendments filed on 2026 May 18 have been fully considered. Response to these amendments is provided below.
Summary of Amendment/ Arguments and Examiner’s Response:
The applicant has amended the claims and has argued that the prior art does not teach the claimed limitations.
All arguments are moot in view of new grounds of rejection, below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 15 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 15 recites the limitation “the first output” in lines 5-6. It is unclear as to which first output the applicant is referring to, because the applicant previously claims “first outputs”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5-9, 16-18 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent Application Publication NO. 2019018848 (Madani et al) in view of U.S. Patent No. 20220156992 (Harikumar et al).
Regarding claim 18, Madani et al discloses a method (fig. 5, fig. 1) of image processing apparatus (fig. 4), comprising: receiving medical image data (fig. 5, item 510, “receive medical image”, fig. 1, item 120) and providing the medical image data as an input to an input layer of a main trained machine learning model, (fig. 1, item 120 input to item 160, fig. 5, item 540), wherein the main trained machine learning model comprises the input layer, i.e. the input layer of fig. 1, item 160, an output layer (fig. 1, item 195), and a plurality of layers between the input layer and the output layer, the layers between the CNN of 160 of fig. 1, and the output layer of fig. 1, item 195; receiving text data including clinical information relating to the medical image data (fig. 5, item 510, “receive…associated medical report”); applying an auxiliary trained machine learning model to the text data to produce a model of the text (fig. 1, item 170, 180, fig. 5, item 530) that represents a target processing region based on the text data (fig. 1, item 130, 150 describes target processing regions); applying the model of the text to a feature result produced at at least one of the plurality of layers between the input layer and the output layer of the main trained machine learning model (fig. 1, items 160 180 are concatenated and processed in item 190, fig. 5, item 550,560); and outputting, at the output layer of the main trained machine learning model, a segmentation of a specified anatomical feature and/or pathology (fig. 1, item 195 outputs item 199, fig. 5, item 580-590).
Madani et al does not disclose expressly a model of the text is an attention function comprising a spatially dependent filter that represents the target processing region, and applying the spatially dependent filter to a feature map that is the feature result from the image (corresponding to the descriptor of Madani, fig. 1, item 160).
Harikumar et al discloses an attention function comprising a spatially dependent filter that represents a target processing region, i.e. the attention function/ text encoding output by fig. 2, item 210 and representing a target region (sunflower), and applying the spatially dependent filter (fig. 2, applying 210 in convolution in fig. 2, item 212) to a feature map that is the feature result from the image (fig. 2, item 204 is features mapped of fig. 2, item 102).
Madani et al and Harikumar et al are combinable because they are from the same field of endeavor, i.e. text and image processing.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use the text as a filter.
The suggestion/motivation for doing so would have been to provide a more robust system by providing a convenient way to combine and transform features in a spatially aware way.
Therefore, it would have been obvious to combine the method of Madani et al with filtering of Harikumar et al to obtain the invention as specified in claim 18.
Claim 1 is rejected for the same reasons as claim 18. Thus, the arguments analogous to that presented above for claim 18 are equally applicable to claim 1. Claim 1 distinguishes from claim 18 only in that claim 1 is a medical image processing apparatus with processing circuitry configured to carry out a broader version of claim 1, in which a space derived from the medical image data could be specified instead of the target processing region. Because all the required limitations are addressed above, and because Madani et al discloses an apparatus comprising processing circuitry (fig. 4), prior art applies.
Regarding claim 3, Madani et al discloses the main trained machine learning model is configured to include or receive conditioning information, i.e. the input of LTSM fig. 1, item 180, or the weights of page 6, paragraph 51, to condition first outputs of the main trained machine learning model such that the first outputs depend on both the medical image data and the conditioning information (fig. 1, item 190 outputs depend on fig. 1, item 120 and weights of MLP, page 5, paragraph 51, and additionally on the LSTM data of fig. 1, item 180).
Regarding claim 5, Madani et al discloses the auxiliary trained machine learning model comprises a text encoder network that is configured to generate embeddings for tokens that represent text of the text data (fig. 1, item 170), and to combine the embeddings into a vector, the vector formed by fig. 1, item 180, that is or forms part of the conditioning information for the main machine learning model, i.e. conditioning of fig. 1, item 180).
Regarding claim 6, Madani et al discloses the processing circuitry is further configured to apply an additional conditioning process so as to ensure or encourage that spatial information in the text data is taken into account by the main machine learning model, i.e. the spatial data of the character and word recognition of text carried out in fig. 1, item 140.
Regarding claim 7, Madani et al discloses the additional conditioning process is such as to alter an attention vector representing spatial attention that is input to or included in the main trained machine learning model, i.e. weights of the MLP (page 6, paragraph 51) which provide the amount of attention required for each feature and thus altering the attention vector by multiplying it by the weight and representing the spatial attention/ importance of the vector (page 6, paragraph 51).
Regarding claim 8, Madani et al discloses the additional conditioning process is such as to condition the main trained machine learning model with a representation of a shape or other property of a pathology or other feature of interest, because the weights are learned with ground truth images and their annotated features of interest (page 6, paragraph 51).
Regarding claim 9, Madani et al discloses the additional conditioning process is such as to condition the main trained machine learning model using a signed distance field or other loss function (Page 4, paragraph 32).
Regarding claim 16, Madani et al discloses the text data comprise at least one radiology report and/or clinician notes or other user notes (fig. 1, item 130).
Regarding claim 17, Madani et al discloses the image data comprises at least one of magnetic resonance imaging (MRI) data, CT (computed tomography) data, cone-beam CT data, X-ray data (fig. 1, item 120), ultrasound data, positron emission tomography (PET) data or single photon emission computed tomography (SPECT) data.
Claims 10 and 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over Madani et al in view of Harikumar et al, as applied to claim 3 above, and further in view of U.S. Patent Application Publication No. 20210279863 (Jackenkow et al).
Regarding claim 10, Madani et al (as modified by Harikumar et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Madani et al further discloses training of the auxiliary trained machine learning model (page 4, paragraph 32).
Madani et al (as modified by Harikumar et al) does not disclose expressly training comprises applying a penalty to a loss function in order to train against structured data.
Jackenkow et al discloses training comprises applying a penalty to a loss function in order to train against structured data (page 6, paragraph 118).
Madani et al (as modified by Harikumar et al) & Jackenkow et al are combinable because they are from the same field of endeavor, i.e. training neural networks.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply a penalty in a loss function.
The suggestion/motivation for doing so would have been provide a more robust system by encouraging learning an informative function..
Therefore, it would have been obvious to combine the apparatus of Madani et al (as modified by Harikumar et al) with the learning with loss function penalty of Jackenkow et al to obtain the invention as specified in claim 10.
Regarding claim 11, Madani discloses training of the main trained machine learning model (page 4, paragraph 34) comprises applying a penalty to a loss function in order to train against structured data. Jackenkow et al discloses training comprises applying a penalty to a loss function in order to train against structured data (page 6, paragraph 118).
Claims 12 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Madani et al in view of Harikumar et al, as applied to claim 3 above, and further in view of U.S. Patent Application Publication No. 20090310836 (Krishnan et al).
Regarding claim 12, Madani et al (as modified by Harikumar et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Madani et al discloses the machine learning model is trained using training data (page 6, paragraph 53)
Madani et al (as modified by Harikumar et al) does not disclose expressly the training data includes counterfactual training data.
Krishnan et al discloses the training data includes counterfactual training data, negative training data (Page 4, paragraph 42).
Madani et al (as modified by Harikumar et al) & Krishnan et al are combinable because they are from the same field of endeavor, i.e. medical image processing.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use negative training data.
The suggestion/motivation for doing so would have been to provide a more accurate recognition.
Therefore, it would have been obvious to combine the apparatus of Madani et al (as modified by Harikumar et al) with the negative training data of Krishnan et al to obtain the invention as specified in claim 12.
Regarding claim 13, Krishnan et al discloses the training data comprises at least some training data that represent counterfactual examples (page 4, paragraph 42) or other property that encourage the main machine learning model to use or be more influenced by the conditioning information and/or the output of an auxiliary trained machine learning model.
Claims 14 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Madani et al in view of Harikumar et l, as applied to claims 3 and 5 above and further ran view of U.S. Patent Application Publication No. 20210166807 (Quennesson et al).
Regarding claim 14, Madani et al (as modified by Harikumar et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Madani et al further discloses the processing circuitry is configured to receive user input (page 7, paragraph 58).
Madani et al (as modified by Harikumar et al) does not disclose expressly alter at least one of spatial attention mechanisms of the main machine learning model in dependence on the user input and/or augment or otherwise modify the text data based on the user input.
Quennesson et al discloses otherwise modifying the text data, corresponding to the medical report, based on the user input (pages 12-13, paragraph 159).
Madani et al (as modified by Harikumar et al) & Quennesson et al are combinable because they are from the same field of endeavor, i.e. medical image and report processing.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the text data.
The suggestion/motivation for doing so would have been to provide a more robust system by allowing a user to correct incorrect data.
Therefore, it would have been obvious to combine the apparatus of Madani et al (as modified by Harikumar et al) with the modifying of text data of Quennesson et al to obtain the invention as specified in claim 14.
Regarding claim 15, Madani et al discloses the processing circuitry is configured to receive user input (page 7, paragraph 58). Quennesson et al discloses use the user input in a feedback loop (page 13, paragraph 163) that comprises modifying at least one of input to the auxiliary machine learning model, since the correction is reprocessed in the NLP module (page 13, paragraph 163), and recalculating the output of the main machine learning model by retraining the model for verification (page 13, paragraph 153).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kathleen Yuan Dulaney whose telephone number is (571)272-2902. The examiner can normally be reached M-F: 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at 5712703717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN Y DULANEY/Primary Examiner, Art Unit 2666 6/12/2026
Prosecution Insights