DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Barisoni et al. (Digital Pathology and Computational Image Analysis in Nephropathology, Nov. 2020, hereinafter “Barisoni”)
Hakala et al. (US 20220184421 A1, hereinafter “Hakala”)
Peng et al (US 20210019342 A1, hereinafter “Peng”)
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/17/2025 has been entered.
Response to Arguments
Applicant’s Arguments/Remarks filed on 12/17/2025 have been fully considered and are not persuasive.
Applicant Arguments/Remarks: Barisoni is silent to "providing the first embedding determined by the first machine learning model as input to a second machine learning model in a machine learning system, the first embedding representing the plurality of medical images, the metadata, and one or more tokens," as recited in claims, 1, 8, and 15. None of the other cited art discloses this limitation. The cited art is further silent to and does not disclose "determining, using the first machine learning model, a first embedding, wherein determining the first embedding comprises: converting the plurality of medical images and metadata into the first embedding," as recited in claim 1.
Examiner’s Response: examiner disagrees with applicant assertion above that the combination of the cited prior arts does not teach or suggest applicant invention as claimed especially as argued above by the applicant. Barisoni teaches AI models, including ML and DL, can be used for various purposes, including the automatic detection, segmentation and quantification of histological parameters and structural changes, and for disease diagnosis. These tasks are considered to be ‘low- level’ tasks. This is in contrast to ‘high- level tasks’, in which AI models are used to concurrently interrogate and integrate multiple classes of primary data — for example, histopathological image data spatially coupled to transcriptomic data — providing opportunities for the prediction of processes such as disease aggression (for example, the biological potential of a malignancy), patient outcome, organ engraftment survival and therapeutic response. Thus, use of AI has the potential to elevate digital images from their basic use as a tool for visual assessment of disease status to a much more complex and comprehensive role as a tool to facilitate prediction of disease trajectory. By contrast, supervised learning uses an annotated training dataset to learn the relationship between individual features and categorical labels. Supervised and unsupervised learning can be used to convert digital pathology images into minable data, using two general techniques: ‘hand- crafted pathomics’ (also known as conventional pathomics) and ‘discovery pathomics with DL’. The Received mages and annotations (metadata) are used in a first model and information garnered are also added to generate embedded data. The embedded data is provided to a second model for prognostication and selection of targeted treatment for the subject.
Although, Barisoni teaches using pass data information or data from built data based corresponding to the subject or subject diseases. It is not clear if the metadata is historical data from the subject itself.
However, Hakala teaches receiving calibration characteristic at a machine learning 408 and generates treatment attribute prediction. The treatment attribute prediction is provided to another machine learning model 412. The machine leaning model 412 receive the treatment attributed and treatment attribute labels and generates calibration parameter for a prediction adjuster and the prediction adjuster uses the calibration parameter and treatment attribute prediction generated by the machine learning 408 to generate adjusted/calibrated treatment attribute prediction (confidence score). The score adjuster is a neural network. The neural network may adjust the confidence score for the new treatment attribute according to the same calibration parameter that was used to calibrate the confidence score for the first patient (Hakala [0075], [0076], [0085], [0096]). Peng further show that additional information such as token can be added to the data such that missing information is added accordingly (Peng [0012], [0017]-[0022], [0044], [0063], [0100], figs. 7 and 10-13).
Therefore, one of ordinary skill in the art before the effective filing date of the application can clearly see that the combination of the cited prior art teaches creating an embedded data using a fist machine leaning model, adding information that is missing to the embedded data and provides the embedded data to another machine learning model such that better treatment and/or better prediction of a particular treatment can be more accurately obtained.
Applicant tries to argue the cited references separately when the rejection is based on the combination of the references. Applicant is reminded that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Using a machine learning for embedding image information and add information to the embedded data where possible and providing the embedded data to another machine learning model for further processing.
Therefore, examiner maintains that claims 1, 4-8, 11-15, 18-21 are not allowable over the cited prior arts above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4-8, 11-15, 18-21-20 are rejected under 35 U.S.C. 103 as being unpatentable over Barisoni et al. (Digital Pathology and Computational Image Analysis in Nephropathology, Nov. 2020, hereinafter “Barisoni”) in view of Hakala et al. (US 20220184421 A1, hereinafter “Hakala”) and in view of Peng et al (US 20210019342 A1, hereinafter “Peng”).
Regarding claims 1, 8 and 15:
Barisoni teaches a computer-implemented method for processing digital pathology images to determine a treatment for one or more subjects (Barisoni Abstract, where Barisoni teaches integrated, biologically and clinically homogeneous disease categories, to identify patients at risk of progression, and shifting current paradigms for the treatment and prevention of kidney diseases), comprising:
receiving a plurality of medical images of at least one pathology specimen, the pathology specimen being associated with a subject (Barisoni Abstract, Key points page 670, where Barisoni teaches The introduction of digital pathology in clinical research, trials and practice has catalyzed the development of novel machine- learning models for tissue interrogation with the potential to improve our ability to discover disease mechanisms, identify comprehensive, patient- specific phenotypes, classify kidney patients into clinically relevant categories, predict disease outcome and, ultimately, identify more targeted therapies);
receiving metadata corresponding to the plurality of medical images, the metadata comprising data regarding previous medical treatment of the subject (Barisoni page 678, fig. 2, machine vision technology as supported in nephropathology, The common goals of these consortia are to better understand the pathogenesis of kidney diseases and to improve their classification and treatment through comprehensive analysis of clinical, morphological and molecular data — a process that has required the establishment of digital pathology repositories (DPRs) for the banking and organization of digital images from renal biopsies and their associated metadata);
providing the first embedding determine by the first machine learning model as input to a second machine learning model in the a machine learning system, the first embedding represent the plurality of medical images, the metadata and one or more tokens the machine learning system having been trained by receiving as input historical treatment information and digital images labeled with a predicted treatment regimen (Barisoni pages 677, 678 and 682, fig. 3 last paragraph, supervised learning uses an annotated training dataset to learn the relationship between individual features and categorical labels. Supervised and unsupervised learning can be used to convert digital pathology images into minable data, using two general techniques: ‘hand- crafted pathomics’ (also known as conventional pathomics) and ‘discovery pathomics with DL’…. The nephropathology digital ecosystem, fig. 3, For example, qualitative and quantitative automatic detection of features of acute tubular injury may predict the course of the disease and response to therapy: the presence of only a few areas of vacuolization with specific qualitative characteristics could predict rapid recovery from an episode of acute renal failure, with normalization of serum creatinine levels, compared with a renal biopsy containing much greater levels of vacuolization; and
outputting, by the machine learning system, a treatment effectiveness assessment (Barisoni pages 678 and 682 The nephropathology digital ecosystem, fig. 3, Finally, in the actionable intelligence phase, the knowledge obtained from the digital images (input data) is integrated with other data types, for example, omics and clinical data, and used to diagnose, prognosticate and select targeted treatments (output data) for the patient.
Barisoni fails to explicitly teach where the machine learning system having been trained by receiving as input historical treatment information and digital images labeled with a predicted treatment regimen.
providing the plurality of medical images and metadata to a trained embedding system including a first machine learning model; determining, using the first machine learning model, a first embedding, wherein determining the first embedding comprises:
converting the plurality of medical images and metadata into the first embedding; identifying one or more missing data points in the first embedding; and replacing the one or more missing data points with one or more tokens for processing;
providing the plurality of medical images, metadata, and first embedding as input to a second machine learning model in a machine learning system,
However, Barisoni teaches the common goals of these consortia are to better understand the pathogenesis of kidney diseases and to improve their classification and treatment through comprehensive analysis of clinical, morphological and molecular data — a process that has required the establishment of digital pathology repositories (DPRs) for the banking and organization of digital images from renal biopsies and their associated metadata. The analytical tools of AI and ML will reveal new opportunities with which to develop large- scale, kidney- centric data ecosystems, to combine datatypes and datasets for discovery and validation, and for the practice of precision nephrology. Machine vision can also be used to build models to aid prognostication. For example, qualitative and quantitative automatic detection of features of acute tubular injury may predict the course of the disease and response to therapy: the presence of only a few areas of vacuolization with specific qualitative characteristics could predict rapid recovery from an episode of acute renal failure, with normalization of serum creatinine levels, compared with a renal biopsy containing much greater levels of vacuolization (Barisoni page 678, fig. 2, machine vision technology as supported in nephropathology, d) prediction ). Hakala teaches a network calibration for radiography wherein a machine learning for adjusting score for prediction of treatments receives historical data and treatment attributes labels in the form of image. Upon determining the calibration parameters, the end-user device 140 may feed patient data for new patients into the machine learning model and use the calibration parameters to calibrate the output confidence scores for different treatment attributes (Hakala [0007]-[008], [0036], [0038]-[0039],[0087]-[0090], [0094], figs.2 and 4, in other words the first embedding determine that new patient data is missing and therefore using new patient data to update or add new information to the embedding). Peng further teaches a method for generating semantic labels which are used for similarity ranking among images in the reference library. The images 702 are fed to a second machine learning model 704, e.g., a deep convolutional neural network pattern recognizer which generates semantic labels for the images, such as healthy, benign, or assigns a tissue type label. The system includes a module configured to perform at least one of the following operations on the additional portions of the larger medical image that it found:
a) highlight the additional portions, e.g., by shading in a particular color or showing a bounding box or border around the additional portion;
b) provide annotations for the additional portions, such as probability cancerous or having a particular Gleason score; or
c) providing quantifications for the additional portions, such as size data.
(Peng [0012], [0017]-[0022], [0044], [0063], [0100], figs. 7 and 10-13). In other words, Peng teaches obtaining a first embedding wherein the first embedding includes information on a portion of an image then add additional missing information on the image or the embedding such that the missing information is included in the image ).
Therefore, taking the teachings of Barisoni, Hakala and Peng as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to provide not only historical data but also image with corresponding label regarding expected or predicted treatment and also add information wherever missing or when new information is available in order to properly train the model to be able to more accurately determine the best treatment possible corresponding a particular ailment or disease and to update and/or replace the treatment when new information are available.
Regarding claims 4, 11 and 18:
Barisoni in view of Hakala and in view of Peng teaches wherein the trained system outputs the plurality of medical images with marking to display the predicted effects of the treatment effectiveness assessment (Barisoni page 678, fig. 2, d) prediction, where Barisoni teaches Machine vision can also be used to build models to aid prognostication. For example, qualitative and quantitative automatic detection of features of acute tubular injury may predict the course of the disease and response to therapy: the presence of only a few areas of vacuolization with specific qualitative characteristics could predict rapid recovery from an episode of acute renal failure, with normalization of serum creatinine levels, compared with a renal biopsy containing much greater levels of vacuolization; Peng [0012], [0044], [0063], [0100], figs. 7 and 10-13).
Regarding claims 5, 12 and 19:
Barisoni in view of Hakala and in view of Peng teaches wherein the metadata may further include information describing a tissue type of the pathology specimen for the medical specimen (Barisoni page 673 and 675 left column, last paragraph; Hakala [0002], [0015]; Peng [0012], [0044], [0063], [0100], figs. 7 and 10-13).
Regarding claims 6, 13 and 20:
Barisoni in view of Hakala and in view of Peng teaches further comprising:
inputting the received medical images that correspond to previously treated medical specimen into a second trained system; and
determining a score to measure the effectiveness of past treatment, wherein the score defines a damage of previously healthy slides and additional damage to previously cancerous regions of the inputted slides (Barisoni page 373; Hakala Abstract [0007]-[0008], [0064]-[0067]; Peng [0012], [0044], [0063], [0100], figs. 7 and 10-13).
Regarding claims 7, 14 and 21:
Barisoni in view of Hakala and in view of Peng teaches wherein the treatment effectiveness assessment comprises a treatment type and a treatment dosage for the patient (Hakala [0024], [0048], [0065]; Peng [0012], [0044], [0063], [0100], figs. 7 and 10-13).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 February 6, 2026