DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Specifically, claim 1 recites “the first dataset” and “the second dataset”. However, there is insufficient antecedent basis for these limitations in the claims. The Examiner notes that these two datasets may be intended to refer to the previously introduced “first image data set” and “second image data set”; if so, appropriate correction should be made to resolve ambiguity. Claims 11 and 17 contain similar defects as claim 1. Claims 2-10, 12-16, and 18-20 are contain the same defect as they ultimately depend on claims 1, 11, or 17. Claims 1-20 are therefore rejected under 35 U.S.C. 112(b) as indefinite. In the furtherance of compact prosecution, an attempt will be made to interpret the claims in their current form for the purposes of prior art rejection.
Additionally, claim 1 recites “wherein the second dataset has a value of a presence of the medical condition in the metadata label.” However, it is noted that the recited second image data set includes clinical labels (see lines 4-5), not metadata labels. Claims 11 and 17 contain similar defects as claim 1. Claims 2-10, 12-16, and 18-20 are contain the same defect as they ultimately depend on claims 1, 11, or 17. Claims 1-20 are therefore rejected under 35 U.S.C. 112(b) as indefinite. In the furtherance of compact prosecution, an attempt will be made to interpret the claims in their current form for the purposes of prior art rejection.
Additionally, claim 3 recites “the second AI model”. There is insufficient antecedent basis for this limitation in the claim. Claim 3 is therefore separately rejected as indefinite under 35 U.S.C. 112(b).
Additionally, claim 7 recites “the biomarker data”. There is insufficient antecedent basis for this limitation in the claim. Claim 7 is therefore separately rejected as indefinite under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-5, 7, 11, 13, 15, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sowrirajan (“MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models”, copy provided, see PTO-892) in view of Neumann (U.S.P.G. Pub. No. 2020/0356864).
Regarding claim 1, Sowrirajan discloses:
A method for asymmetric training an Al model, the method comprising:
receiving a multi-modal dataset including a first image data set and a second image data set (page 3, two datasets are used- ImageNet and CheXpert), wherein the first image data set includes a metadata label as a medical condition in an electronic medical record of a patient (page 2, ImageNet-labelled X-rays), wherein the second image data set includes clinical labels (page 3, CheXpert images are labeled for the presence or absence of several diseases)
performing training of an Al model using the first dataset using the metadata labels to adjust first weights in the Al model (Figure 2, page 4, the models are initialized with ImageNet pre-trained weights); and
performing contrastive learning of the Al model using the second dataset (page 4, momentum contrast (MoCo) training is performed using the CheXpert data set), wherein the Al model includes a first portion having the first weights (page 5, backbone model) and a second portion having second weights (page 5, linear classifier on top of the backbone model), wherein the contrastive learning held constant the first weights of the Al model (page 5, the backbone model is frozen) and adjusted the second weights of the Al model via a contrastive loss function using the clinical labels (Figure 2, pages 4-5, MoCo pretraining is used on ImageNet-initialized models; page 5, the MoCo trained weights are applied to the linear classifier)
Sowrirajan does not explicitly disclose:
wherein the second dataset has a value of a presence of the medical condition in the metadata label
Neumann (U.S.P.G. Pub. No. 2020/0356864) discloses:
wherein the second image data set includes clinical labels and wherein the second dataset has a value of a presence of the medical condition in the metadata label (paragraph [0055], second training set includes prognostic label; paragraph [0042], prognostic label includes a disease/disorder)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Neumann with the system of Sowrirajan such that the second image data set would have included clinical labels and wherein the second dataset has a value of a presence of the medical condition in the label as described in Neumann. The suggestion/motivation would have been in order to implement a system capable of “organiz[ing the dataset] according to…association with a list of significant conditions” (paragraph [0054] of the Neumann reference) such that “an unsupervised machine-learning process may be free to discover [the] relationships [of the label] provided in the data” (paragraph [0068] of the Neumann reference)
Regarding claim 3, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 1).
Neumann additionally discloses:
outputting, via a report or display, classifier output of the second Al model, wherein the classifier output is used for diagnosis of a disease or a medical condition (paragraph [0106], the associations can be output the user in narrative language to the display)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Neumann with the system of Sowrirajan such that the system would output, via a report or display, classifier output of the second Al model, wherein the classifier output is used for diagnosis of a disease or a medical condition as described in Neumann. The suggestion/motivation would have been in order to implement a system capable of “organiz[ing the dataset] according to…association with a list of significant conditions” (paragraph [0054] of the Neumann reference) such that “an unsupervised machine-learning process may be free to discover [the] relationships [of the label] provided in the data” (paragraph [0068] of the Neumann reference) and inform the user thereof.
Regarding claim 4, Sowrirajan additionally discloses:
wherein the first data set comprises image data from a medical scan (page 2, Imagenet chest X-ray images)
Regarding claim 5, Sowrirajan additionally discloses:
wherein the first data set comprises image data from a sensor (page 2, Imagenet chest X-ray digital images imply an image sensing element to produce a digital image from the X-ray)
Regarding claim 7, Sowrirajan additionally discloses
wherein the second portion of the Al model comprises a linear layer appended to the first portion (page 5, linear classifier trained on top of the backbone model)
Regarding claim 11, arguments analogous to claim 1 are applicable. The processor and memory having instructions to perform the disclosed methods is taught by Section 3.1 “Materials” (page 9) of Sowrirajan.
Regarding claim 13, arguments analogous to claim 5 are applicable. The processor and memory having instructions to perform the disclosed methods is taught by Section 3.1 “Materials” (page 9) of Sowrirajan.
Regarding claim 15, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 11).
wherein the second portion of the Al model comprises at least one of (i) a linear layer appended to the first portion (page 5, linear classifier trained on top of the backbone model) or (ii) a semantic segmentation head appended to the first portion.
Regarding claim 17, arguments analogous to claims 1 and 11 are applicable. The computer readable medium is inherently taught as evidenced by Section 3.1 “Materials” (page 9) of Sowrirajan which describes computing components executing instructions to perform the disclosed methods.
Regarding claim 19, arguments analogous to claims 7 and 15 are applicable. The computer readable medium is inherently taught as evidenced by Section 3.1 “Materials” (page 9) of Sowrirajan which describes computing components executing instructions to perform the disclosed methods.
Claim(s) 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Sowrirajan in view of Neumann, in further view of Krizhevsky (“ImageNet Classification with Deep Convolutional Neural Networks”, copy provided, see PTO-892).
Regarding claim 6, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 1).
The combination of Sowrirajan and Neumann does not explicitly disclose:
wherein the first portion of the Al model comprises an auto- encoder
Krizhevsky discloses:
wherein the first portion of the Al model comprises an auto-encoder (page 90, an autoencoder can be trained to compress vectors to short binary codes)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Krizhevsky with the combination of Sowrirajan and Neumann such that the first portion of the Al model would comprise an auto-encoder as described in Krizhevsky. The suggestion/motivation would have been in order to avoid “using Euclidean distance…[which] is inefficient” and instead make it “efficient by training an autoencoder to compress…vectors to short binary codes” (page 90 of the Krizhevsky reference).
Regarding claim 14, arguments analogous to claim 6 are applicable. The processor and memory having instructions to perform the disclosed methods is taught by Section 3.1 “Materials” (page 9) of Sowrirajan.
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Sowrirajan in view of Neumann, in further view of Rajaraman (“Improved Semantic Segmentation of Tuberculosis—Consistent Findings in Chest X-rays Using Augmented Training of Modality-Specific U-Net Models with Weak Localizations”, copy provided, see PTO-892).
Regarding claim 8, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 1).
The combination of Sowrirajan and Neumann does not explicitly disclose:
wherein the second portion of the Al model comprises a semantic segmentation head appended to the first portion.
Rajaraman discloses:
wherein the second portion of the Al model comprises a semantic segmentation head appended to the first portion (page 4, a segmenting U-net can be appended to an ImageNet pretrained model including DenseNet-121 or ResNet-18)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Rajaraman with the combination of Sowrirajan and Neumann such that the second portion of the Al model comprises a semantic segmentation head appended to the first portion as described in Rajaraman. The suggestion/motivation would have been in order to extend the utility of the system by incorporating “the principal segmentation model architecture to be used for natural and medical segmentation tasks” (page 4 of the Rajaraman reference).
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sowrirajan in view of Neumann, in further view of Yu (“Hyper-reflective foci segmentation in SD-OCT retinal images with diabetic retinopathy using deep convolutional neural networks”, copy provided, see PTO-892).
Regarding claim 9, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 7).
The combination of Sowrirajan and Neumann does not explicitly disclose:
wherein the biomarker data includes at least one of: Intraretinal Fluid (IRF), Diabetic Macular Edema (DME), and Intra-Retinal Hyper- Reflective Foci (IRHRF)
Yu discloses:
wherein the biomarker data includes at least one of: Intraretinal Fluid (IRF), Diabetic Macular Edema (DME), and Intra-Retinal Hyper- Reflective Foci (IRHRF) (pages 4503-4504, HRF segmentation)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Yu with the combination of Sowrirajan and Neumann such that the biomarker data included Intra-Retinal Hyper- Reflective Foci (IRHRF) as described in Yu. The suggestion/motivation would have been in order to implement a system capable of “automatically and accurately segmenting hyper-reflective foci (HRF)” (page 4502, “Purpose” of the Yu reference).
Claim(s) 10, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sowrirajan in view of Neumann, in further view of Autio (“On the neural network classification of medical data and an endeavour to balance non-uniform data sets with artificial data extension”, copy provided, see PTO-892).
Regarding claim 10, the combination of Sowrirajan and Neumann discloses the method of the parent claim (claim 1).
The combination of Sowrirajan and Neumann does not explicitly disclose:
wherein the training operation is configured to:
compute a distribution of unique identifier for subjects throughout an unlabeled data set; and sample for the training operation based on the computed distribution.
Autio discloses:
wherein the training operation is configured to:
compute a distribution of unique identifier for subjects throughout an unlabeled data set (Abstract and pages 388-390, a distribution of the data is calculated); and
sample for the training operation based on the computed distribution (Abstract and pages 388, 391, 394, to avoid imbalance between classes, sampling is done according to a distribution between classes)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the system of Autio with the combination of Sowrirajan and Neumann such that the system would have been configured to compute a distribution of unique identifier for subjects throughout an unlabeled data set as described. The suggestion/motivation would have been in order to implement a system capable of avoiding “badly disrupt[ing] a machine learning task, resulting in unsuccessful distinguishing of classes” (page 388 of the Autio reference).
Regarding claim 16, arguments analogous to claim 10 are applicable. The processor and memory having instructions to perform the disclosed methods is taught by Section 3.1 “Materials” (page 9) of Sowrirajan.
Regarding claim 20, arguments analogous to claim 10 are applicable. The computer readable medium is inherently taught as evidenced by Section 3.1 “Materials” (page 9) of Sowrirajan which describes computing components executing instructions to perform the disclosed methods.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN R WALLACE whose telephone number is (571)270-1577. The examiner can normally be reached Monday-Friday from 8:30-5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN R WALLACE/ Primary Examiner, Art Unit 2682