DETAILED ACTION
Notice of AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
2. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding claims 1-20 under the broadest reasonable interpretation, the terms of the claims are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skilled in the art. See MPEP 2111.
The claims are directed a process that can be mental performed with aiding of simple tools by annotating images to produce feature maps and decide severity of retinal tear by comparing the feature maps to predetermined standard. The claims do not fall within at least one of the four categories of patent eligible subject matter because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea).
The claims do not have any limitations that are indicative of how to integrate judicial exception into a practical application such as improvements to functioning of a computer or a technical field, using any particular machine, effect a transformation of a particular article to a different state or thing, or apply the judicial exception in any meaningful way beyond generally linking the use to a particular technological environment. Therefore, the claims as a whole do not amount to significantly more than the judicial exception.
A patent may be obtained for “any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof," 35 U.S.C. § 101, but “laws of nature, natural phenomena, and abstract ideas are not patentable.”
Step 1
Claims 1 is directed to one of the four statutory categories of eligible subject matter (process): thus, the claim pass Step 1 of the Subject Matter Eligibility Test.
Step 2A, prong 1 analysis
Claims 1, 2, and 13 direct to three stage multimodal ophthalmic machine learning models for retinal tear detection and obtaining severity score. Images of different modalities are fed into corresponding input ML model and turned into feature maps which are fed into an intermediate ML mode that fuses model information into a final feature map. The final feature map is fed to an output ML model trained on ophthalmic pathology for identifying retinal tear and obtaining a severity score.
A trained physician looking at an image of an eye from each modality and annotate the image to produce a masked feature map associated with each modality. The physician further compare the annotated feature maps from two different modalities and mark common features to produce a final feature map. Then compare the final feature map with predetermined samples to decide severity of retinal tear.
Dependent claims 3-12 and 14-20 define sources of images and types of ML models applied as a general computer for processing the images.
Step 2A, prong 2 analysis
Other than defining modality and listing example of different ML models, the claims do not have any limitations that are indicative of integration of the judicial exception into a practical application such as improvements to functioning of a computer or a technical field, using any particular machine, effect a transformation of a particular article to a different state or thing, or how to apply the judicial exception in any meaningful way beyond generally linking the use to a particular technological environment.
Step 2B
Further, the claims do not include other additional elements that are beyond what is well-understood, routine, conventional activities in the field and sufficient to amount to significantly more than the judicial exception.
Conclusion:
The claims do not include additional elements amount to significantly more in terms of improving functionalities of a computer/device itself, improving another technology or technical field, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine claim to a particular useful application or by use of a particular machine that is unconventional. In conclusion, the claims 1-20 do not comply with the current standards for patent eligible subject matter under 35 USC § 101.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
41066.. Claims 1-3, 6, 8, 10-13, 16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kihara et al (Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and Optical Coherence Tomography Imaging, July/2022) and in further view of Kecskemethy et al (US Pub: 2019/0197366) and Mulyukov et al (WO Pub: 2021/220138).
Regarding claim 1, Kihara et al teaches: A system comprising: one or more processing devices and one or more memory devices coupled to the one or more processing devices, the one or more memory devices storing executable code that, when executed by the one or more processing devices, causes the one or more processing devices to: for each imaging modality of a plurality of imaging modalities: process one or more images according to each imaging modality using an input machine learning model of a plurality of input machine learning models corresponding to each imaging modality to obtain an input feature map, the one or more images being images of an eye of a patient [page 3: p03, page 4: p04, page 5: p02]; process the input feature maps for the plurality of imaging modalities using an intermediate machine learning model to obtain a final feature map; and process the final feature map using one or more output machine learning models to obtain one or more estimated representations of a pathology of the eye of the patient [page 5: p02]; the one or more estimated representations of the pathology of the eye of the patient comprising a severity score for the diagnosis [page 2: methods].
Kihara et al combines intermediate model and output model to fuse and output rather than separately. In the same field of endeavor, Kecskemethy et al teaches: process the input feature maps for the plurality of imaging modalities using an intermediate machine learning model to obtain a final feature map; and process the final feature map using one or more output machine learning models [figs. 1 and 3, p0014 (Shared/joint representation module takes in and fuses input tensors/feature maps to produce a final feature map to be sent to the classifier.)]. Therefore, given Kesckemethy et al’s disclosure on separating intermediate ML model from output ML model, it would have been an obvious alternative for an ordinary skilled in the art before the effective filing date of the claimed invention to have separate models for fusing and outputting per design choice.
Kihara et al in view of Kecskemethy et al does not specify diagnosis of retinal tear. In the same field of endeavor, Mulyukov et al teaches: the one or more estimated representations of the pathology of the eye of the patient comprising a diagnosis of a retinal tear and a severity score for the diagnosis [abstract, p00008-p00010]. Therefore, given Mulyukov et al’s prescription, applying Kihara et al in view of Kecskemethy et al’s ML mode for retinal tear diagnosis would have been obvious within grasp and practice of an ordinary skilled in the art.
Claim 2 has been analyzed and rejected with regard to claim 1.
Regarding claim 3, the rationale applied to the rejection of claim 2 has been incorporated herein. Kilara et al further teaches: The system of claim 2, wherein the plurality of imaging modalities include at least one of multispectral imaging (MSI) or optical coherence tomography (OCT) [page 2: Purpose].
Regarding claim 6, the rationale applied to the rejection of claim 2 has been incorporate herein. Kecskemethy et al further teaches: The system of claim 2, wherein the input feature map for each imaging modality is an output of a hidden layer of the input machine learning model for each imaging modality [p0053 (Encoder has multilayers so it incorporates a hidden layer.)].
Regarding claim 8, the rationale applied to the rejection of claim 2 has been incorporated herein. Kecskemethy et al further teaches: The system of claim 2, wherein the final feature map is an output of a hidden layer of the intermediate machine learning model [p0053 (Joint representation 25 is a hidden layer that outputs the final feature map to the output ML model.)].
Regarding claim 10, the rationale applied to the rejection of claim 2 has been incorporated herein. Kihara et al further teaches: The system of claim 2, wherein the one or more estimated representations of the pathology of the eye of the patient comprises a diagnosis of the pathology [page 2: keywords].
Regarding claim 11, the rationale applied to the rejection of claim 10 has been incorporated herein. Mulyukov et al further teaches: The system of claim 10, wherein the one or more estimated representations of the pathology of the eye of the patient comprises a severity score for the diagnosis [p00023-p00027].
Regarding claim 12, the rationale applied to the rejection of claim 10 has been incorporated herein. Kecskemethy et al further teaches: The system of claim 2, wherein the one or more estimated representations of the pathology of the eye of the patient comprise one or more biomarker segmentation maps [p0053, p0067 (Joint representation sends its outputs to decoders which reconstruct modality specific images.)].
Claim 13 has been analyzed and rejected with regard to claim 2.
Regarding claim 16, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 16 has been analyzed and rejected with regard to claims 6 and 8.
Regarding claim 18, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 18 has been analyzed and rejected with regard to claims 10 and 11.
Regarding claim 19, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 19 has been analyzed and rejected with regard to claim 12.
Regarding claim 20, the rationale applied to the rejection of claim 13 has been incorporated herein. Mulyukov et al further teaches further teaches: The method of claim 13, wherein the pathology of the eye includes at least one of:
[Symbol font/0xB7] Retinal tear(s)
[Symbol font/0xB7] Retinal detachment
[Symbol font/0xB7] Diabetic retinopathy
[Symbol font/0xB7] Hypertensive retinopathy
[Symbol font/0xB7] Sickle cell retinopathy
[Symbol font/0xB7] Central retinal vein occlusion
[Symbol font/0xB7] Epiretinal membrane
[Symbol font/0xB7] Macular hole(s)
[Symbol font/0xB7] Macular degeneration (including age-related Macular Degeneration)
[Symbol font/0xB7] Retinal pigmentosa
[Symbol font/0xB7] Glaucoma
[Symbol font/0xB7] Alzheimer’s disease
[Symbol font/0xB7] Parkinson’s disease [p00008-p00010].
51066.. Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Kihara et al (Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and Optical Coherence Tomography Imaging, July/2022), Kecskemethy et al (US Pub: 2019/0197366), and Mulyukov et al (WO Pub: 2021/220138); and in further view of Akita et al (US Pub: 2016/0073876).
Regarding claim 4, the rationale applied to the rejection of claim 2 has been incorporated herein. Kihara et al in view of Kecskemethy et al and Mulyukov et al does not specify integrating MSI and OCT. In the same field of endeavor, Akita et al teaches: The system of claim 2, wherein the plurality of imaging modalities include multispectral imaging (MSI) and optical coherence tomography (OCT) [p0025]. Therefore, it would have been obvious for an ordinary skilled in the art before the effective filing date of the claimed invention to integrate MSI with OCT for more optimal output image.
Regarding claim 14, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 14 has been analyzed and rejected with regard to claim 4.
61066.. Claims 5, 7, 9, 15, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kihara et al (Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and Optical Coherence Tomography Imaging, July/2022), Kecskemethy et al (US Pub: 2019/0197366), and Mulyukov et al (WO Pub: 2021/220138); and Lee et al (US Pub: 20220138493).
Regarding claim 5, the rationale applied to the rejection of claim 2 has been incorporated herein. Kihara et al in view of Kecskemethy et al further teaches: The system of claim 2, wherein each input machine learning model is one of a neural network, a deep neural network (DNN), a convolution neural network (CNN), a recurrent neural network (RNN), a region-based CNN (R-CNN), and an autoencoder (AE) [Kilhara: page 3: p02; Kecskemethy: p0032, p0056].
It has been well known in the art that a machine learning model can be a DNN, CNN, RNN, R-CNN, LSTM, AE, GAN or variants thereof. Nevertheless, Lee et al teaches a ML model can be any of those structures per design choice [p0048].
Regarding claims 7 and 9, the rationale applied to the rejection of claim 2 has been incorporated here. Claims 7 and 9 have been analyzed and rejected with regard to claim 5.
Regarding claim 15, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 15 has been analyzed and rejected with regard to claim 5.
Regarding claim 17, the rationale applied to the rejection of claim 13 has been incorporated herein. Claim 17 has been analyzed and rejected with regard to claim 9.
Contact
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to FAN ZHANG whose telephone number is (571)270-3751. The examiner can normally be reached on Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Tieu can be reached on 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Fan Zhang/
Patent Examiner, Art Unit 2682