DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
2. Claims 9 and 14 are objected to because of the following informalities:
In each of claims 9 and 14, “the stored data” should read “
Appropriate correction is required.
Claim Rejections - 35 USC § 112
3. The following is a quotation of 35 USC 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 USC 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
4. Claims 1-15 are rejected under 35 USC 112(b) or 35 USC 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 USC 112, the applicant), regards as the invention.
Regarding claim 1, line 13, the applicant claims “provid[ing] a grade related to severity level.” However, it is unclear whether applicant intends to refer to the severity level of the symptoms or of the identified ophthalmic diseases. For the purposes of this Office action, the examiner will interpret such that the claim reads “provide a grade related to severity level of the identified ophthalmic disease(s).”
On line 14, applicant claims “a detailed report.” The term “detailed” is a relative term which renders the claim indefinite. The term “detailed” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. For the purposes of this Office action, the examiner will interpret such that the claim reads “a Claim 9 is rejected for similar reasons.
Claims 2-8 and 10-15 inherit the issues of clarity posed by claims 1 and 9, respectively.
Regarding claim 3, the applicant claims “wherein the retinal video is preferably 2 to 3 second in duration.” The indefinite language “preferably” is used. The claim is considered unclear because there is a question or doubt as to whether the features introduced by the indefinite language are required features of the claim. See MPEP § 2173.05(c-d). For the purposes of this Office action, the examiner will interpret such that the claim reads “video is
Claim Rejections - 35 USC § 102
5. The following is a quotation of the appropriate paragraphs of 35 USC 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
6. Claims 1, 7-9, 11, 13, and 15 are rejected under 35 USC 102(a)(1) as being anticipated by Joskowicz et al. (WO 2022249184 A1).
Regarding claim 1, as best understood, Joskowicz discloses an Artificial Intelligence (AI) based system for detection of ophthalmic diseases (Abstract, machine learning), comprising:
one or more image capturing units ([0068]-[0069]) for capturing a retinal video of a patient ([00109]), wherein the retinal video comprises a plurality of retinal images ([00126]) of both left and right eyes ([0064]);
a plurality of modules stored in a memory ([0083], memory 206), comprising:
a pre-processing module configured to select one or more suitable retinal images from the retinal video and transform the retinal images into canonical image formats ([00127], “IR image and OCT slices were extracted from the video file (.png format)”);
a feature extraction module configured to extract one or more features from the selected retinal images ([0017]-[0018]);
a data analysis module configured to match the features with pre-stored images to identify potential symptoms indicating presence of ophthalmic diseases ([0013]);
an AI grading module configured to analyze the potential symptoms ([0074]) and provide a grade related to severity level ([0074], “quantifying the severity”); and
a report generation module configured to generate a detailed report including macular degeneration and geographic atrophy ([0056]).
Joskowicz does not explicitly disclose wherein the retinal video comprises images of both left and right eyes.
However, a reference disclosure can anticipate a claim when the reference describes the limitations but "'d[oes] not expressly spell out' the limitations as arranged or combined as in the claim, if a person of skill in the art, reading the reference, would ‘at once envisage’ the claimed arrangement or combination." Kennametal, Inc. v. Ingersoll Cutting Tool Co., 780 F.3d 1376, 1381, 114 USPQ2d 1250, 1254 (Fed. Cir. 2015) (quoting In re Petering, 301 F.2d 676, 681 (CCPA 1962)). See MPEP § 2131.02, section III.
In this instance, a PHOSITA would at once envisage that a “technique [which] provides for automated analysis of imaging studies of patients with advanced dry [age-related macular degeneration]” ([0064]) would address both the left and right eyes of each patient, rather than being limited to addressing only one eye of each patient.
Regarding claim 7, Joskowicz discloses wherein the storage module stores the retinal video, retinal images, canonical image formats, potential symptoms, grade, and detailed report according to a profile of the patient ([0025], “utilizing state information of the computer readable program instructions to personalize the electronic circuitry”).
Regarding claim 8, Joskowicz discloses wherein the system employs deep learning algorithms for analysis of retinal images and videos ([0072], [00147]).
Regarding claim 9, as best understood, Joskowicz discloses a method for AI-based detection of ophthalmic diseases using an AI-based system (Abstract, machine learning), the method comprising:
capturing one or more retinal images of an eye of a patient ([00126]) using one or more image capturing units ([0068]-[0069]), wherein the retinal images include a plurality of retinal images ([00126]) of both left and right eyes ([0064]);
pre-processing the retinal images to transform them into one or more canonical image formats (“IR image and OCT slices were extracted from the video file (.png format)”);
extracting one or more features from the pre-processed retinal images ([0017]-[0018]);
analyzing the features to identify potential symptoms indicative of one or more eye diseases ([0013]);
grading the potential symptoms to determine a severity level of the identified eye diseases ([0074], “quantifying the severity”); and
generating a detailed report based on the analysis and grading of the retinal images ([0056]), wherein the detailed report includes macular degeneration and geographic atrophy ([0056]).
Joskowicz does not explicitly disclose retinal images of both left and right eyes.
However, a reference disclosure can anticipate a claim when the reference describes the limitations but "'d[oes] not expressly spell out' the limitations as arranged or combined as in the claim, if a person of skill in the art, reading the reference, would ‘at once envisage’ the claimed arrangement or combination." Kennametal, Inc. v. Ingersoll Cutting Tool Co., 780 F.3d 1376, 1381, 114 USPQ2d 1250, 1254 (Fed. Cir. 2015) (quoting In re Petering, 301 F.2d 676, 681 (CCPA 1962)). See MPEP § 2131.02, section III.
In this instance, a PHOSITA would at once envisage that a “technique [which] provides for automated analysis of imaging studies of patients with advanced dry [age-related macular degeneration]” ([0064]) would address both the left and right eyes of each patient, rather than being limited to addressing only one eye of each patient.
Regarding claim 11, Joskowicz discloses wherein the storing step further includes organizing the stored data according to specific characteristics of the patient for efficient retrieval and analysis ([0064], according to their dry AMD).
Regarding claim 13, Joskowicz discloses wherein the analyzing step further comprises comparing the extracted features with a set of pre-stored images to identify patterns indicative of specific ophthalmic diseases ([0013]).
Regarding claim 15, Joskowicz discloses wherein the pre-processing step includes employing edge detection algorithms to identify boundaries of blood vessels or lesions ([00114]), and texture analysis techniques to detect abnormal patterns or colors in the retinal images ([00149]).
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. The factual inquiries for establishing a background for determining obviousness under 35 USC 103 are summarized as follows:
1) Determining the scope and contents of the prior art.
2) Ascertaining the differences between the prior art and the claims at issue.
3) Resolving the level of ordinary skill in the pertinent art.
4) Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claim 2 is rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Ahmad (WO 2010131944 A2).
Regarding claim 2, Joskowicz fails to disclose wherein the image capturing unit is a fundus camera.
However, Ahmad teaches a method of screening retinopathy (page 2 lines 24-27), and discloses wherein a fundus photograph taken by a fundus camera is analyzed (page 2 lines 24-27).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Ahmad such that the image capturing unit was a fundus camera, motivated by “identify[ing] pathologies for grading purposes” (page 2 lines 27).
10. Claims 3-4 are rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Zhao et al. (CN 115393345 A).
Regarding claim 3, as best understood, Joskowicz fails to disclose wherein the retinal video is preferably 2 to 3 seconds in duration.
However, Zhao teaches the use of deep learning technology to diagnose eye problems ([0002]), and discloses wherein a video of an eye is 2 seconds in duration ([0055]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Zhao such that the retinal video was preferably 2 to 3 seconds un duration, motivated by measuring vibration of a user’s eye ([0054]).
Regarding claim 4, Joskowicz fails to disclose wherein the pre-processing module further processes a RGB image received by the image capturing unit.
However, Zhao teaches the use of deep learning technology to diagnose eye problems ([0002]), and discloses processing a RGB image received by an image capturing unit ([0055]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Zhao such that the pre-processing module was to further process a RGB image received by the image capturing unit, motivated by RGB color providing images in greater detail.
11. Claim 5 is rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Sheetz et al. (US 20080234351 A1).
Regarding claim 5, Joskowicz discloses wherein the feature extraction module extracts features including lesions ([0012]), blood vessels ([0068]), fovea ([0043]), and abnormal blood vessel growth ([00161]).
Joskowicz fails to explicitly disclose wherein the feature extraction module extracts features including microaneurysms, hemorrhages, hard-exudates, soft-exudates, venous beading, intraretinal microvascular abnormalities (IRMA), neovascularization at the disc (NVD), neovascularization of the retina elsewhere (NVE), optic disc, and laser mark.
However, Sheetz teaches methods for addressing diabetic retinopathy in a patient (Abstract), and discloses wherein features including lesions ([0035]), blood vessels (page 4, ETDRS Final Retinopathy Severity Scale), microaneurysms (page 4, ETDRS Final Retinopathy Severity Scale), hemorrhages (page 4, ETDRS Final Retinopathy Severity Scale), hard-exudates (page 4, ETDRS Final Retinopathy Severity Scale), soft-exudates (page 4, ETDRS Final Retinopathy Severity Scale), venous beading (page 4, ETDRS Final Retinopathy Severity Scale), intraretinal microvascular abnormalities (IRMA) (page 4, ETDRS Final Retinopathy Severity Scale), neovascularization at the disc (NVD) (page 4, ETDRS Final Retinopathy Severity Scale), neovascularization of the retina elsewhere (NVE) (page 4, ETDRS Final Retinopathy Severity Scale), fovea ([0004]), optic disc (page 4, ETDRS Final Retinopathy Severity Scale), laser mark ([0010]), and abnormal blood vessel growth (page 4, ETDRS Final Retinopathy Severity Scale) are addressed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Sheetz such that the feature extraction module was to extract features including lesions, blood vessels, microaneurysms, hemorrhages, hard-/soft-exudates, venous beading, IRMA, NVD, NVE, fovea, optic disc, laser mark, and abnormal blood vessel growth, motivated by comprehensively assessing the health of a patient.
12. Claim 6 is rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Rom et al. (US 20230281816 A1).
Regarding claim 6, Joskowicz fails to disclose wherein the AI grading module provides grades including no apparent retinopathy, mild non-proliferative diabetic retinopathy, moderate non-proliferative diabetic retinopathy, severe non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy.
However, Rom teaches an AI-based system for detection of ophthalmic diseases (Abstract & [0084]), and discloses a grading scale which includes no apparent retinopathy, mild non-proliferative diabetic retinopathy, moderate non-proliferative diabetic retinopathy, severe non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy ([0116]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Rom such that an AI grading scale was to provide grades including “no apparent”/mild/moderate/severe/proliferative diabetic retinopathy, motivated by quantifying the severity of retinopathy.
13. Claims 10 and 12 are rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Rowe et al. (US 20220084659 A1).
Regarding claim 10, Joskowicz fails to explicitly discloses wherein the pre-processing step includes adjusting resolution, color space, and aspect ratio of the retinal images to create consistent baseline for analysis.
However, Rowe teaches the use of machine/deep learning to predict diagnostic test results ([0123]), and discloses validating the quality of images based on their resolution ([0092], “not acceptable blur and/or noise level in an image”), color space ([0092], “color balancing”), and aspect ratio ([0092], “identified spatial coordinates of a crop of the image”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Rowe such that resolution, color space, and aspect ratio of the retinal images were adjusted to create consistent baseline analysis, motivated by promoting the quality of a set of images.
Regarding claim 12, Joskowicz fails to explicitly discloses wherein the pre-processing step includes adjusting resolution, color space, and aspect ratio of the retinal images.
However, Rowe teaches the use of machine/deep learning to predict diagnostic test results ([0123]), and discloses validating the quality of images based on their resolution ([0092], “not acceptable blur and/or noise level in an image”), color space ([0092], “color balancing”), and aspect ratio ([0092], “identified spatial coordinates of a crop of the image”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Rowe such that resolution, color space, and aspect ratio of the retinal images were adjusted, motivated by promoting the quality of a set of images.
14. Claim 14 is rejected under 35 USC 103 as being unpatentable over Joskowicz in view of Wang et al. (CN 114927188 A).
Regarding claim 14, Joskowicz fails to discloses wherein the storing step further includes anonymizing the stored data to ensure patient privacy.
However, Wang teaches using A.I. for medical diagnoses ([0003]), and discloses anonymizing stored data to ensure patient privacy ([0038]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Joskowicz and Wang such that stored data was anonymized to ensure patient privacy, motivated by “reduc[ing] the risk of hospital privacy data leakage” ([0038]).
Conclusion
15. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Lavoie et al. (WO 2023272395 A1)
Nam (KR 20230059180 A)
De Fauw, Jeffrey, et al. “Clinically applicable deep learning for diagnosis and referral in retinal disease.” Nature medicine 24.9 (2018): 1342-1350.
16. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniel Jeffery Jordan whose telephone number is 571-270-7641. The examiner can normally be reached 9:30a-6:00p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephone Allen can be reached at 571-272-2434. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D. J. J./Examiner, Art Unit 2872
/STEPHONE B ALLEN/Supervisory Patent Examiner, Art Unit 2872