DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on June 25, 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 9, 11, 13 and 17-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by a machine translation of WO 2018222136 (hereinafter WO ‘136).
Regarding independent claim 1, WO ‘136 discloses A method (abstract, “the present invention relates to an image processing method and system”) comprising using at least one hardware processor (paragraph 0083, “The computational model can be implemented, but is not limited to, a desktop workstation (with or without a GPU).” paragraph 0101, “a laptop, a smart phone, a tablet, a computer, and an optical center”) to:
receive ophthalmic image data (paragraph 0079, "According to a calculation model using an embodiment of the present invention, a user's retina fundus image is used Compared with the reference image;" both of the users image and reference image must be received in order to be compared);
apply a machine-learning classifier (paragraph 0081, "Based on a plurality of reference images having classification codes, the AI engine is trained to construct a calculation model to perform an alignment analysis on the retinal fundus image of the user, and a classification code of the retinal fundus image of the user is obtained;" paragraph 0082, "The server storing the plurality of reference images and the calculation model having the classification code may be implemented in the form of deep learning ("DL") or deep neural network ("DNN"). The process of supervising learning machine learning (Machine Learning, "ML") or artificial intelligence (AI) frameworks. The obtained DNN algorithm and the calculation model can be used for comparing and analyzing the retinal fundus image of the user with the reference image in the server to obtain the classification code of the retinal fundus image of the user."), trained using a domain dataset of ophthalmic images that have been labeled with one or more of a plurality of classifications, to classify the received ophthalmic image data into at least one of the plurality of classifications (abstract, "The method comprises: receiving an initial user file from a user terminal, the initial user file comprising user data and a user image; loading the initial user file to a server, the server having stored therein reference images and a calculation model, the reference images comprising a plurality of reference images having classification code 1 and a plurality of reference images having classification code 2;;" paragraph 0072, "The image processing method of the embodiment of the present invention further includes training the AI engine based on the retinal fundus image of the at least one user file having the classification code of 2. Retinal fundus images that are classified as abnormal by the system can be sent to an expert for classification. Once classified, the expert classified retinal fundus image can be used to further train the AI engine;;" paragraph 0081, "Based on a plurality of reference images having classification codes, the AI engine is trained to construct a calculation model to perform an alignment analysis on the retinal fundus image of the user, and a classification code of the retinal fundus image of the user is obtained."), wherein the plurality of classifications comprise a normal classification and one or more disorder classifications (paragraph 0102, "Specifically, the classification code 1 may indicate that the corresponding retinal fundus image is classified as "normal" or "low eye disease risk", for example, an example of a "normal" retinal fundus image as shown in FIG. 14A. Classification code 2 may indicate that the corresponding retinal fundus image is classified as "abnormal" or "high eye disease risk". The sub-category code 2-1 may indicate that the corresponding retinal fundus image is classified as "abnormal but not urgent". The sub-category code 2-2 may indicate that the corresponding retinal fundus image is classified as "an abnormal and urgent situation" including, for example, a retinal fundus image having a "cataract" image feature as shown in FIG. 14B, as shown in FIG. 14C"), wherein the one or more disorder classifications comprise at least one of age- related macular degeneration (AMD), diabetic retinopathy (DR), glaucoma, or retinal vein occlusion (RVO) (paragraph 0064, "Classification code 2 indicates Corresponding retinal fundus images are classified as "abnormal" or "high eye disease risk", including, for example, a retinal fundus image having a "diabetic retinopathy" image feature as shown in FIG. 8B, having "glaucoma" as shown in FIG. 8C. Retinal fundus image of the image features (where 802 represents the cup-to-disk ratio (CDR) greater than or equal to 0.75), a retinal fundus image with "cataract" image features as shown in Figure 8D, and as shown in Figure 8E. Age-related macular degeneration (AMD) "image features of the retinal fundus image;"); and
provide a report that indicates the at least one classification of the received ophthalmic image data (paragraph 0092, "An example report according to the above description is as follows: Classification color marker symbol report meaning first category (green) (-) User will not be advised to consult an ophthalmologist for further examination; The second category (red) ( +) users will be advised to consult an ophthalmologist;").
Regarding dependent claim 9, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses wherein the machine-learning classifier comprises a convolutional neural network (paragraph 0087, “the architecture of the classification model is employed, but is not limited to the use of Convolutional Neural Networks (CNN).”).
Regarding dependent claim 11, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses wherein the ophthalmic image data comprises an image of an internal structure of a human eye (paragraph 0084, "An image processing method according to an embodiment of the present invention includes accessing a database including digitized retinal fundus images, and storing information for processing;" retinal fundus images are read as internal structure of the eye images)
Regarding dependent claim 13, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses wherein, when the at least one classification is one of the one or more disorder classifications, the report comprises:
(a) one or more recommendations for treatment of a disorder associated with the one disorder classification (paragraph 0092, "An example report according to the above description is as follows: Classification color marker symbol report meaning first category (green) (-) User will not be advised to consult an ophthalmologist for further examination; The second category (red) (+) users will be advised to consult an ophthalmologist;" the treatment recommendation is read as following up with the ophthalmologist as opposed to those who have no treatment recommendation which is read as healthy; paragraph 0100, "For user files whose sub-category code is determined to be 2-2, the method and system generate a fifth follow-up code and add an updated user file for suggesting that the user immediately consult an ophthalmologist or for suggesting a medical institution/ophthalmologist immediately The user is given emergency medical treatment for further screening nd necessary treatment (i.e., emergency), as shown in block 1360.”);
(b) an image from the ophthalmic image data that shows areas of importance used by the machine-learning classifier (NOTE: limitation not analyzed because claim only requires one of a, b or c); or
(c) probabilities of the plurality of classifications based on a Softmax function (NOTE: limitation not analyzed because claim only requires one of a, b or c).
Regarding dependent claim 17, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses wherein receiving ophthalmic image data comprises, at a mobile device, capturing ophthalmic image data using an ophthalmoscope that is detachably coupled to the mobile device (paragraph 0094, "another embodiment of the present invention may include: (i) a portable fundus camera for capturing a retinal fundus image of a user and imaging the user's retina fundus with user data Combining to create an initial user file with a user retinal fundus image and user data; (ii) a mobile phone for transmitting the initial user file to a mobile phone tower connected to the network;" one well known method of portable cameras are mobile cameras).
Regarding dependent claim 18, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses wherein the ophthalmic image data comprises a retinal image or a fundus image (paragraph 0084, "An image processing method according to an embodiment of the present invention includes accessing a database including digitized retinal fundus images, and storing information for processing"), and/or wherein the one or more disorder classifications are a plurality of disorder classifications (NOTE: due to “and/or” limitation is not required; paragraph 0078, "According to the present embodiment, the reference image to which the classification code is added, the engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts."), and wherein the plurality of disorder classifications further comprise at least one of a cataract (paragraph 0078, "According to the present embodiment, the reference image to which the classification code is added, the engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts."), myopia, kidney disease, hypertension, or stroke.
Regarding dependent claim 19, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses A system (abstract, “The present invention relates to an image processing method and system. ”) comprising:
at least one hardware processor (paragraph 0083, “The computational model can be implemented, but is not limited to, a desktop workstation (with or without a GPU).” paragraph 0101, “a laptop, a smart phone, a tablet, a computer, and an optical center”); and
one or more software modules configured to, when executed by the at least one hardware processor, perform the method of Claim 1 (paragraph 0083, “The computational model can be implemented, but is not limited to, a desktop workstation (with or without a GPU).” … “The computational model can choose a high-level programming language and platform (including but not limited to Matlab®, Python, C++ and R, etc., as well as the packaging of these platforms) implementation.”).
Regarding dependent claim 20, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 further discloses A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to perform the method of Claim 1 (paragraph 0083, “The computational model can be implemented, but is not limited to, a desktop workstation (with or without a GPU).” … “The computational model can choose a high-level programming language and platform (including but not limited to Matlab®, Python, C++ and R, etc., as well as the packaging of these platforms) implementation.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136.
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, the normal classification is defined as:
(a) optic discs presenting with sharp margins and a cup-to-disc ratio within a predetermined range, striated sheen from a healthy retinal nerve fiber layer, no lesions, no apparent sub-retinal disruptions, no pigmentary changes, no tumors, no scars, no molds, and normal vasculature, with an exception for drusens indicative of normal age progression; or
(b) no clinically abnormal features, no known ocular disease, and hemoglobin A1C levels less than 6.0%, with an exception for less than five drusens indicative of normal age progression. However, WO ‘136 discloses classifying images as normal versus abnormal (paragraph 0064, "assign a classification code to each reference retinal fundus image, wherein the classification code 1 indicates that the corresponding retinal fundus image is classified as "normal" or "low eye disease risk"; A "normal" retinal fundus image as shown in Fig. 8A. Classification code 2 indicates Corresponding retinal fundus images are classified as "abnormal" or "high eye disease risk", including, for example, a retinal fundus image having a "diabetic retinopathy" image feature as shown in FIG. 8B, having "glaucoma" as shown in FIG. 8C."). Further, WO '136 performs this analysis based on features within the image for example a CDR ratio (paragraph 0064, "Retinal fundus image of the image features (where 802 represents the cup-to-disk ratio (CDR) greater than or equal to 0.75), a retinal fundus image with "cataract" image features as shown in Figure 8D, and as shown in Figure 8E."). Additionally, WO '136 performs the analysis of comparing normal to abnormal by comparing an input image to known references (paragraph 0064, "(f) using the calculation model, comparing the retinal fundus image in the initial user file with the reference retinal fundus image in the server to determine the classification code of the user image as one of 1 and 2;"). Thus, though not explicitly stated, any features that can be derived from images or seen in images can be compared to determine a relevant normal or abnormal classification. Additionally, it is well known in the art that there are a variety of features that can be relevant to certain diseases. For example, some diseases may be classified based on a cup to disc ratios, or lesions present. Thus a normal feature of a healthy eye can be analyzed by multiple features as well (a normal cup to disc ratio or lack of lesions). Optometrists may be focused on a variety of different features in order to consider something normal versus abnormal. Further, as noted in WO '136, the network is trained off of reference images (see paragraph 0034), and in order to perform a specific task, the system must be trained to perform that task. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of WO '136 in order to vary parameters for determining a normal versus abnormal classification based upon the features in question as related to the eye.
Regarding dependent claim 6, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, glaucoma is defined as having one or both of concentric expansion with an optic cup-to-disc ratio greater than or equal to 0.5, or narrowing of a disc at either a superior or inferior rim with a localized nerve fiber layer defect.
However, WO ‘136 discloses diagnosing glaucoma (paragraph 0078, "According to the present embodiment, the reference image to which the classification code is added, the engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts."). Additionally, WO '136 discloses a CTDR of larger than .75 at paragraph 0102, "an increased cup-to-disk ratio (cup-to-disk ratio greater than or equal to 0.75)." Further, paragraph 0004-0009 discloses the variety of glaucoma diagnosis and basic understanding of how the disease can occur. Finally, at paragraph 0031, WO '136 discloses, "Preferably, the user image is a retinal fundus image of the user, wherein comparing the user image with the reference image further comprises comparing using at least one of the following eye state determination elements: (a) multiple retinal vessels present in the image; (b) the cup-to-disk ratio is less than 0.3; (c) Missing at least one of the following elements: (i) visible media turbidity; (ii) a diabetic retinopathy indicator comprising at least one of blotting-like bleeding, microaneurysms, and hard exudates; (iii) diabetic macular degeneration; (iv) macular edema; (v) exudates near the macula; (vi) exudate on the macula; (vii) laser scars; (iii) cataract; (ix) glaucoma." As such, and as noted in claim 2 analysis above, various diseases are categorized according to various features. Some physicians may want a different range for a threshold of cup-to-disc ratios as related to glaucoma in order to either only classify definitive glaucoma, or those who may have a possibility of glaucoma. Further, as noted in WO '136, the network is trained off of reference images (see paragraph 0034) and in order to diagnose glaucoma at a specific threshold, the system must be trained to diagnose based on the known references at that threshold. Additionally, with regard to the concentric expansion, this is a feature that can be analyzed from an image (i.e. has it grown or not in a consistent manner). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of WO '136 in order to classify glaucoma based on a different threshold only noting confirmed glaucoma (and not suspected glaucoma) and further analyzing additional features that may aid in a more clear diagnosis.
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, glaucoma is defined as having two or more of a vertical optic cup-to-disc ratio greater than or equal to 0.8, superior or inferior disc notch or rim thinning, or a retinal nerve fiber layer (RNFL) defect radiating from an optic nerve head.
However, WO '136 does disclose diagnosing glaucoma (paragraph 0078, "According to the present embodiment, the reference image to which the classification code is added, the engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts."). Additionally, WO '136 discloses a CTDR of larger than .75 at paragraph 0102, "an increased cup-to-disk ratio (cup-to-disk ratio greater than or equal to 0.75)." Further, paragraph 0004-0009 discloses the variety of glaucoma diagnosis and basic understanding of how the disease can occur. Finally, at paragraph 0031, WO '136 discloses, "Preferably, the user image is a retinal fundus image of the user, wherein comparing the user image with the reference image further comprises comparing using at least one of the following eye state determination elements: (a) multiple retinal vessels present in the image; (b) the cup-to-disk ratio is less than 0.3; (c) Missing at least one of the following elements: (i) visible media turbidity; (ii) a diabetic retinopathy indicator comprising at least one of blotting-like bleeding, microaneurysms, and hard exudates; (iii) diabetic macular degeneration; (iv) macular edema; (v) exudates near the macula; (vi) exudate on the macula; (vii) laser scars; (iii) cataract; (ix) glaucoma." As such, and as noted in claim 2 analysis above, various diseases are categorized according to various features. Some physicians may want a different range for a threshold of cup-to-disc ratios as related to glaucoma in order to either only classify definitive glaucoma, or those who may have a possibility of glaucoma. Further, as noted in WO '136, the network is trained off of reference images (see paragraph 0034) and in order to diagnose glaucoma at a specific threshold, the system must be trained to diagnose based on the known references at that threshold. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of WO '136 in order to classify glaucoma based on a different threshold only noting confirmed glaucoma (and not suspected glaucoma).
Claim(s) 3 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of U.S. Patent No. 7,220,000 to Alster et al. (hereinafter Alster).
Regarding dependent claim 3, the rejection of 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, AMD is defined as advanced and late stage AMD with apparent macular damage from either dry AMD or wet AMD.
However, Alster discloses wherein, within the dataset, AMD is defined as advanced and late stage AMD (column 48, line 43, "AMD patients may generally be clinically divided into groups such as patients having early AMD without high-risk characteristics (HRC), patients having early AMD with HRC, patients having late AMD with geographic atrophy (GA), and patients having advanced AMD with choroidal neovascularization (CNV).") with apparent macular damage from either dry AMD or wet AMD (column 1, line 25, “the key identifier for dry AMD is small, round, white-yellow lesions (also known as Drusen) in the macula;” column 1, line 38, “wet AMD is characterized by the development of choroidal angiogenesis which causes severe, and potentially rapid, visual deterioration” ).
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal (abstract). Alster is directed toward "Methods and devices for detecting eye disease and for assessing the clinical stage of an eye disease in an individual (abstract).” As can be seen by one of ordinary skill in the art at the time of the claimed invention, both WO '136 and Alster are directed toward similar methods of endeavor of image processing for diagnosis. Additionally, WO'136 discloses diagnosing AMD (paragraph 0078, "Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts."). However, one of ordinary skill in the art before the effective filing date of the claimed invention would realize that there are a variety of types of AMD, and the treatments and progression are different for the various types of AMD. Further, it is well known in the art that in order to perform a specific task, a network must be trained to perform that specific task. For example, in order to classify the difference between wet and dry AMD, the network must be trained off of data sets to perform that (and more specifically off of labeled sets). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Alster in order to perform a task of distinguishing between varying AMD types allowing for a more specific diagnosis and treatment.
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of a machine translation of DE 112010002000 (hereinafter DE ‘000).
Regarding dependent claim 4, the rejection of 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, DR is defined as moderate to proliferative DR, as classified according to the International Clinical Diabetic Retinopathy Disease Severity Scale.
However, DE ‘000 discloses wherein, within the dataset, DR is defined as moderate to proliferative DR, as classified according to the International Clinical Diabetic Retinopathy Disease Severity Scale (page 7, "Typically, the classification of diabetic retinopathy and diabetic maculopathy is due to the pathologies present in the retinal vasculature, as in US Pat 2-A and 2 B which represent two tables, namely the international clinical severity scale of diabetic retinopathy disease and the international clinical severity scale of diabetic macular edema disease [diabetic macular edema is a particular type of diabetic maculopathy].").
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal (abstract). DE '000 is directed at analyzing eye images to determine classification of diabetic retinopathy (abstract). As can be seen by one of ordinary skill in the art at the time of the claimed invention, both WO '136 and DE '000 are directed toward similar methods of endeavor of image processing for diagnosis. Further, it can be easily understood by one of ordinary skill in the art that as physicians diagnose diseases, having one consistent scale is helpful in communication. For example, two different physicians could classify one patient as a 2 on one scale, and an 8 on another scale, aiding in confusion when communicating about the same patient. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of DE '000 in order to ensure there is one consistent staging system, allowing for ease and clarity of communication.
Claim(s) 5 rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 and DE ‘000 as applied to claim 4 above, and further in view of U.S. Publication No. 2007/0287932 to Huang et al. (hereinafter huang).
Regarding dependent claim 5, the rejection of 4 is incorporated herein. Additionally, WO ‘136 and DE ‘000 in the combination fails to explicitly disclose wherein, within the dataset, diabetic macular edema (DME) is defined as one or more of retinal thickening within 500 micrometers of a macular center, hard exudates within 500 micrometers of the macular center with adjacent retinal thickening, or one or more disc diameters of retinal thickening that are within one disc diameter of the macular center.
However, Huang discloses wherein, within the dataset, diabetic macular edema (DME) is defined as one or more of retinal thickening within 500 micrometers of a macular center (paragraph 0004, "Macular edema is an important cause of visual loss and legal blindness in patients with diabetic retinopathy. In the Early Treatment Diabetic Retinopathy Study (ETDRS) [1], focal laser photocoagulation was demonstrated to reduce the risk of moderate vision loss in diabetic patients with an entity termed clinically significant macular edema (CSME)"... " Three definitions of CSME were adopted by the ETDRS investigators: (1) presence of any retinal thickening within 500 .mu.m of the foveal center," paragraph 0053, "If any retinal edema was present (even if not CSME), a grade of DME was assigned. Using this nonexclusive scheme, it is important to note that a case of edema within 500 .mu.m of the foveal center and with an area>1 Macular Photocoagulation Study DA would be classified as containing CSME1, CSME2, and DME"), hard exudates within 500 micrometers of the macular center with adjacent retinal thickening, or one or more disc diameters of retinal thickening that are within one disc diameter of the macular center.
As noted above, WO '136 and DE '000 are directed toward processing images to diagnose a patient's eye disease. Further, Huang is directed toward, "a method for comparing the detection of clinically significant diabetic macular edema by an optical coherence tomography (OCT) (abstract)." Thus, as can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, WO '136, DE '000 and Huang are all directed toward similar methods of endeavor of processing medical images. Further, it can be easily understood by one of ordinary skill in the art that as physicians diagnose diseases, having a consistent metric to classify a disease is helpful for grading. For example, two different physicians could classify one patient as a 2 on their own threshold, and a 8 on another threshold, increasing confusion when communicating about the same patient. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Huang in order to ensure there is a consistent metric for disease analysis and classification.
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of Rehak J, Rehak M. Branch retinal vein occlusion: pathogenesis, visual prognosis, and treatment modalities. Curr Eye Res. 2008 Feb;33(2):111-31.doi: 10.1080/02713680701851902. PMID: 18293182; PMCID: PMC2430176. (hereinafter Rehak).
Regarding dependent claim 8, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein, within the dataset, RVO is defined as non- ischemic and ischemic central RVO and major and macular branch RVO.
However, Rehak discloses wherein, within the dataset, RVO is defined as non- ischemic (page 123, “5 eyes on non-ischaemic BRVO”) and ischemic central RVO (page 112, left column, “of the two main types of RVO, central retinal vein occlusion (CRVO) and branch retina vein occlusion (BRVO)”) and major and macular branch RVO (Major and macular branch: page 112, left column, "BRVO is divided into two distinct entities: major BRVO, when one of the major branch retinal veins is occluded, and macular BRVO, when one of the macular venules is occluded. In 66% of eyes with BRVO, there is occlusion of the major branch in the superotemporal quadrant followed by 22-43% of eyes with occlusion of the major branch in the inferotemporal quadrant.").
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal (abstract). Rehak is directed toward analyzing images of branch retinal vein occlusion (abstract). It can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention that WO '136 and Rehak are directed toward similar methods of endeavor of image processing for diagnosis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would realize that there are a variety of types of RVO, and the treatments and progression are different for the various types of RVO. Further, it is well known in the art that in order to perform a specific task, a network must be trained to perform that specific task. For example, in order to classify the difference between non-ischemic and ischemic RVO, the network must be trained off of data sets to perform that specific task. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Rehak in order to classify the difference between additional diseases in question such as ischemic and non-ischemic RVO.
Claim(s) 12 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of a machine translation of WO 2020/009292 (hereinafter WO ‘292).
Regarding dependent claim 12, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 discloses further comprising using the at least one hardware processor to, when the at least one classification is one of the one or more disorder classifications (paragraph 0078, "According to the present embodiment, the reference image to which the classification code is added, the engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts;" paragraph 0087, "The architecture of the classification model is employed, but is not limited to the use of Convolutional Neural Networks (CNN).") However, WO ‘136 fails to explicitly disclose as further recited.
However, WO ‘292 discloses use regression analysis to determine a severity of a disorder associated with the one disorder classification (page 21, "Alternatively, the neural network model may be a multiple classification model that classifies input data into a plurality of class classes for particular characteristics (eg, degree of disease progression). Alternatively, the neural network model may be implemented as a regression type model that outputs a specific value related to a specific disease,”), wherein the report further indicates the determined severity of the disorder (page 15, "The server device may output the result of the diagnostic assistance process using the neural network model to the client device;" page 16, "Although not shown, the client device may further include an output unit. The output unit may include a display that outputs an image or an image or a speaker that outputs an audio. The output unit may output the image or the image data obtained by the obtained imaging unit. The output unit may output diagnostic assistance information obtained from the diagnostic apparatus;" the output of the diagnostic data is read as the report (i.e. the data displayed can be read as a report itself)).
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal (abstract). WO '292 is directed toward "a method for determining the quality of an eye fundus image (abstract)." As can be easily seen by one of ordinary skill in the art, both WO '136 and WO '292 are directed toward similar methods of endeavor of image analysis. Further, it can be easily understood by one of ordinary skill in the art that upon determining a disease itself, it is often helpful for one to determine how severe a disease is (for example, a low level cancer or a highly aggressive cancer). Severity information of a disease can aid in determining treatment protocols that should be used, as well as predicting possible outcomes (i.e. how quickly a patient may become blind). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of WO '292 in order to determine a disease severity to generate the most optimal treatment plan ideally resulting in a better patient outcome.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of WO 2017011532 (hereinafter WO ‘532).
Regarding dependent claim 14, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose further comprising using the at least one hardware processor to, after receiving the ophthalmic image data and before applying the machine- learning classifier:
determining a type of the ophthalmic image data; and
selecting the machine-learning classifier, that is associated with the determined type of the ophthalmic image data, from a plurality of different machine-learning classifiers associated with a plurality of different types of ophthalmic image data.
However, WO ‘532 discloses further comprising using the at least one hardware processor to, after receiving the ophthalmic image data and before applying the machine- learning classifier:
determining a type of the ophthalmic image data (Paragraph 0040, "The result of step 211 is an anatomical location context for the current candidate abnormality. For purposes of illustration it is assumed that the anatomical location contexts considered for the current organ are designated context A, context B, among others. For each context, a different set of subsequent steps are taken, all analogous to the steps shown in FIG. 2 for context A;" the anatomical location context is read as the type of data (i.e. the image could be of a retina vs a pupil)); and
selecting the machine-learning classifier, that is associated with the determined type of the ophthalmic image data, from a plurality of different machine-learning classifiers associated with a plurality of different types of ophthalmic image data (Paragraph 0040, "The result of step 211 is an anatomical location context for the current candidate abnormality. For purposes of illustration it is assumed that the anatomical location contexts considered for the current organ are designated context A, context B, among others. For each context, a different set of subsequent steps are taken, all analogous to the steps shown in FIG. 2 for context A;" based on the context (i.e. type) the different processing is performed; further paragraphs 0051-0052 explain how machine learning and neural networks can be used to process the varying image features and are already well known in the art).
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal (abstract). WO '532 is directed toward "automated detection and processing of abnormalities in medical imagery (abstract). "As can be easily understood by one of ordinary skill in the art, both WO '136 and WO '532 are directed toward similar methods of endeavor of image analysis. Further, neural networks are often trained to perform specific tasks on specific inputs; thus it would logically follow that if a different input than is expected is input, the system may produce inaccurate results or not perform optimally. For example, if a system is trained to predict glaucoma based on an eye image, and an image of the stomach is input, the system may determine an image of the stomach is predictive of glaucoma (which would be a wrong diagnosis). Thus, it would have been obvious to a person having ordinary skill in the art before the effetivef filing date of the claimed invention to incorporate the teaching of WO '532 in order to ensure only inputs that can be accurately analyzed by the system are included, to prevent inaccurate diagnosis.
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science(), vol 9351. Springer, Cham. (hereinafter Ronneberger).
Regarding dependent claim 15, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 discloses optionally wherein an activation function after each convolutional layer in the U-net architecture comprises a rectifier linear unit (ReLU) (paragraph 0088, "The CNN can be designed to have a specific number of layers consisting of interconnected nodes. The link between each node in a different function is defined by a function consisting of weights and deviations. Activation functions such as RELU are often used to update the weights and deviations of functions."). WO ‘136 fails to explicitly disclose as further recited. However, Ronneberger discloses wherein applying the machine-learning classifier comprises segmenting vessels in the ophthalmic image data using a U-net architecture (Figure 1, "Fig. 1. U-net architecture (example for 32x32 pixels in the lowest resolution);" Figure 3, "(c) generated segmentation mask (white: foreground, black: background);" page 240, "The u-net architecture achieves very good performance on very different biomedical segmentation applications;" different biomedical segmentation applications are read as alternative solutions available for the segmentation of vessels).
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal using neural networks (abstract, paragraph 0087). Ronneberger is directed toward utilizing deep networks for segmentation to be efficient and accurate (abstract). Thus, as can be seen by one of ordinary skill in the art before the effective filing date of the claimed invention, WO '136 and Ronneberger are both directed toward similar methods of endeavor of image analysis. Further, WO '136 uses a neural network to perform the classification of the input data set (see paragraph 0087). One well known desire of neural networks is to perform increasingly efficient analysis and reduce human error as associated with a specific task. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Ronneberger in order to further increase efficiency as associated with the U-net architecture (abstract, "Segmentation of a 512x512 image takes less than a second").
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over WO ‘136 as applied to claim 1 above, and further in view of U.S. Patent No. 5,943,116 to Zeimer (hereinafter Zeimer).
Regarding dependent claim 16, the rejection of claim 1 is incorporated herein. Additionally, WO ‘136 fails to explicitly disclose wherein the ophthalmic image data comprises video, and wherein the method further comprises using the at least one hardware processor to, before applying the machine-learning classifier, stich a plurality of frames of the video together to generate a composite image to which the machine-learning classifier is applied. However, Zeimer discloses wherein the ophthalmic image data comprises video (abstract, "A system for obtaining images of the fundus of an eye includes an illumination device which directs a light beam onto a portion of the fundus of the eye, and a video camera which records the portion of the light reflected from the fundus of the eye), and wherein the method further comprises using the at least one hardware processor to, before applying the machine-learning classifier, stich a plurality of frames of the video together to generate a composite image (column 4, line 22, “generates a wide-field composite image”) to which the machine-learning classifier is applied.
With regard to performing this processing before the machine-leaming classifier of WO '136 is used, any data processing can be performed to ensure the input data is in proper format for processing. For example, the neural network of WO '136 may only accurately process input images of a specific size (i.e. the composite image size). Thus, generating the composite images before the neural network processing would allow for the correct input to be analyzed. Further, if the neural network was trained on a specific size composite image, the system should operate once trained on only composite images of the specific size. Had an image of an improper size been included, the output could be inaccurate.
WO '136 is directed toward "an image processing method and system (abstract)" which classifies the input images as 1 or 2 being abnormal or normal using neural networks (abstract, paragraph 0087). Zeimer is directed toward, "A system for obtaining images of the fundus of an eye (abstract)." Thus, as can be easily seen by one of ordinary skill in the art, both WO '136 and Zeimer are directed toward similar methods of endeavor of image obtaining and processing. Further, Zeimer allows for obtaining multiple images (i.e. a video, being consecutive images) and generating a composite image. Composite images can be well known in the art to provide different or additional information than included in just one image. This additional/different information could lead to a more accurate diagnosis, as opposed to a traditional image that may not have caught a feature in question. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Zeimer in order to allow the most accurate image with as much information as possible contained within the image to be analyzed, resulting in a more accurate diagnosis.
Double Patenting
Non-statutory
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 and 14-18 of U.S. Patent No. 12,293,830 (hereinafter ‘830). Although the claims at issue are not identical, they are not patentably distinct from each other because the current invention is more broad in scope than the claims of US ‘830.
Claim 1: Regarding claim 1, claim 1 compares to claim 1 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 1 of the current application is more broad in scope than claim 1 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 1 of the ‘830 patent.
Instant application – Claim 1
US ‘830 – Claim 1
Notes
A method comprising
A method comprising:
Verbatim
using at least one hardware processor to:
a) using at least one hardware processor to:
Substantially the same
receive ophthalmic image data;
receive ophthalmic image data;
Verbatim
apply a machine-learning classifier, trained using a domain dataset of ophthalmic images that have been labeled with one or more of a plurality of classifications, to classify the received ophthalmic image data into at least one of the plurality of classifications, wherein the plurality of classifications comprise a normal classification and one or more disorder classifications, wherein the one or more disorder classifications comprise at least one of age- related macular degeneration (AMD), diabetic retinopathy (DR), glaucoma, or retinal vein occlusion (RVO); and
apply a machine-learning classifier, trained using a domain dataset of ophthalmic images that have been labeled with one or more of a plurality of classifications, to classify the received ophthalmic image data into at least one of the plurality of classifications, wherein the machine-learning classifier comprises a convolutional neural network, wherein the plurality of classifications comprise a normal classification and one or more disorder classifications, wherein the one or more disorder classifications comprise at least one of age-related macular degeneration (AMD), diabetic retinopathy (DR), glaucoma, or retinal vein occlusion (RVO); and
Instant application more broad
provide a report that indicates the at least one classification of the received ophthalmic image data.
provide a report that indicates the at least one classification of the received ophthalmic image data; and
verbatim
b) using the at least one hardware processor to train the machine-learning classifier by:
Instant application more broad
initially training the convolutional neural network to discriminate between objects using a non-domain dataset that contains no ophthalmic images labeled with the one or more disorders; and
Instant application more broad
subsequently retraining one or more final layers in the convolutional neural network using the domain dataset.
Instant application more broad
Claim 2: Regarding claim 2, claim 2 compares to claim 2 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 2 of the current application is more broad in scope (when considering dependency) than claim 2 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 2 of the ‘830 patent.
Instant application – Claim 2
US ‘830 – Claim 2
Notes
The method of claim 1, wherein, within the dataset, the normal classification is defined as:
The method of claim 1, wherein, within the dataset, the normal classification is defined as:
Verbatim
(a) optic discs presenting with sharp margins and a cup-to-disc ratio within a predetermined range, striated sheen from a healthy retinal nerve fiber layer, no lesions, no apparent sub-retinal disruptions, no pigmentary changes, no tumors, no scars, no molds, and normal vasculature, with an exception for drusens indicative of normal age progression; or
(a) optic discs presenting with sharp margins and a cup-to-disc ratio within a predetermined range, striated sheen from a healthy retinal nerve fiber layer, no lesions, no apparent sub-retinal disruptions, no pigmentary changes, no tumors, no scars, no molds, and normal vasculature, with an exception for drusens indicative of normal age progression; or
Verbatim
(b) no clinically abnormal features, no known ocular disease, and hemoglobin AlC levels less than 6.0%, with an exception for less than five drusens indicative of normal age progression.
(b) no clinically abnormal features, no known ocular disease, and hemoglobin A1C levels less than 6.0%, with an exception for less than five drusens indicative of normal age progression.
verbatim
Claim 3: Regarding claim 3, claim 3 compares to claim 3 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 3 of the current application is more broad in scope (when considering dependency) than claim 3 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 3 of the ‘830 patent.
Instant application – Claim 3
US ‘830 – Claim 3
Notes
The method of Claim 1, wherein, within the dataset, AMD is defined as advanced and late stage AMD with apparent macular damage from either dry AMD or wet AMD.
he method of claim 1, wherein, within the dataset, AMD is defined as advanced and late stage AMD with apparent macular damage from either dry AMD or wet AMD.
Verbatim
Claim 4: Regarding claim 4, claim 4 compares to claim 4 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 4 of the current application is more broad in scope (when considering dependency) than claim 4 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 4 of the ‘830 patent.
Instant application – Claim 4
US ‘830 – Claim 4
Notes
The method of Claim 1, wherein, within the dataset, DR is defined as moderate to proliferative DR, as classified according to the International Clinical Diabetic Retinopathy Disease Severity Scale.
The method of claim 1, wherein, within the dataset, DR is defined as moderate to proliferative DR, as classified according to International Clinical Diabetic Retinopathy Disease Severity Scale.
Verbatim
Claim 5: Regarding claim 5, claim 5 compares to claim 5 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 5 of the current application is more broad in scope (when considering dependency) than claim 5 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 5 of the ‘830 patent.
Instant application – Claim 5
US ‘830 – Claim 5
Notes
The method of Claim 4, wherein, within the dataset, diabetic macular edema (DME) is defined as one or more of retinal thickening within 500 micrometers of a macular center, hard exudates within 500 micrometers of the macular center with adjacent retinal thickening, or one or more disc diameters of retinal thickening that are within one disc diameter of the macular center.
The method of claim 4, wherein, within the dataset, diabetic macular edema (DME) is defined as one or more of retinal thickening within 500 micrometers of a macular center, hard exudates within 500 micrometers of the macular center with adjacent retinal thickening, or one or more disc diameters of retinal thickening that are within one disc diameter of the macular center.
Verbatim
Claim 6: Regarding claim 6, claim 6 compares to claim 6 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 6 of the current application is more broad in scope (when considering dependency) than claim 6 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 6 of the ‘830 patent.
Instant application – Claim 6
US ‘830 – Claim 6
Notes
The method of Claim 1, wherein, within the dataset, glaucoma is defined as having one or both of concentric expansion with an optic cup-to-disc ratio greater than or equal to 0.5, or narrowing of a disc at either a superior or inferior rim with a localized nerve fiber layer defect.
The method of claim 1, wherein, within the dataset, glaucoma is defined as having one or both of concentric expansion with an optic cup-to-disc ratio greater than or equal to 0.5, or narrowing of a disc at either a superior or inferior rim with a localized nerve fiber layer defect.
verbatim
Claim 7: Regarding claim 7, claim 7 compares to claim 7 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 7 of the current application is more broad in scope (when considering dependency) than claim 7 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 7 of the ‘830 patent.
Instant application – Claim 7
US ‘830 – Claim 7
Notes
The method of Claim 1, wherein, within the dataset, glaucoma is defined as having two or more of a vertical optic cup-to-disc ratio greater than or equal to 0.8, superior or inferior disc notch or rim thinning, or a retinal nerve fiber layer (RNFL) defect radiating from an optic nerve head.
The method of claim 1, wherein, within the dataset, glaucoma is defined as having two or more of a vertical optic cup-to-disc ratio greater than or equal to 0.8, superior or inferior disc notch or rim thinning, or a retinal nerve fiber layer (RNFL) defect radiating from an optic nerve head.
Verbatim
Claim 8: Regarding claim 8, claim 8 compares to claim 8 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 8 of the current application is more broad in scope (when considering dependency) than claim 8 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 8 of the ‘830 patent.
Instant application – Claim 8
US ‘830 – Claim 8
Notes
The method of claim 1, wherein, within the dataset, RVO is defined as non-ischemic and ischemic central RVO and major and macular branch RVO.
The method of Claim 1, wherein, within the dataset, RVO is defined as non- ischemic and ischemic central RVO and major and macular branch RVO.
Verbatim
Claim 9: Regarding claim 9, claim 9 compares to claim 1 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 9 of the current application is more broad in scope (when considering dependency) than claim 1 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 1 of the ‘830 patent.
Instant application – Claim 9
US ‘830 – Claim 1
Notes
The method of Claim 1, wherein the machine-learning classifier comprises a convolutional neural network.
wherein the machine-learning classifier comprises a convolutional neural network,
verbatim
Claim 10: Regarding claim 10, claim 10 compares to claim 1 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 10 of the current application is more broad in scope (when considering dependency) than claim 1 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 1 of the ‘830 patent.
Instant application – Claim 10
US ‘830 – Claim 1
Notes
The method of Claim 9, wherein the method further comprises using the at least one hardware processor to train the machine-learning classifier by:
b) using the at least one hardware processor to train the machine-learning classifier by:
Substantially the same
initially training the convolutional neural network to discriminate between objects using a non-domain dataset that contains no ophthalmic images labeled with the one or more disorders;
initially training the convolutional neural network to discriminate between objects using a non-domain dataset that contains no ophthalmic images labeled with the one or more disorders; and
verbatim
subsequently retraining one or more final layers in the convolutional neural network using the domain dataset,
subsequently retraining one or more final layers in the convolutional neural network using the domain dataset.
verbatim
optionally wherein a number of images in the non-domain dataset is at least six times greater than a number of images in the domain dataset.
NOTE: being that the limitation is “optional” the reading without the limitation reads the same as claim 1 scope as well
Claim 11: Regarding claim 11, claim 11 compares to claim 9 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 11 of the current application is more broad in scope (when considering dependency) than claim 9 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 9 of the ‘830 patent.
Instant application – Claim 11
US ‘830 – Claim 9
Notes
The method of Claim 1, wherein the ophthalmic image data comprises an image of an internal structure of a human eye.
The method of claim 1, wherein the ophthalmic image data comprises an image of an internal structure of a human eye.
Verbatim
Claim 12: Regarding claim 12, claim 12 compares to claim 10 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 12 of the current application is more broad in scope (when considering dependency) than claim 10 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 10 of the ‘830 patent.
Instant application – Claim 12
US ‘830 – Claim 10
Notes
The method of Claim 1, further comprising using the at least one hardware processor to, when the at least one classification is one of the one or more disorder classifications, use regression analysis to determine a severity of a disorder associated with the one disorder classification, wherein the report further indicates the determined severity of the disorder.
The method of claim 1, further comprising using the at least one hardware processor to, when the at least one classification is one of the one or more disorder classifications, use regression analysis to determine a severity of a disorder associated with the one disorder classification, wherein the report further indicates the determined severity of the disorder.
Verbatim
Claim 13: Regarding claim 13, claim 13 compares to claim 11 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 13 of the current application is more broad in scope (when considering dependency) than claim 11 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 11 of the ‘830 patent.
Instant application – Claim 13
US ‘830 – Claim 11
Notes
The method of Claim 1, wherein, when the at least one classification is one of the one or more disorder classifications, the report comprises:
The method of claim 1, wherein, when the at least one classification is one of the one or more disorder classifications, the report comprises:
Verbatim
(a) one or more recommendations for treatment of a disorder associated with the one disorder classification;
(a) one or more recommendations for treatment of a disorder associated with the one disorder classification; or
Verbatim
(b) an image from the ophthalmic image data that shows areas of importance used by the machine-learning classifier; or
(b) an image from the ophthalmic image data that shows areas of importance used by the machine-learning classifier.
Verbatim
(c) probabilities of the plurality of classifications based on a Softmax function.
Being that the claim is interpreted as “or” the limitation isn’t required, thus the scope of the claims read on eachother
Claim 14: Regarding claim 14, claim 14 compares to claim 12 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 14 of the current application is more broad in scope (when considering dependency) than claim 12 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 12 of the ‘830 patent.
Instant application – Claim 14
US ‘830 – Claim 12
Notes
The method of Claim 1, further comprising using the at least one hardware processor to, after receiving the ophthalmic image data and before applying the machine- learning classifier:
The method of claim 1, further comprising using the at least one hardware processor to, after receiving the ophthalmic image data and before applying the machine-learning classifier:
Verbatim
determining a type of the ophthalmic image data; and
determining a type of the ophthalmic image data; and
Verbatim
selecting the machine-learning classifier, that is associated with the determined type of the ophthalmic image data, from a plurality of different machine-learning classifiers associated with a plurality of different types of ophthalmic image data.
selecting the machine-learning classifier, that is associated with the determined type of the ophthalmic image data, from a plurality of different machine-learning classifiers associated with a plurality of different types of ophthalmic image data.
Verbatim
Claim 15: Regarding claim 15, claim 15 compares to claim 14 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 15 of the current application is more broad in scope (when considering dependency) than claim 14 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 14 of the ‘830 patent.
Instant application – Claim 15
US ‘830 – Claim 14
Notes
The method of Claim 1, wherein applying the machine-learning classifier comprises segmenting vessels in the ophthalmic image data using a U-net architecture,
The method of claim 1, wherein applying the machine-learning classifier comprises segmenting vessels in the ophthalmic image data using a U-net architecture
Verbatim
optionally wherein an activation function after each convolutional layer in the U-net architecture comprises a rectifier linear unit (ReLU).
The method of claim 13, wherein an activation function after each convolutional layer in the U-net architecture comprises a rectifier linear unit (ReLU).
verbatim
Claim 16: Regarding claim 16, claim 16 compares to claim 15 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 16 of the current application is more broad in scope (when considering dependency) than claim 15 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 16 of the ‘830 patent.
Instant application – Claim 16
US ‘830 – Claim 15
Notes
The method of Claim 1, wherein the ophthalmic image data comprises video, and wherein the method further comprises using the at least one hardware processor to, before applying the machine-learning classifier, stich a plurality of frames of the video together to generate a composite image to which the machine-learning classifier is applied.
The method of claim 1, wherein the ophthalmic image data comprises video, and wherein the method further comprises using the at least one hardware processor to, before applying the machine-learning classifier, stich a plurality of frames of the video together to generate a composite image to which the machine-learning classifier is applied.
verbatim
Claim 17: Regarding claim 17, claim 17 compares to claim 16 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 17 of the current application is more broad in scope (when considering dependency) than claim 16 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 16 of the ‘830 patent.
Instant application – Claim 17
US ‘830 – Claim 16
Notes
The method of Claim 1, wherein receiving ophthalmic image data comprises, at a mobile device, capturing ophthalmic image data using an ophthalmoscope that is detachably coupled to the mobile device.
The method of claim 1, wherein receiving ophthalmic image data comprises, at a mobile device, capturing ophthalmic image data using an ophthalmoscope that is detachably coupled to the mobile device.
verbatim
Claim 18: Regarding claim 18, claim 18 compares to claims 17 and 18 of the ‘830 patent as indicated below. As can be seen from the chart below, claim 18 of the current application is simply a combination of claims 17 and 18 of the ‘830 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 17 and 18 of the ‘830 patent.
Instant application – Claim 18
US ‘830 – Claim 17/18
Notes
The method of Claim 1, wherein the ophthalmic image data comprises a retinal image or a fundus image, and/or wherein the one or more disorder classifications are a plurality of disorder classifications, and wherein the plurality of disorder classifications further comprise at least one of a cataract, myopia, kidney disease, hypertension, or stroke.
The method of claim 1, wherein the ophthalmic image data comprises a retinal image or a fundus image.
The method of claim 1, wherein the one or more disorder classifications are a plurality of disorder classifications, and wherein the plurality of disorder classifications further comprise at least one of a cataract, myopia, kidney disease, hypertension, or stroke.
Verbatim when combining two claims previously unrelated
Claim 19: Regarding claim 19, claim 19 is a system using hardware and software to implement the method of claim 1 of the current application. The ‘830 patent fails to explicitly claim a system using hardware and software to implement the respective methods of claim 1 in the ‘830 patent. However, a double patent rejection between a method and apparatus claim is appropriate when the claims of the method claim could easily be carried out by the apparatus of the apparatus claims. Thus, the apparatus of claim 19, could easily carry out the method of claim 1 in both the instant application and the ‘830 patent. Therefore, any patent granted on it would result in the unjustifiable timewise extension on the monopoly granted on claim 1 of the ‘830 patent.
Claim 20: Regarding claim 20, claim 20 compares to claim 1 of the ‘830 patent in that claim 20 is read as a method to implement claim 1 of the instant application. One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand the method of claim 1 in the current application and claim 1 in the ‘830 patent can be implemented using a non-transitory computer-readable medium. Therefore, any patent granted on it would result in the unjustifiable timewise extension on the monopoly granted on claim 1 of the ‘830 patent.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and overcoming the above noted non-statutory double patenting rejection.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of utilizing neural networks to process ophthalmic images for diagnosis. However, none of them alone or in any combination teaches first training the CNN using a non-domain data set using unlabeled data, then retraining one or more final layers in the CNN with a domain dataset.
The closest prior art WO '136 discloses an image processing method to determine classification of a disease (abstract). Further, WO'136 performs this method using a convolutional neural network (paragraph 0028) that has been previously trained through a variety of methods (paragraph 0081).
However, WO '136 fails to disclose first training the CNN using a non-domain data set using unlabeled data, then retraining one or more final layers in the CNN with a domain dataset.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Publication No. 2017/0357879 discloses “Disclosed herein are systems, methods, and devices for classifying ophthalmic images according to disease type, state, and stage (abstract).”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
COURTNEY J. WINDSOR
Primary Examiner
Art Unit 2661
Prosecution Insights