DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 7/6/2016 have been fully considered but they are not persuasive.
The Examiner withdraws the 101 rejections.
Regarding the 112 arguments, no arguments are on the record.
In response to Applicant’s arguments, The Examiner agrees that Abramoff notes “[0122] The methods and systems can employ Artificial Intelligence techniques such as machine learning and iterative learning. Examples of such techniques include, but are not limited to, expert systems, case based reasoning, Bayesian networks, behavior based AI, neural networks, fuzzy systems, evolutionary computation (e.g. genetic algorithms), swarm intelligence (e.g. ant algorithms), and hybrid intelligent systems (e.g. Expert inference rules generated through a neural network or production rules from statistical learning).” Abramoff explicitly discloses that the method/systems can be accomplished using one or more of many AI techniques.
Abramoff has an express disclosure of per-location indications of objects of interest (paragraph 89),determine diagnosis from detected objects (paragraph 94-109) . Abramoff (paragraph 122) expressly discloses substituting AI techniques in place of the method/system steps disclosed. This generic statement is sufficient to anticipate Applicant’s generic recitation of a supervised learning machine learning model used for diagnostics using the same inputs as Abramoff
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-16,21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “obtaining a plurality of training examples … for a portion of pixels in the given input image, an indication that the given input image contains an object of interest at the portion of pixels”. The Examiner is unable to find support for this limitation. While the Specification teaches having a heat map during the inference, but doesn’t teach a heat map/pointwise input or in particular “for a portion off pixels in the given input image, an indication that the given input image contains an object of interest at the portion of pixels” during training. At best it discloses in paragraphs 26 & 28 images having/not having feature of interest anywhere.
Claim 1 last limitation recites “training … until the diagnostic model achieves a performance threshold”. Paragraph 24-25 defines threshold/thresholding as “[024] A "threshold" is defined as a level, point, or value above which something is true or will take place and below which it is not or will not, such levels, points, or values include probabilities, sizes in pixels, and values representing pixel brightness. [025] "Thresholding" is defined as modifying pixels that contain a characteristic either above or below a selected threshold value.”
The original disclosure fails to disclose this limitations. A threshold is only disclosed in paragraphs 5, 35, 37 40 and 77. None in the same context as the specification. There does not appear to be any disclosure of terminating the training step. For instance, paragraph 40, states “(g)repeating steps (d)-(g) until an objective performance threshold is reached” There are no details on what is compared (ie the diagnostic model) to the performance threshold.
Claim 2 recites “a mathematical model of the object of interest”. While the disclosure does support heat maps and point-wise outputs, it does not expressly use the phrase “mathematical model”, nor does it support anything broader than the heat map or point-wise outputs.
Claim 21 recites “adjusting the diagnostic model to fit the plurality of training examples and achieves the performance threshold, thereby causing the diagnostic model to become trained.” This limitation is not supported by the original disclosure.
Claim 9 is rejected under similar grounds as claim1.
Claims 2-8 and 10-21 are rejected as dependent upon a rejected claim.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 5, 9, 10, 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2 & 10 recites “for each of one or more locations”. This limitation lacks antecedent basis.
Claims 5 &13 recites “the objects of interest”. This limitation lacks antecedent basis.
Claim 9 recites “a non-transitory computer readable medium” (preamble), yet the last limitation store the model on “the computer readable medium”. This claim is circular. The Examiner recommends removing the last computer readable medium.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-16 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Abramoff (PGPub 2013/0301889).
Abramoff discloses 1. (Original) A method for training a diagnostic model for diagnosing a disease condition in a patient, the method comprising:
accessing a plurality of input images, each input image including a portion of body of a patient selected from a plurality of patients; accessing a label for each input image of the plurality of input images, wherein the label indicates whether the selected patient in the input image has a disease condition; (Abramoff“ [0058] In an aspect, the method of FIG. 1 can comprise a training stage and a classification or test stage as shown, as shown in FIG. 1. As an example, during the training stage, a set of images (e.g., digital images with expert-annotated objects), can be used to build one or more object filters, and to optimize the classification.”) (Abramoff, “[0079] In direct sampling, the target lesions (e.g., typical lesions and positive lesion confounders) can be annotated on a training dataset. As an example, the annotation comprises an indication of the center of the lesions, or segments the lesions. In an aspect, a candidate lesion detector can be used to find a center of the lesion within the segmented region. As an example, a set of sample images can represent all images and all local image variation that the proposed filter framework may encounter to optimize the framework.”; additionally see paragraph 135, where each image has an associated label)
selecting a plurality of training examples from the plurality of input images, each training example corresponding to a given input image and comprising: for a portion of pixels in the given input image, an indication that the given input image contains an object of interest at the portions of pixels, wherein the object of interest is indicative of a disease, and the label of the given input image; and(Abramoff, “[0079] In direct sampling, the target lesions (e.g., typical lesions and positive lesion confounders) can be annotated on a training dataset. As an example, the annotation comprises an indication of the center of the lesions, or segments the lesions. In an aspect, a candidate lesion detector can be used to find a center of the lesion within the segmented region. As an example, a set of sample images can represent all images and all local image variation that the proposed filter framework may encounter to optimize the framework.”)
for a diagnostic model, the diagnostic model comprising a machine learning model that is configured to output a diagnosis of a disease condition based on an input of indications of whether there is an object of interest at each location within a sample image (Abramoff, “[0059] In an aspect, internally the training set of images can be divided into a first subset and a second subset. As an example, the first subset can be used to build an optimal set of object filters. As a further example, the second subset can be used to perform feature selection on the optimal set of object filter, resulting in selected filters. A classifier can be trained based on the selected filters, resulting in a trained vision machine for which the performance can be determined.”)(Abramoff, “[0089] In step 402, a preprocessing step can be used to identify and normalize candidate target lesions in an image. In step 404, for each pixel pi;j selected in the preprocessing step 402, the normalized neighborhood of the pixel--the sample--is input to the classifier. As an example, the risk of presence of the target lesion is computed by the classifier for pi;j and a lesion risk map is thus obtained, at 406. In step 408, if the lesions need to be segmented, the risk map defined in step 406 is thresholded, and the connected foreground pixels are identified using morphological labeling. Each connected component is regarded as an automatically detected lesion, and this lesion is assigned a risk value defined as the maximal pixelwise risk of presence within the connected component. In step 410, if a probabilistic diagnosis for the entire image is desired/required, the risks assigned to each automatically detected lesions are fused. Typically, if a single type of lesions is detected, the probabilistic diagnosis for the image is simply defined as the maximum risk of presence of a target lesion.”)(Abramoff, “[0122] The methods and systems can employ Artificial Intelligence techniques such as machine learning and iterative learning. Examples of such techniques include, but are not limited to, expert systems, case based reasoning, Bayesian networks, behavior based AI, neural networks, fuzzy systems, evolutionary computation (e.g. genetic algorithms), swarm intelligence (e.g. ant algorithms), and hybrid intelligent systems (e.g. Expert inference rules generated through a neural network or production rules from statistical learning).”):
training the diagnostic model by repeatedly applying a training example from the plurality of training examples to the diagnostic model and updating parameters of the diagnostic model to improve an objective performance threshold thereof, and stopping the training after the objective performance threshold satisfies a condition. (Abramoff, “[0098] In an aspect, probability maps can be generated with different optimal filters chosen by Sequential Forward Selection (SFS) for each iteration. As an example, the metric can evaluate the probability maps and give the AUC for the ROC curve for the whole set of probability maps. As a further example, SFS first selects out one filter with the highest AUC, then adds a new filter from the remaining filters such that the two filters have the highest AUC. In an aspect, SFS can add a new filter from the remaining filters to give the highest AUC until the stop criteria of the feature selection is met. As an example, the feature selection stops when the number of selected filters reaches the maximal number of filters, or the AUC starts to decline.”)
In paragraph 89, Abramoff determines a probabilistic diagnosis using a model which comprises a machine learning model (Paragraph 59, 86&87) used for determining object filter. In paragraph 89, The diagnosis model uses the machine learned object models to identify target lesions and generates a lesion map[indicators of objects of interest per pixel location]. This lesion map is determined using the probabilistic model shown in paragraphs 94-95, thus Abramoff doesn’t explicitly say that this stage is machine-learned model. Paragraph 122 further discloses “The methods and systems can employ Artificial Intelligence techniques such as machine-learning …”
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to replace the probabilistic model with a machine learned model as explicitly disclosed by Abramoff (paragraph 122)
The suggestion/motivation for doing so would have been from Abramoff (paragraph 122). Additionally Abramoff already uses machine learning models in (paragraphs 89&98) and already discloses having the training data sets.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to modify Abramoff based on its own suggestion to obtain the invention as specified in claim 1
Abramoff discloses 2. (Currently amended) The method of claim 1, wherein the indication that the given input image contains an object of interest for each of one or more locations comprises a mathematical model of the retinal object of interest. (Abramoff, “[0064] As an example, in the expert-driven approach (e.g., mathematical modeling 104), a modeler can be configured to translate verbal descriptions with clinicians, as well as intensity distributions in a limited number of samples selected by the clinicians, into mathematical equations. In an aspect, the mathematical equations can be based upon a continuous model of the intensity distribution of the modeled lesions, and were defined for typical lesions, for positive lesion confounders if any are identified, and for negative lesion confounders, if any are identified. As a further example, the intensity profile of the lesions can be modeled by generalized Gaussian functions of normalized intensity, for example, such as the following function:
profile(r;.beta.,.delta.)=.delta.e.sup.-r.sub..beta.”)
Abramoff discloses 3. (Original) The method of claim 1, wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a heat map indicating the likelihood that the sample image contains an object of interest for each location in the sample image.(Abramoff, paragraph 89)
Abramoff discloses 4. (Original) The method of claim 1, wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a point-wise output corresponding to indications that the sample image contains an object of interest at each location in the sample image. (Abramoff, paragraph 89)
Abramoff discloses 5. (Original) The method of claim 1, wherein one or more of the objects of interests is indicative of disease. (Abramoff, paragraph 89)
Abramoff discloses 6. (Currently amended) The method of claim 1, wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of a disease condition in the patient comprises a diagnosis of a disorder manifesting in a retina. (Abramoff, paragraph 2-3)
Abramoff discloses 7. (Original) The method of claim 6, wherein one or more of the object of interests is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.(Abramoff, paragraph 39)
Abramoff discloses 8. (Original) The method of claim 1, wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x- ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.(Abramoff, paragraph 3)
Claims 9-16 are rejected under similar grounds as claims 1-8 as shown above.
Claim 21 is rejected under similar grounds as claim 1.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GANDHI THIRUGNANAM whose telephone number is (571)270-3261. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GANDHI THIRUGNANAM/ Primary Examiner, Art Unit 2672
Prosecution Insights