DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more. Independent claim 1 recites “generating a wound score based on the patient image, the score indicative of a healing progress of the wound under care” which falls under the grouping of Mental Processes because a person can visually inspect a would image and assign a score. Claim 1 further recites “computing, based on a comparison of images of other wounds with the patient image, whether additional care is needed for the wound under care; and rendering a recommendation including the wound score and the additional care” which fall under the grouping of Mental Processes because a doctor can visually inspect the image and compare with other images to assign a score and recommend additional care. The limitation “receiving a patient image from a personal device, the image containing an image of a wound under care” is merely a data gathering step which is an insignificant extra-solution activity. The claim does not recite additional elements that would integrate the abstract idea into a practical application nor does it provide an inventive concept.
Dependent claim 2 recites “identifying a plurality of attributes, each of the attributes indicative of a healing state of the wound under care; comparing the attributes of the wound under care to the attributes of the wound images in the database; generating, for each of the attributes, an evaluation score for each of the attributes of the wound under care based on the comparison; and computing the wound score based on each of the evaluation scores”, all of which fall under the grouping of Mental Processes because a doctor can visually inspect an image, compare the image features with other images, assign a score for different attributes such as size, depth etc. and finally mentally compute a score. The other limitation recites “accessing a database of wound images” which is merely a data gathering step which is an insignificant extra-solution activity. The claim does not recite additional elements that would integrate the abstract idea into a practical application nor does it provide an inventive concept.
Dependent claim 3 recites “wherein the attributes include: at least one text attribute based on a numeric value corresponding to the respective image; at least one image attribute based on the wound under care” which merely describe the images that are received, which include text attributes and image attributes.
Dependent claim 4 recites “concluding a need for the additional care based on comparing the wound under care to, for a plurality of the decision support evaluations, the evaluated wound image and the narrative segment corresponding to evaluated wound image” which falls under the grouping of Mental Processes because a doctor can visually inspect an image and compare with previously evaluated images and textual descriptions to make a recommendation for care. The other limitations recite “accessing a database of decision support evaluations, the decision support evaluations each including: an evaluated wound image; and a narrative segment; the narrative segment indicative of a state of healing of the wound under care” which is merely a data gathering step which is an insignificant extra-solution activity. The claim does not recite additional elements that would integrate the abstract idea into a practical application nor does it provide an inventive concept.
Dependent claim 5 recites “wherein the need for additional care includes at least one of continuing current care, referring for additional care, and referring for urgent care” which falls under the grouping of Mental Processes because a doctor can visually inspect an image and recommend additional care etc.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 9-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Dependent claim 2 recites “at least one text attribute based on a numeric value corresponding to the respective image”. It is not clear what the numeric value is for the respective image nor is it clear how the text is based on the numeric value.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-10, 12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by D1.1
With regard to claim 1, D1 teach receiving a patient image from a personal device, the image containing an image of a wound under care (§ 2 ¶ 1: smartphone to capture image of wound); generating a wound score based on the patient image, the score indicative of a healing progress of the wound under care (§ 3.3 ¶ 1: score of wound; § 4.5: healing progression); computing, based on a comparison of images of other wounds with the patient image, whether additional care is needed for the wound under care (see abstract, § 2.2, § 3.6: machine learning using training data sets, inherently based on comparison with features from training data set images, care decision); and rendering a recommendation including the wound score and the additional care (see abstract: care decision, § 3.3 ¶ 1, p. 12 table: total score).
With regard to claim 2, D1 teach accessing a database of wound images (see § 3.6 ¶ 1: training data set, see also § 3.5); identifying a plurality of attributes, each of the attributes indicative of a healing state of the wound under care (see p. 12 table, § 2.1 ¶¶ 2-3: Photograph Wound Assessment Tool or PWAT comprising plurality of attributes and corresponding scores indicative of wound state); comparing the attributes of the wound under care to the attributes of the wound images in the database (see §§ 3.6, 4.2: machine learning algorithms to assess wounds inherently compare features of images in database); generating, for each of the attributes, an evaluation score for each of the attributes of the wound under care based on the comparison; and computing the wound score based on each of the evaluation scores (see § 3. 5, p. 12 table: evaluation score for each of the attributes and total score).
With regard to claim 3, D1 teach at least one text attribute based on a numeric value corresponding to the respective image (see §§ 3.4, 3.5: text features, see table 3); at least one image attribute based on the wound under care (see §§ 3.4, 3.5, § 4.2: visual image features).
With regard to claim 4, D1 teach accessing a database of decision support evaluations, the decision support evaluations (see abstract: wound care decisions) each including: an evaluated wound image (see abstract, §§ 3.4, 3.5: imager features, see § 4.2: image visual features); and a narrative segment (see §§ 3.4, 3.5, 4.2: image and text or narrative features, see table 3: text description of wound); the narrative segment indicative of a state of healing of the wound under care (see table 3, §§ 3.4, 3.5: text or narrative segment describing wound) ; and concluding a need for the additional care based on comparing the wound under care to, for a plurality of the decision support evaluations, the evaluated wound image and the narrative segment corresponding to evaluated wound image (see §§ 5.1, ¶¶ 1-2: wound care decision based on visual and textual features, see § 4.2).
With regard to claim 5, D1 teach wherein the need for additional care includes at least one of continuing current care, referring for additional care, and referring for urgent care (see § 6: deciding what treatment a chronic wound requires).
With regard to claim 6, D1 teach wherein accessing a database of decision support evaluations further comprises: training a wound care model for bimodal classification based on text attributes and visual attributes from a training dataset, the training dataset including images of wounds and a corresponding treatment conclusion indicative of a need for additional care (see §§ 3.4-3.5, 4.2: classification based on text and visual features).
With regard to claim 7, D1 teach for each of a plurality of entries in the training dataset, generating: a text vector based on the text attributes and an image vector based on the visual attributes (see §§ 3.4-3.5, 4.2: text and visual features); and concatenating the text vector and the image vector to form a multimodal vector (see § 4.3 ¶ 1: concatenating document embedding and visual features).
With regard to claim 8, D1 teach comparing the patient image to the wound care model by: extracting visual features of the patient image (see § 4.2: visual features); matching the extracted visual features to the wound care model; and generating the recommendation based on text attributes of the entries having a correspondence of the visual features of the patient image ((see §§ 4.2-4.3, 3.4-3.5: text and visual features used by classification model to generate recommendation of care).
With regard to claim 9, D1 teach identifying a plurality of features descriptive of the wound under care based on the patient image (see §§ 3.4-3.5, 4.2: visual and textual features); comparing the wound under care to the wound healing model (see §§ 3.4-3.5, 4.2: machine learning model using visual and textual features); retrieving, based on the comparison, an integer score for each feature of the plurality of features; and computing the wound score based on an aggregation of the integer scores (see table 2, p. 12 table: PWAT scores and overall score; see § 5.2).
With regard to claim 10, D1 teach receiving a training corpus of a plurality of images of healing wounds (see §§ 3.6, 4.3: training data sets); annotating, for each image of healing wounds, an integer score for each feature of the plurality of features (see §§ 3.4-3.5: text annotations; see table 3: annotations of % of wound necrotic tissue).
With regard to claim 12, D1 teach wherein the plurality of features concern healing progression of a wound, including size, depth, necrotic tissue type, necrotic tissue amount, granulation tissue type, granulation tissue amount, edges, periulcer skin viability (see p. 12 table: features include size, depth etc.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over D1.
With regard to claim 11, D1 fails to teach wherein training the wound healing model further comprises: invoking a jigsaw puzzle generator for identifying and learning different granularity regions in each respective image in the training corpus. However, Examiner takes Official Notice to the fact that jigsaw puzzle generators are extremely well known in the field of machine learning and one skilled in the art would have been motivated to incorporate known teachings in to the configuration of D1 yielding predictable and enhanced results by improving spatial reasoning, improving robustness etc.
With regard to claim 14, D1 teach a personal device, the personal device configured for receiving a patient image of a wound under care (see (§§ 2 ¶ 1, 3.1: smartphone to capture image of wound); a smartphone app configured for generating a wound score based on the patient image, the score indicative of a healing progress of the wound under care (§ 3.3 ¶ 1: score of wound; § 4.5: healing progression; § 3.1: smartphone app); a wireless link coupling the personal device to a serversmartphone inherently has a wireless link to a server) see abstract, § 2.2, § 3.6: machine learning using training data sets, inherently based on comparison with features from training data set images, care decision); and rendering a recommendation including the wound score and the additional care (see abstract: care decision, § 3.3 ¶ 1, p. 12 table: total score).
D1 fails to explicitly teach engaging the server to perform analysis of the image, however Examiner takes Official Notice to the fact that it is extremely well known in the art to perform computationally expensive image processing on the server side and transmitting the results back to the smartphone and one skilled in the art would have been motivated to incorporate known teachings into the configuration of
Claims 13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over D1 and further in view of D2.2
With regard to claim 13, D1 fails to explicitly teach further comprising reconstructing the patient image, further comprising: identifying, for the received patient image, an ambient lighting, angle and distance of the personal device relative to the wound under care; evaluating a sufficiency of a patient image for analysis, the patient image depicting a wound under care; normalizing, if the patient image is sufficient for analysis, a shading of the patient image for comparison with other wound images, the shading based on the ambient lighting; reconstructing the patient image for accommodating variations in the angle and the distance.
D2 teach identifying, for the received patient image, an ambient lighting, angle and distance of the personal device relative to the wound under care (see abstract, § 3.5: images taken under different lighting conditions, distances, camera configuration, angle etc.); normalizing, if the patient image is sufficient for analysis, a shading of the patient image for comparison with other wound images, the shading based on the ambient lighting (see § 5.2: calibration of color shade).
One skilled in the art before the effective filing date would have found it obvious to combine the teachings which are in the same field of endeavor to arrive at the claimed invention. In particular, it would have been obvious to incorporate known teachings of calibration of images to normalize lighting as taught by D2 into the configuration of D1 yielding predictable and enhanced results. The motivation for normalizing the lighting to would have been to improve and standardize wound recognition and classification.
D1 and D2 fail to explicitly teach evaluating a sufficiency of a patient image for analysis and reconstructing the patient image for accommodating variations in the angle and the distance. However, Examiner takes Official Notice to the fact that checking whether the quality of the image is sufficient and normalizing the image for distance and angle are well known in the art before the effective filing date and would have been obvious for one skilled in the art to incorporate known teachings into the configuration of D1 yielding predictable results. The motivation would have been to improve would recognition and classification by normalizing the images taken at different angles so they appear to be taken at a certain angle, improving image analysis. Likewise, the motivation for checking whether the image is of sufficient quality before analyzing would improve would classification and recognition by ensuring the image is of sufficient quality.
With regard to claim 15, D1 teach receiving a patient image from a personal device, the image containing an image of a wound under care (§ 2 ¶ 1: smartphone to capture image of wound); § 3.3 ¶ 1: score of wound; § 4.5: healing progression); computing, based on a comparison of images of other wounds with the patient image, whether additional care is needed for the wound under care (see abstract, § 2.2, § 3.6: machine learning using training data sets, inherently based on comparison with features from training data set images, care decision); and rendering a recommendation including the wound score and the additional care (see abstract: care decision, § 3.3 ¶ 1, p. 12 table: total score).
D2 teach normalizing, if the patient image is sufficient for analysis, a shading of the patient image for comparison with other wound images, the shading based on the ambient lighting (see § 5.2: calibration of color shade).
One skilled in the art before the effective filing date would have found it obvious to combine the teachings which are in the same field of endeavor to arrive at the claimed invention. In particular, it would have been obvious to incorporate known teachings of calibration of images to normalize lighting as taught by D2 into the configuration of D1 yielding predictable and enhanced results. The motivation for normalizing the lighting to would have been to improve and standardize wound recognition and classification.
D1 and D2 fail to explicitly teach evaluating a sufficiency of a patient image for analysis and reconstructing the patient image for accommodating variations in the angle and the distance. However, Examiner takes Official Notice to the fact that checking whether the quality of the image is sufficient and normalizing the image for distance and angle are well known in the art before the effective filing date and would have been obvious for one skilled in the art to incorporate known teachings into the configuration of D1 yielding predictable results. The motivation would have been to improve would recognition and classification by normalizing the images taken at different angles so they appear to be taken at a certain angle, improving image analysis. Likewise, the motivation for checking whether the image is of sufficient quality before analyzing would improve would classification and recognition by ensuring the image is of sufficient quality.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AVINASH YENTRAPATI whose telephone number is (571)270-7982. The examiner can normally be reached on 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AVINASH YENTRAPATI/Primary Examiner, Art Unit 2672
1 Nguyen, Holly, et al. "Machine learning models for synthesizing actionable care decisions on lower extremity wounds." Smart Health 18 (2020): 100139.
2 Chairat, Sawrawit, et al. "AI-assisted assessment of wound tissue with automatic color and measurement calibration on images taken with a smartphone." Healthcare. Vol. 11. No. 2. MDPI, 2023.
Prosecution Insights