DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/29/2026 has been entered.
Response to Arguments
Applicant’s arguments, see Remarks page 6, filed 04/29/2026, with respect to the objection of claim 10 have been fully considered and are persuasive. The objection of claim 10 has been withdrawn.
Applicant’s arguments, see Remarks page 6, filed 04/29/2026, with respect to the rejections of claims 10-15 and 17-20 under 35 U.S.C. 101 have been fully considered and are persuasive. The rejections of claims 10-15 and 17-20 have been withdrawn.
Applicant’s arguments, see Remarks pages 6-9, filed 04/29/2026, with respect to the rejections of amended claim(s) 1 & 10 under 35 U.S.C. 103 have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below) necessitated by Applicant’s amendment to the claim(s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-10, 12-15, 17-18, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Andersen (WO-2020125906-A1) in view of Skladnev et al. (US-6993167-B1) and Kim (KR-101959880-B1).
Regarding claim 1, Andersen discloses: An imaging method for imaging an anatomical site having a feature on a body, comprising the step of: operating a computing device having a processor and a memory, wherein the memory includes executable instructions that, when executed by the processor, cause the computing device to undertake the steps (Andersen: Page 33: “The operations of the server device 208 may be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (e.g., the memory module 401) and are executed by the processor module 402.”) of: receiving from an imaging device a first 2D image of a first anatomical site on the body, wherein the first anatomical site has the feature (Andersen: Abstract).
Andersen does not disclose expressly: obtaining a second 2D image of a second anatomical site on the body that is free of the feature, wherein the first anatomical site and the second anatomical site are separated from one another on the body; subtracting the second 2D image from the first 2D image to identify the feature in the first 2D image.
Skladnev discloses: a method for collecting, storing, and displaying dermatological images for monitoring and diagnosing skin conditions and cancers (Skladnev: Abstract), wherein the process comprises: receiving from an imaging device a first 2D image of a first anatomical site on the body, wherein the first anatomical site has the feature; obtaining a second 2D image of a second anatomical site on the body that is free of the feature, wherein the first anatomical site and the second anatomical site are separated from one another on the body (Skladnev: Col 9: Lines 14-19: “To assist the image analysis process of automatically identifying the boundary between clean skin and lesion by providing for each lesion or group of lesions one or more images of adjacent clean skin, to the same scale as that of the lesion image, for use as a statistical skin colour reference”); subtracting the second 2D image from the first 2D image to identify the feature in the first 2D image (Skladnev: Col 36-37: Lines 48-19: “Actual distinction between skin and lesion can be done in a number of ways…
The more difficult case where there is no obvious watershed is shown in FIG. 24B. In this case, it is possible to fit a known histogram shape for one class ( a light”) class, e.g., skin) to the unknown histogram (for the lesion image) and to then subtract it. This leaves a smaller histogram representing the second class ( the dark class, in this case a lesion)…
Returning to FIG. 23, in step 415 the remaining unmasked area of the lesion image is analysed in conjunction with the skin statistics obtained in the previous step to allow a distinction to be drawn between skin and lesion. Subsequent processing then focuses solely on pixels in the lesion area, henceforth referred to as the "lesion area." The distinction is managed in the form of a mask image similar to that used for hair and bubbles.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the method of capturing and subtracting lesion and clean skin images for the extraction of a lesion area taught by Skladnev by capturing and subtracting a user’s clean skin and stomal images for the extraction of the stomal area disclosed by Andersen. The suggestion/motivation for doing so would have been “To facilitate automatically (by software) identifying the boundary between clean skin and lesion with accuracy, that is, given information about what clean skin should look like, distinguish between clean skin and a lesion” (Skladnev: Col 9-10: Lines 66-2.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Andersen in view of Skladnev does not disclose expressly: converting the first 2D image into a grayscale image after subtraction; and plotting the grayscale image into the 3D image to generate a 3D representation of the identified feature.
Kim discloses: converting the first 2D image into a grayscale image; and plotting the grayscale image into the 3D image to generate a 3D representation of the identified feature (Kim: 0024: “Meanwhile, the method of the present invention includes: (A) a step of a gray color conversion unit converting a two-dimensional skin surface image captured by a microscope camera into a gray image; (B) a step of a three-dimensional skin surface point generation unit converting an intensity level for each pixel of the gray image into depth data to form three-dimensional skin surface points; (C) a step of a three-dimensional skin surface generation unit connecting the three-dimensional skin surface points into a mesh to form a three-dimensional skin surface image”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the method for generating a 3D surface image taught by Kim to generate a 3D image of the extracted stoma disclosed by Andersen in view of Skladnev. The suggestion/motivation for doing so would have been “The multimodal rendering system provides users (dermatologists or skin specialists) with precise skin roughness information, which is crucial for diagnosing skin diseases, as well as visual investigations for accurate virtual tactile and skin examination.” (Kim: 0035). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Andersen in view of Skladnev with Kim to obtain the invention as specified in claim 1.
Regarding claim 2, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, wherein the feature is a stoma and/or a peristomal skin (Wherein limitation including “and/or” is interpreted as requiring either a stoma or peristomal skin) (Andersen: Abstract).
Regarding claim 3, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, further including determining at least one dimension of the feature from the first 2D image (Andersen: Page 30: “transforming stoma image data may comprise centering the stoma image data about a center, perimeter, or center region of the stoma (e.g. identified as a first or second stoma reference identifier in S108E)”; Wherein the determining of a center, perimeter, or center region of the stoma constitutes determining at least 1 dimension of the anatomical site.).
Regarding claim 5, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, further including generating contours of the anatomical site using the first 2D image (Andersen: Figure 10: BL_1-BL_3; Pages 35-36: “Fig. 10 shows an exemplary second ostomy representation OR_2 based on four appliance image representations as also described in relation to Fig. 9. The second ostomy representation OR_2 comprises a first boundary line BL_ 1 (red line) indicative of a circumference or edge of the adhesive surface of the ostomy appliance…The second ostomy representation OR_2 comprises second boundary line BL_2 (green line) indicative of a circumference or edge of the stomal opening of the adhesive surface…The second ostomy representation OR_2 comprises third boundary line BL_3 (blue line) indicative of a boundary between a discoloured part (output leak) and a non-discoloured part (clean) of the adhesive surface.”; Wherein figure 10 is an image containing boundary lines for different contours of the stoma area.).
Regarding claim 6, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, wherein the step of plotting the grayscale image into the 3D image includes determining a depth of each pixel in the grayscale from an intensity value of the pixel, wherein the intensity value ranges from black to white, and wherein the intensity value is plotted along a z-axis (Kim: 0088: “Here, z is a depth value directly calculated from the intensity level (I(x,y)) at the pixel coordinate (x, y) of the gray image”; Wherein the image’s intensity value based on gray scaled values constitutes an intensity range from black to white.).
Regarding claim 7, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 6, the step of plotting the grayscale image into the 3D image further includes a scaling process, wherein the z-axis is scaled based on estimated pixel-based height and width measurements. (Kim: 0073-0074: “The above visual rendering unit (230) uses Phong shading (PHONG, 1975) to visualize a three-dimensional skin surface for realistic visual rendering using a two-dimensional skin surface image. The above display (240) displays the visual three-dimensional skin surface.”; 0113: “the 3D skin surface point generation unit (200) generates 3D skin surface points (x, y, z) by converting the intensity level (KIM and LEE, 2015) for each pixel expressed by the x and y axes from the converted gray image into a position value on the z axis (converting it into depth data) to generate 3D depth information (S200).”; Wherein the plotting of 2D grayscale image onto a displayed 3D plot, which displays estimated height, width, and depth values, constitutes the scaling of the z-axis based on the x and y axis, and thus estimated height and width measurements.).
Regarding claim 8, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, wherein the step of plotting the grayscale image into the 3D image further includes a clean-up step using a background subtraction process (Skladnev: Col 36-37: Lines 48-19: “Actual distinction between skin and lesion can be done in a number of ways…
The more difficult case where there is no obvious watershed is shown in FIG. 24B. In this case, it is possible to fit a known histogram shape for one class ( a light”) class, e.g., skin) to the unknown histogram (for the lesion image) and to then subtract it. This leaves a smaller histogram representing the second class ( the dark class, in this case a lesion)…
Returning to FIG. 23, in step 415 the remaining unmasked area of the lesion image is analysed in conjunction with the skin statistics obtained in the previous step to allow a distinction to be drawn between skin and lesion. Subsequent processing then focuses solely on pixels in the lesion area, henceforth referred to as the "lesion area." The distinction is managed in the form of a mask image similar to that used for hair and bubbles.”; Wherein the subtraction is performed prior to the image’s conversion to grayscale.).
Regarding claim 9, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 1, further including determining a color of the anatomical site from the first 2D image (Andersen: Page 9: “determining one or more image representations based on the image data comprises determining a base color parameter, e.g. including a first base color parameter and/or a second base color parameter, and determining the one or more image representations and/or one or more ostomy parameters based on the base color parameter. The base color parameter may be based on red channel data of the ostomy image data.”).
Regarding claim 10, Andersen discloses: An imaging method for imaging an anatomical site having a feature on a body, comprising the step of: operating a computing device having a processor and a memory, wherein the memory includes executable instructions that, when executed by the processor, cause the computing device to undertake the steps (Andersen: Page 33: “The operations of the server device 208 may be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (e.g., the memory module 401) and are executed by the processor module 402.”) of:
receiving from an imaging device a first 2D image of a first anatomical site on the body, wherein the first anatomical site has the feature (Andersen: Abstract);
and determining at least one characteristic of an identified feature from the 2D image (Andersen: Page 22: “transforming the image data comprises identifying a first stoma reference indicator on the stomal area. The transformed image data may be based on the first stoma reference indicator. The first stoma reference indicator may be a perimeter of the stoma or other parameters relating to the stoma, e.g. a center of the stoma.”).
Andersen does not disclose expressly: obtaining a second 2D image of a second anatomical site on the body that is free of the feature, wherein the first anatomical site and the second anatomical site are separated from one another on the body; subtracting the second 2D image from the first 2D image to identify the feature in the first 2D image; and determining at least one characteristic of the identified feature from the 2D image after subtraction.
Skladnev discloses: a method for collecting, storing, and displaying dermatological images for monitoring and diagnosing skin conditions and cancers (Skladnev: Abstract), wherein the process comprises: receiving from an imaging device a first 2D image of a first anatomical site on the body, wherein the first anatomical site has the feature; obtaining a second 2D image of a second anatomical site on the body that is free of the feature, wherein the first anatomical site and the second anatomical site are separated from one another on the body (Skladnev: Col 9: Lines 14-19: “To assist the image analysis process of automatically identifying the boundary between clean skin and lesion by providing for each lesion or group of lesions one or more images of adjacent clean skin, to the same scale as that of the lesion image, for use as a statistical skin colour reference”); subtracting the second 2D image from the first 2D image to identify the feature in the first 2D image (Skladnev: Col 36-37: Lines 48-19: “Actual distinction between skin and lesion can be done in a number of ways…
The more difficult case where there is no obvious watershed is shown in FIG. 24B. In this case, it is possible to fit a known histogram shape for one class ( a light”) class, e.g., skin) to the unknown histogram (for the lesion image) and to then subtract it. This leaves a smaller histogram representing the second class ( the dark class, in this case a lesion)…
Returning to FIG. 23, in step 415 the remaining unmasked area of the lesion image is analysed in conjunction with the skin statistics obtained in the previous step to allow a distinction to be drawn between skin and lesion. Subsequent processing then focuses solely on pixels in the lesion area, henceforth referred to as the "lesion area." The distinction is managed in the form of a mask image similar to that used for hair and bubbles.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the method of capturing and subtracting lesion and clean skin images for the extraction of a lesion area taught by Skladnev by capturing and subtracting a user’s clean skin and stomal images for the extraction of the stomal area disclosed by Andersen prior to the determination of stomal characteristics. The suggestion/motivation for doing so would have been “To facilitate automatically (by software) identifying the boundary between clean skin and lesion with accuracy, that is, given information about what clean skin should look like, distinguish between clean skin and a lesion” (Skladnev: Col 9-10: Lines 66-2.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Andersen in view of Skladnev does not disclose expressly: generating a 3D representation of the identified feature; and detecting an abnormality associated with the identified feature before an onset of symptoms in accordance with the 3D representation.
Kim discloses: generating a 3D representation of an identified feature (Kim: 0024: “Meanwhile, the method of the present invention includes: (A) a step of a gray color conversion unit converting a two-dimensional skin surface image captured by a microscope camera into a gray image; (B) a step of a three-dimensional skin surface point generation unit converting an intensity level for each pixel of the gray image into depth data to form three-dimensional skin surface points; (C) a step of a three-dimensional skin surface generation unit connecting the three-dimensional skin surface points into a mesh to form a three-dimensional skin surface image”); and detecting an abnormality associated with the identified feature before an onset of symptoms in accordance with the 3D representation (Kim: 0033-0035: “The developed system takes a single skin image and automatically converts it into a 3D mesh in real time.…Analyze the roughness estimation. The multimodal rendering system provides users (dermatologists or skin specialists) with precise skin roughness information, which is crucial for diagnosing skin diseases, as well as visual investigations for accurate virtual tactile and skin examination.”;
0040: “this system can be a good tool to examine changes in skin condition before and after using skin care products (cosmetics). Furthermore, the proposed 2D skin roughness estimation method can be applied to mobile applications to provide an online roughness evaluation tool with a simple phone camera.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the method for generating a 3D surface image taught by Kim to generate a 3D image of the extracted stoma disclosed by Andersen in view of Skladnev. The suggestion/motivation for doing so would have been “The multimodal rendering system provides users (dermatologists or skin specialists) with precise skin roughness information, which is crucial for diagnosing skin diseases, as well as visual investigations for accurate virtual tactile and skin examination.” (Kim: 0035). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Andersen in view of Skladnev with Kim to obtain the invention as specified in claim 10.
Regarding claim 12, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 10, further including generating contours of the anatomical site using the first 2D image after subtraction (Andersen: Figure 10: BL_1-BL_3; Pages 35-36: “Fig. 10 shows an exemplary second ostomy representation OR_2 based on four appliance image representations as also described in relation to Fig. 9. The second ostomy representation OR_2 comprises a first boundary line BL_ 1 (red line) indicative of a circumference or edge of the adhesive surface of the ostomy appliance…The second ostomy representation OR_2 comprises second boundary line BL_2 (green line) indicative of a circumference or edge of the stomal opening of the adhesive surface…The second ostomy representation OR_2 comprises third boundary line BL_3 (blue line) indicative of a boundary between a discoloured part (output leak) and a non-discoloured part (clean) of the adhesive surface.”; Wherein figure 10 is an image containing boundary lines for different contours of the stoma area, and wherein the 2D image has been subtracted as is taught by Skladnev.).
Regarding claim 13, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 10, further including determining a color of the anatomical site from the first 2D image after subtraction (Andersen: Page 9: “determining one or more image representations based on the image data comprises determining a base color parameter, e.g. including a first base color parameter and/or a second base color parameter, and determining the one or more image representations and/or one or more ostomy parameters based on the base color parameter. The base color parameter may be based on red channel data of the ostomy image data.”; Wherein the image data has been subtracted as is taught by Skladnev.).
Regarding claim 14, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 13, wherein the step of determining a color of the anatomical site comprises determining a redness of the anatomical site or a contralateral skin image (Andersen: Page 9: “determining one or more image representations based on the image data comprises determining a base color parameter, e.g. including a first base color parameter and/or a second base color parameter, and determining the one or more image representations and/or one or more ostomy parameters based on the base color parameter. The base color parameter may be based on red channel data of the ostomy image data.”;
Page 11: “In one or more exemplary methods and/or devices, the one or more image representations comprises a stomal opening image representation, e.g. based on appliance image data and/or transformed appliance image data.”; Wherein the stoma/stomal opening is identified based on red channel data.).
Regarding claim 15, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 10, wherein the anatomical site is a stoma and/or a peristomal skin (Wherein limitation including “and/or” is interpreted as requiring either a stoma or peristomal skin) (Andersen: Abstract).
Regarding claim 17, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 13, wherein determining the color of the anatomical site includes using quantitative redness analysis to identify the stoma in the image (Andersen: Page 9: “determining one or more image representations based on the image data comprises determining a base color parameter, e.g. including a first base color parameter and/or a second base color parameter, and determining the one or more image representations and/or one or more ostomy parameters based on the base color parameter. The base color parameter may be based on red channel data of the ostomy image data.”;
Page 11: “In one or more exemplary methods and/or devices, the one or more image representations comprises a stomal opening image representation, e.g. based on appliance image data and/or transformed appliance image data.”; Wherein the stoma/stomal opening is identified based on red channel data.).
Regarding claim 18, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 17, wherein the quantitative redness analysis comprises analyzing red hue values of the image to identify a range of red objects from a background (Andersen: Page 9: “the method comprises determining the fourth stoma discoloration representation based on red channel data of the image data/stoma image data. In one or more exemplary methods, determining one or more image representations based on the image data comprises determining a base color parameter, e.g. including a first base color parameter and/or a second base color parameter, and determining the one or more image representations and/or one or more ostomy parameters based on the base color parameter. The base color parameter may be based on red channel data of the ostomy image data.”; Wherein the red channel data is used to determine multiple representations/areas within the image.).
Regarding claim 21, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 10, wherein subtracting the second 2D image from the first 2D image results in a 2D representation of the feature (Skladnev: Col 36-37: Lines 48-19: “Actual distinction between skin and lesion can be done in a number of ways…
The more difficult case where there is no obvious watershed is shown in FIG. 24B. In this case, it is possible to fit a known histogram shape for one class ( a light”) class, e.g., skin) to the unknown histogram (for the lesion image) and to then subtract it. This leaves a smaller histogram representing the second class ( the dark class, in this case a lesion)…
Returning to FIG. 23, in step 415 the remaining unmasked area of the lesion image is analysed in conjunction with the skin statistics obtained in the previous step to allow a distinction to be drawn between skin and lesion. Subsequent processing then focuses solely on pixels in the lesion area, henceforth referred to as the "lesion area." The distinction is managed in the form of a mask image similar to that used for hair and bubbles.”)
and wherein generating the 3D representation of the feature comprises developing a height associated with a pixel of the 2D representation in accordance with a grayscale intensity associated with the pixel (Kim: 0024: “Meanwhile, the method of the present invention includes: (A) a step of a gray color conversion unit converting a two-dimensional skin surface image captured by a microscope camera into a gray image; (B) a step of a three-dimensional skin surface point generation unit converting an intensity level for each pixel of the gray image into depth data to form three-dimensional skin surface points; (C) a step of a three-dimensional skin surface generation unit connecting the three-dimensional skin surface points into a mesh to form a three-dimensional skin surface image”).
Claim(s) 4 & 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Andersen in view of Skladnev and Kim, and further in view of Crawford et al. (US 2021/0104043 Al) hereinafter referenced as Crawford.
Regarding claim 4, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 3.
Andersen in view of Skladnev and Kim does not disclose expressly: wherein the step of determining at least one dimension includes utilizing a bounding box method to identify a height and a width of the identified feature in a real time.
Crawford discloses: a real-time method (Crawford: 0048: “the skin abnormality monitoring system 100 utilizes deep neural network models optimized to run on mobile devices to automatically assess and screen patient images prior to clinician review.”) for detecting skin anomalies in an image by detecting key points in an image, cropping and classifying images of the key points, and determining parameters of the key points in the cropped images, such as contours, height, and width (Crawford: Figure 13: 1314; 0086: “Using the fine-tuned model, a new candidate keypoint with no prior knowledge of type association can be processed and assigned with a predicted classification label along with a confidence level. Given the coordinates of a new keypoint, the neighbor pixels around the keypoint are cropped in step 1102 and processed as the input data to the fine-tuned neural network”;
0090: “Once keypoints are located and classified, geometrical properties such as the height, width, perimeter and area are computed.”;
0094: “The contour of the moles/anomalies is determined using, for example, the built-in OpenCV function, findcontour, in step 1320. OpenCV provides the properties of this contour such as the perimeter and the area. Finally, the height and width are determined by enclosing the contour in an ellipse, of which the major and minor axes correspond to the height and width, respectively, in step 1322. The output of this process yields parameter values in pixels, which are then converted to millimeters using the previously calculated pixel-to-mm ratio in step 1326. The results include the width, height, and area properties in mm.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to substitute the method for determining stoma reference indicators disclosed by Andersen in view of Skladnev and Kim with the algorithms for key point detection, cropping, and perimeter detection disclosed by Crawford. The suggestion/motivation for doing so would have been to reduce the computation time and power required by minimizing the size of each processed image (Crawford: 0068). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Andersen in view of Skladnev and Kim with Crawford to obtain the invention as specified in claim 4.
Regarding claim 11, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 10.
Andersen in view of Skladnev and Kim does not disclose expressly: wherein the step of determining at least one characteristic is performed utilizing a bounding box method to identify a height and a width of the anatomical site in a real time.
Crawford discloses: a real-time method (Crawford: 0048: “the skin abnormality monitoring system 100 utilizes deep neural network models optimized to run on mobile devices to automatically assess and screen patient images prior to clinician review.”) for detecting skin anomalies in an image by detecting key points in an image, cropping and classifying images of the key points, and determining parameters of the key points in the cropped images, such as contours, height, and width (Crawford: Figure 13: 1314; 0086: “Using the fine-tuned model, a new candidate keypoint with no prior knowledge of type association can be processed and assigned with a predicted classification label along with a confidence level. Given the coordinates of a new keypoint, the neighbor pixels around the keypoint are cropped in step 1102 and processed as the input data to the fine-tuned neural network”;
0090: “Once keypoints are located and classified, geometrical properties such as the height, width, perimeter and area are computed.”;
0094: “The contour of the moles/anomalies is determined using, for example, the built-in OpenCV function, findcontour, in step 1320. OpenCV provides the properties of this contour such as the perimeter and the area. Finally, the height and width are determined by enclosing the contour in an ellipse, of which the major and minor axes correspond to the height and width, respectively, in step 1322. The output of this process yields parameter values in pixels, which are then converted to millimeters using the previously calculated pixel-to-mm ratio in step 1326. The results include the width, height, and area properties in mm.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to substitute the method for determining stoma reference indicators disclosed by Andersen in view of Skladnev and Kim with the algorithms for key point detection, cropping, and perimeter detection disclosed by Crawford. The suggestion/motivation for doing so would have been to reduce the computation time and power required by minimizing the size of each processed image (Crawford: 0068). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Andersen in view of Skladnev and Kim with Crawford to obtain the invention as specified in claim 11.
Claim(s) 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Andersen in view of Skladnev and Kim, and further in view of Gopalakrishnan et al. (Salient Region Detection by Modeling Distributions of Color and Orientation) hereinafter referenced as Gopalakrishnan.
Regarding claim 19, Andersen in view of Skladnev and Kim discloses: The imaging method of claim 17.
Andersen in view of Skladnev and Kim does not disclose expressly: wherein the quantitative redness analysis uses a gaussian fit of redness peaks to generate parameters related to the stoma.
Gopalakrishnan discloses: a method for detection of salient regions in an image based on the fitting of an EM algorithm based on dominant hues selected as based on the maxima in a histogram (Gopalakrishnan: II. COLOR SALIENCY FRAMEWORK: A. Dominant Hue Extraction: “The initialization problem is solved by identifying the dominant hues in the image using a method similar to [26] …The maxima in the final histogram are the dominant hues and the number of dominant hues is set as the number of Gaussian clusters for the EM algorithm. The range of hues around a peak in the hue histogram constitutes a hue band for the corresponding hue. The local minimum (valley) between two dominant hues in the hue histogram is used to identify the boundary of the hue band.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the algorithm for the detection of salient regions based on dominant hues taught by Gopalakrishnan into Andersen in view of Skladnev and Kim by detecting the stoma’s dominant red hue band prior to processing the image with the mask generating CNN. The suggestion/motivation for doing so would have been “The relative spread of a cluster is quantified by the mean of the distance of the component pixels of the cluster to the centroids of the other clusters. With the calculation of relative spread, we can not only eliminate the background colors of large spatial variance, but also bias the colors occurring towards the center of the image with more importance, in case the other competing colors in the image possess similar spatial variance.” (Gopalakrishnan: C. Cluster Compactness (Spatial Domain): SECTION II. Color Saliency Framework; Wherein the biasing towards the colors towards the center allows for greater control over the hue band used for stoma detection). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Andersen in view of Skladnev and Kim with Gopalakrishnan to obtain the invention as specified in claim 19.
Regarding claim 20, Andersen in view of Skladnev, Kim, and Gopalakrishnan discloses: The imaging method of claim 19, wherein the parameters comprise a shape and perimeter of the stoma (Andersen: Figure 10; Page 36: “The second ostomy representation OR_2 comprises second boundary line BL_2 (green line) indicative of a circumference or edge of the stomal opening of the adhesive surface, wherein the second boundary line is based on the second appliance image representation and/or the fourth appliance image representation.”; Wherein the stoma opening boundary line generated from the stoma masks constitutes a shape and perimeter of the stoma.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights