DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 03/07/2025 have been considered and placed in the applicant file.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim 1 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by NGUYEN et al. (US 12118723 B1), hereinafter referenced as NGUYEN.
Regarding claim 1, NYUGEN explicitly teaches a method of analyzing digital dermoscopy images (Fig. 23. Column [29], Lines [14-27]-NYUGEN discloses referring now to FIG. 23, therein is shown is a flow chart of a method of operation 2300 of a compute system 100. At Column [13], Lines [64-68]-NYUGEN discloses referring now to FIG. 1, therein is shown an example of a system architecture diagram of a compute system 100. The compute system 100 provide standardized and objective skin cancer detection across 10 classifications and 122 sub-classes (wherein the compute system may be one or more mobile/non-mobile computing devices that executes software and/or web-based applications). Please also see Fig. 1 and 20), comprising:
receiving a digital dermoscopy image (Fig. 1. Column [29], Lines [14-27]-NYUGEN discloses the method 2300 includes: receiving a patient image in a block 2302. Further at Column [16], Lines [01-08]-NYUGEN discloses patient images 114 are taken and uploaded by the patient and reviewed by a cancer artificial intelligence (AI) module 118 and the clinician. A patient launches the image based skin cancer detection mechanism via the mobile application and logs into the patient's account. The compute system 100 can be prompted to upload or take the patient images 114 of the patient's body or body parts to be analyzed by the cancer AI module 118. Please also see Fig. -3-5, 20 and 23, and read Column [03], Lines [12-18] and Column [18], Lines [34-41]);
PNG
media_image1.png
317
904
media_image1.png
Greyscale
Figure 2 illustrates dermoscopy images that may be obtained from a patient or database, such as the ISIC database, which includes categories for melanoma (MEL), melanocytic nevus (NV), basal cell carcinoma (BCC), actinic keratosis (AK), benign keratosis including solar lentigo, seborrheic keratosis and lichen planus-like keratosis (BKL), dermatofibroma (DF), vascular lesion (VASC), squamous cell carcinoma (SCC), and unknown (UNK).
determining if the digital dermoscopy image meets a quality threshold (Fig. 23. Column [29], Lines [14-27]-NYUGEN discloses the method 2300 includes: segmenting a skin lesion in the patient image in a block 2304; constructing a normalized image by cropping the patient image and adding padding to position the skin lesion at the center of the normalized image in a block 2306. Further at Column [16], Lines [01-08]-NYUGEN discloses the compute system 100 can guide a patient on photo guidelines for the patient images 114 and accepts or rejects the patient images 114 for retake based on a pre-specified criteria, e.g., distance, quality, blur, or a combination thereof. The patient images 114 can be selected and processed based on the images uploaded by the user 112. Please also see Fig. 1, 6 and 20, and read Column [19], Lines [45-65] (wherein images may be assessed for quality before use in machine learning model training));
when the digital dermoscopy image meets the quality threshold (Fig. 23. Column [16], Lines [09-30]-NYUGEN discloses once the patient images 114, as required for analysis, are successfully uploaded, the compute system 100 can send or load the patient images 114 to a skin cancer module 116 for analysis. Further at Column [29], Lines [14-27]-NYUGEN discloses identifying, by a cancer artificial intelligence (AI) already trained, a skin cancer classification, a skin cancer sub-class, and a risk level assessment in a block 2308), determining if the digital dermoscopy image potentially shows a pre-malignant or malignant skin lesion or an atypical melanocytic lesion (Fig. 23. Column [14], Lines [01-41]-NYUGEN discloses the 10 classifications and 122 sub-classes include: MEL: including all types of malignant melanoma such as melanoma in situ, superficial spreading melanoma, nodular melanoma, etc., BCC: including all types of basal cell carcinoma such as superficial basal cell carcinoma, nodular basal cell carcinoma, basosquamous carcinoma, ulcerated basal cell carcinoma, etc. EPI: including all types of epidermal pre-malignant and malignant tumors, MALO: This class contains Merkel cell carcinoma, kaposi sarcoma, dermatofibrosarcoma protuberans, etc. NV: including all types of melanocytic nevus and melanosis. DF: including all types of dermatofibroma BAL: including all types of benign adnexal or appendage lesions, BKL: including all types of benign keratinocytic lesions and lentigines, VASC: including all types of benign vascular lesions and haemorrhages. BENO: including all benign lesions that are not in NV, DF, BAL, BKL and VASC. Moreover, special sub-classes are separated from others to facilitate the estimation of cancer risk level. Spitz nevus is separated from other benign nevi in the class NV) with uncertain malignant potential (Fig. 23. Column [16], Lines [31-43]-NYUGEN discloses the risk level assessment 122 can include a risk level of zero indicating no cancer risk was detected. The risk level assessment 122 of risk level one indicating a precautionary warning, but not active cancer was detected. The risk level 122 can include a risk level two indicating a minor detection of cancer or pre-cancer was detected. The risk level 122 can include a risk level three indicating the detection of a significant risk of cancer has been detected. The risk level 122 can include a risk level four can indicate a risk of melanoma in situ and or non-melanoma skin cancer. The risk level 122 can also include a risk level five indicating a high risk of invasive melanoma or other high grade skin cancers. Please also see claim 3, 10, 17), wherein the determination of the potential showing of the pre-malignant or malignant skin lesion or the atypical melanocytic lesion with uncertain malignant potential is performed by an artificial intelligence system or an augmented intelligence system (Fig. 1, #116, #117, #118, and #119 called a skin cancer module, segmentation module, cancer AI module, and classification module, respectively. Column [02], Lines [34-35]-NYUGEN discloses FIG. 1 is an example of a system architecture diagram of a compute system with an image based skin cancer detection mechanism (wherein the skin cancer module includes segmentation module, cancer AI module, and classification module). Further at Column [16], Lines [09-30]-NYUGEN discloses the segmentation module 117 can be a hardware structure managed by software that can identify the perimeter of a lesion identified in the patient image 114 in order to standardize the images being processed. A cancer AI module 118 can be a machine learning or artificial intelligence structure configured to analyze the images provided by the segmentation module 117 and to generate a classification model 119 to identify a skin cancer classification 120 and a risk level assessment 122) pre-trained (Fig. 1. Column [17], Lines [41-61]-NYUGEN discloses the segmentation module 117 of FIG. 1 is a hardware structure managed by software and trained on the patient images 114 to automatically detect and segment a skin lesion 208. At Column [18], Lines [11-15]-NYUGEN discloses the configuration of classification model 119 of FIG. 1 is trained to classify a given one of the skin lesion 208 of FIG. 2 into 10 different classes as MEL, BCC, EPI, MALO, NV, DF, BAL, BKL, VASC, or BENO. The classification model 119 can handle multi-label problem that is one image that can belong to two or more classes (collision), type of images that is dermoscopic, macro, or irrelevant. At Column [18], Lines [50-55]-NYUGEN discloses the compute system 100, the cancer AI module 118, or a combination thereof can be trained to identify each sub-class individually) with a dataset of diverse dermoscopy images (Fig. 1. Column [21], Lines [33-45]-NYUGEN discloses the compute system 100 of FIG. 1 with the image based skin cancer detection mechanism, has the following three distinctive features: a large, diversified learning dataset of about 100 thousand images, a granular hierarchical class/sub-class classification, and innovative loss functions. The hierarchical classification system helps the cancer AI 118 of FIG. 1 learn better and become more robust. At Column [17], Lines [41-61]-NYUGEN discloses the segmentation module 117 was trained using ISIC dataset. At Column [20], Lines [01-06]-NYUGEN discloses the classification model 119 was trained using ISIC dataset. At Column [02], Lines [38-49]-NYUGEN discloses An ISIC classification system for skin cancer detection contains nine classes: melanoma (MEL), melanocytic nevus (NV), basal cell carcinoma (BCC), actinic keratosis (AK), benign keratosis including solar lentigo, seborrheic keratosis and lichen planus-like keratosis (BKL), dermatofibroma (DF), vascular lesion (VASC), squamous cell carcinoma (SCC), and unknown (UNK). This system covers the most common malignant and benign classes of skin lesions. It is easy for doctors to annotate, and also more informative than the binary classification “malignant vs benign”. Please also see Fig. 6 and 23, and read Column [18], Lines [50-65]); and
PNG
media_image2.png
724
1432
media_image2.png
Greyscale
Figure 1 illustrates a skin lesion analysis and clinical decision-making support system that uses multiple pre-trained artificial intelligence modules.
visually indicating to a user results of the determination (Fig. 23. Column [16], Lines [43-49]-NYUGEN discloses based on analysis results, the compute system 100 can display information to the patient. At Column [21], Lines [55-62]-NYUGEN discloses referring now to FIG. 12, therein are shown an example of a skin cancer display 1201 for analysis of a Melanoma cancer 1202 as performed by cancer AI 118 of FIG. 1. Please also see Fig. 11-13 and 17-18).
PNG
media_image3.png
1051
1688
media_image3.png
Greyscale
Figure 12 illustrates a skin cancer display 1201 for melanoma performed by the cancer AI model #118 that visually depicts the risk level assessment 122, the skin cancer classification 120, and the skin cancer sub-class 1004.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over NGUYEN et al. (US 12118723 B1), hereinafter referenced as NGUYEN in view of GAMAGE et al. (Gamage L, Isuranga U, Meedeniya D, De Silva S, Yogarajah P. Melanoma Skin Cancer Identification with Explainability Utilizing Mask Guided Technique. Electronics. January 2024; 13(4):680. https://doi.org/10.3390/electronics13040680), hereinafter referenced as GAMAGE.
Regarding claim 2, NYUGEN explicitly teaches a method of analyzing digital dermoscopy images as recited in claim 1, although NYUGEN explicitly teaches further comprising generating a digital dermoscopy image, wherein the digital dermoscopy image represent differing degrees of likelihood of positive dermoscopy features associated with possible pre-malignant and/or malignant tissue or atypical melanocytic nevi (Fig. 23. Column [14], Lines [01-41]-NYUGEN discloses the 10 classifications and 122 sub-classes include: MEL: including all types of malignant melanoma such as melanoma in situ, superficial spreading melanoma, nodular melanoma, etc., BCC: including all types of basal cell carcinoma such as superficial basal cell carcinoma, nodular basal cell carcinoma, basosquamous carcinoma, ulcerated basal cell carcinoma, etc. EPI: including all types of epidermal pre-malignant and malignant tumors, MALO: This class contains Merkel cell carcinoma, kaposi sarcoma, dermatofibrosarcoma protuberans, etc. NV: including all types of melanocytic nevus and melanosis. DF: including all types of dermatofibroma BAL: including all types of benign adnexal or appendage lesions, BKL: including all types of benign keratinocytic lesions and lentigines, VASC: including all types of benign vascular lesions and haemorrhages. BENO: including all benign lesions that are not in NV, DF, BAL, BKL and VASC. Moreover, special sub-classes are separated from others to facilitate the estimation of cancer risk level. Spitz nevus is separated from other benign nevi in the class NV. Further at Column [16], Lines [31-43]-NYUGEN discloses the risk level assessment 122 can include a risk level of zero indicating no cancer risk was detected. The risk level assessment 122 of risk level one indicating a precautionary warning, but not active cancer was detected. The risk level 122 can include a risk level two indicating a minor detection of cancer or pre-cancer was detected. The risk level 122 can include a risk level three indicating the detection of a significant risk of cancer has been detected. The risk level 122 can include a risk level four can indicate a risk of melanoma in situ and or non-melanoma skin cancer) of uncertain malignant potential (Fig. 23. Column [16], Lines [31-43]-NYUGEN discloses the risk level assessment 122 can include a risk level of zero indicating no cancer risk was detected. The risk level assessment 122 of risk level one indicating a precautionary warning, but not active cancer was detected. The risk level 122 can include a risk level two indicating a minor detection of cancer or pre-cancer was detected. The risk level 122 can include a risk level three indicating the detection of a significant risk of cancer has been detected. The risk level 122 can include a risk level four can indicate a risk of melanoma in situ and or non-melanoma skin cancer. The risk level 122 can also include a risk level five indicating a high risk of invasive melanoma or other high grade skin cancers. Please also see claim 3, 10, 17 and read Column [03], Lines [12-18]).
NYUGEN fails to explicitly teach further comprising generating a digital dermoscopy image heatmap overlay, wherein different colors shown in the digital dermoscopy image heatmap overlay represent differing degrees of likelihood of positive dermoscopy features.
However, GAMAGE explicitly teaches further comprising generating a digital dermoscopy image heatmap overlay (Fig. 20. Page [02], Paragraph [02]-GAMAGE discloses this study presents a computational model for melanoma identification using a deep learning model with transfer learning and XAI. The main focus of this study is to achieve high performance in skin image classification and show the model’s explainability to increase the trustworthiness of the proposed solution. The explainable heatmaps are based on gradient-weighted class activation mapping (Grad-CAM) [7] and Grad-CAM++ [8]. At page [23], Paragraph [01]-GAMAGE discloses we have deployed the proposed model as a support tool named LU Bio Vision and a sample GUI is shown in Figure 20. The web application provides a user-friendly interface to test a skin image for melanoma conditions. The application allows to upload a skin image. Here, the user can select the proposed CNN model or the ViT-based model to classify the image. Then the system generates the corresponding heatmaps together with the predicted class with the accuracy probability. Additionally, users can generate and download the medical report in PDF format with dermatologist feedback (wherein the HAM10000 and ISIC datasets are used for model training, which contains dermoscopic images for 7 categories of skin lesions, including Nevi and melanoma, and dermoscopic attribute masks for pigment networks, negative network, streaks, and globules). Please also see Fig. 2 and 17), wherein different colors shown in the digital dermoscopy image heatmap overlay represent differing degrees of likelihood of positive dermoscopy features (Fig. 17. Page [20], Paragraph [01]-GAMAGE discloses these results affirm the performance improvements of our SM-ViT model when compared to the baseline ViT model, highlighting its effectiveness in accurately classifying melanoma and underscoring its potential for enhancing medical image identification tasks. Both the SM-ViT and Baseline ViT models have demonstrated higher accuracy in identifying Nevi lesions compared to melanoma. Therefore, both models exhibit good sensitivity when it comes to Nevi detection. Further at Page [20], Paragraph [02]-GAMAGE discloses we generated heatmaps for the images utilizing XAI techniques, namely, Grad-CAM and Grad-CAM++. At Page [20], Paragraph [03]-GAMAGE discloses we evaluated the performance of our model against ground truth and generated maps using the ISIC 2017 binary mask dataset and ISIC 2018 attribute mask dataset. The color range in the explainable heatmap indicates the regions that have contributed more and less to the classification, using red and blue, respectively. The attribute masks corresponding to each image are also used to validate the explanations by comparing them with the predicted heatmap outputs. Please also see Fig. 2 and 20).
PNG
media_image4.png
209
822
media_image4.png
Greyscale
FIGURE 17 illustrates a comparison of explainability heat maps using neural network models and a color range for quantifying the contributions of areas.
PNG
media_image5.png
899
943
media_image5.png
Greyscale
FIGURE 2 illustrates a high-level architecture for melanoma/nevi identification that uses artificial intelligence models (e.g. a CNN) for classification, multiple diverse training databases (e.g. ISIC, HAM10000) containing dermoscopic images and color coded heat maps for explaining the level of contribution to each classification.
PNG
media_image6.png
964
933
media_image6.png
Greyscale
FIGURE 20 illustrates a report generated for a skin lesion with explainability heatmaps using a support tool named LU Bio Vision and a sample GUI.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of NGUYEN of having a method of analyzing digital dermoscopy images, with the teachings of GAMAGE of having further comprising generating a digital dermoscopy image heatmap overlay, wherein different colors shown in the digital dermoscopy image heatmap overlay represent differing degrees of likelihood of positive dermoscopy features.
Wherein NGUYEN’s method having further comprising generating a digital dermoscopy image heatmap overlay, wherein different colors shown in the digital dermoscopy image heatmap overlay represent differing degrees of likelihood of positive dermoscopy features associated with possible pre-malignant and/or malignant tissue or atypical melanocytic nevi of uncertain malignant potential.
The motivation behind the modification would have been to obtain a method that improves the analysis of skin lesions and detection of skin cancer, since both NGUYEN and GAMAGE concern skin lesion detection and dermoscopy image analysis. Wherein NGUYEN’s provides improves the automatic detection of skin cancer using deep learning and image analysis, while GAMAGE’s systems and methods provide improvements to the training and accuracy of machine learning models for skin lesion detection. Please see NGUYEN et al. (US 12118723 B1), Abstract and GAMAGE et al. (Gamage L, Isuranga U, Meedeniya D, De Silva S, Yogarajah P. Melanoma Skin Cancer Identification with Explainability Utilizing Mask Guided Technique. Electronics. January 2024; 13(4):680. https://doi.org/10.3390/electronics13040680), Abstract.
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over NGUYEN et al. (US 12118723 B1), hereinafter referenced as NGUYEN in view of MISHRA et al. (US 20210209754 A1), hereinafter referenced as MISHRA.
Regarding claim 3, NYUGEN explicitly teaches the method of analyzing digital dermoscopy images as recited in claim 1, NYUGEN fails to explicitly teach wherein the determination of the potential showing of the pre-malignant or malignant skin lesion or the atypical melanocytic nevus of uncertain malignant potential further comprises feature extraction from the digital dermoscopy image, wherein features are analyzed by the artificial intelligence system or the augmented intelligence system for a presence of parameters selected from the group consisting 1 of asymmetry, atypical network, blue-white-grey-violet structures, radial streams, pseudopods, irregular diffuse pigmentation, irregular dots and globules, regression patterns, and combinations thereof.
However, MISHRA explicitly teaches wherein the determination of the potential showing of the pre-malignant or malignant skin lesion or the atypical melanocytic nevus of uncertain malignant potential (Fig. 9. Paragraph [0133]-MISHRA discloses FIG. 9 is a flow diagram depicting an illustrative method 200 of diagnosing melanoma in dermoscopy or other images (wherein method 200 is performed by digital dermoscopy system 100 depicted in FIG. 8, which includes an image analyzer #104 with each model or component for lesion analysis and the imaga analyzer #104 may be a mobile computing device). The method 200 include receiving an image from image source (block 202), pre-processing the image (block 204) (wherein the image may also be segmented following pre-processing by segmenter #116), performing median color split (block 206), performing vessel detection (block 208), performing atypical network detection (block 210), performing salient point detection (block 212), performing color detection (block 214), providing a handcrafted analysis result (block 216), performing deep learning feature detection (block 218), providing a deep learning analysis result (block 220), and/or providing an overall analysis result (block 222)). In paragraph [0080]-MISHRA discloses the method includes identifying lesion change and/or concerning lesion-related information of significance in melanoma and other skin cancer diagnosis. In paragraph [0078]-MISHRA discloses the method includes diagnosing benign melanocytic nevi, which may show a visible pigment network that is fairly symmetric and regular, and/or diagnosing irregular or atypical pigment network (APN). Please also see Fig. 8 and read paragraph [0099-0101]) further comprises feature extraction from the digital dermoscopy image (Fig. 8, #102 called an image. Paragraph [0117]-MISHRA discloses the image analyzer 104 receives an image 112 from the image source 102 and analyzes the image 112 to facilitate diagnosis of a skin affliction such as BCC, SCC, or melanoma. In paragraph [0060]-MISHRA discloses as used herein, the term “dermoscopy” refers to a body imaging technique that involves viewing skin lesions with 8x or more magnification (wherein the technique involves limiting surface reflectance through the use of, for example, a fluid, gel, mineral oil, or alcohol between the skin and a glass plate, or by using cross polarized light for illumination). The term “dermoscopy image” refers to a photograph of a skin lesion using a dermoscopy technique (wherein the image analyzer 104 may be accessible as a web-based service, the image source 102 may be, for example, a smartphone camera, specialized dermoscopy imaging platform, or digital camera with an add-on device with 8-30 magnification, and images may be digital photographs, machine vision image files, etc.). Please also see Fig. 9), wherein features are analyzed by the artificial intelligence system or the augmented intelligence system (Fig. 8, #118, #120 and #138 called a Handcrafted Feature Component, a Deep Learning Feature Component and an Overall Classifier, respectively. Paragraph [0126]-MISHRA discloses the system 100 includes a handcrafted feature component 118, a deep learning feature component 120, and an overall classifier 138. The handcrafted feature component 118 is configured to provide a first analysis result (e.g., a first determination decision or prediction of melanoma), the deep learning feature component 120 is configured to provide a second analysis result (e.g., a second determination decision or prediction of melanoma), and the overall classifier 138 is configured to provide an overall analysis result (e.g., an overall determination decision or prediction of melanoma) based on the first and second analysis results (wherein the method includes extracting morphological features, extracting texture features, and/or extracting color features). Further in paragraph [0094]-MISHRA discloses the method includes using a deep learning (DL) technique which may rely upon transfer learning of a deep residual network or ResNet. Please also read paragraph [0064, 0095-0097, and 0138]) for a presence of parameters selected from the group consisting 1 of asymmetry (Fig. 9. Paragraph [0128]-MISHRA discloses the atypical pigment network detection model 126 is configured for (e.g., when executed by the processor 106) determining asymmetry based on eccentricity of detected blocks. Please also read paragraph [0078 and 0085]), atypical network (Fig. 1. Paragraph [0078]-MISHRA discloses the method includes diagnosing benign melanocytic nevi, which may show a visible pigment network that is fairly symmetric and regular, and/or diagnosing irregular or atypical pigment network (APN), which may have pigment network whose structure varies in size and/or shape. APN areas having relatively high variance in the red and relative-red color planes. The method includes using a green-to-blue threshold for each pixel to remove the false positive granular structures that were detected as APN. A plurality (e.g., 52) of features for APN include color and texture features as for pink areas. FIG. 5A is a sample image showing APN areas. FIG. 5B is a version of the sample image of FIG. 5A, in which an APN overlay is applied. Please also read paragraph [0153-0159]), blue-white-grey-violet structures (Fig. 9. Paragraph [0128]-MISHRA discloses the atypical pigment network detection model 126 is configured for detecting areas having relatively high variance in the red and relative-red color planes. The atypical pigment network detection model 126 is configured for (e.g., when executed by the processor 106) applying a green-to-blue ratio threshold to remove false positive granular structures detected as atypical pigment network. The removed blocks may be retained and size-filtered in the blue plane to find small blue-gray peppering (granularity). As granularity is a strong indicator of early melanoma, detecting this granularity allows for early diagnosis of melanoma. Further in paragraph [0130]-MISHRA discloses the color detection model 130 is configured to detect a pink shade and/or a pink blush (e.g., semi-translucency). The pink shade may be dark pink, light pink, or pink-orange. In paragraph [0137]-MISHRA discloses the method 200 includes performing white area detection. Performing white area detection includes identifying white area features, such as size, quantity, lesion decile ratios, location, eccentricity, dispersion, and/or irregularity of the white area. In paragraph [0139]-MISHRA discloses determining white area features includes determining threshold values, such as threshold values of red, green, and blue for marking white areas for an image. Please also read paragraph [0159]), radial streams (Fig. 9. Paragraph [0153]-MISHRA discloses performing atypical network (e.g., pigment network) detection (block 210), as a handcrafted analysis technique, may include determining areas with highest variance in the red plane, which may help determine features such branch streaks, radial streaming, and thickened and irregular lines, all indicative of an irregular pigment network. Please also read paragraph [0071]), pseudopods (Fig. 9. Paragraph [0153]-MISHRA discloses performing atypical network (e.g., pigment network) detection (block 210), as a handcrafted analysis technique, which may help determine features such as pseudopods), irregular diffuse pigmentation (Fig. 9. Paragraph [0078]-MISHRA discloses the method includes diagnosing irregular or atypical pigment network (APN), which may have pigment network whose structure varies in size and/or shape. For example, irregular wide or dark APN aberrations may appear as a brown mesh, a black mesh, a gray mesh, and/or thick lines in dermoscopy images (wherein APN may extract morphological features, color features, and texture features, which includes a uniformity index and extracting color features of includes extracting average intensity and standard deviation of red, green and blue). Further in paragraph [0071]-MISHRA discloses as shown in FIGS. 3A-3C, the method is configured to capture varied and/or radial symmetry of colors (e.g., brown colors) on an image (e.g., FIG. 3A). In paragraph [0085]-MISHRA discloses the method includes identifying white and/or scar-like depigmented areas, such as ones that may represent the process of regression, such as ones that are indicators (e.g., important) of melanoma and other skin cancers, such as indicators of a response of the immune system to melanoma and other skin cancers. The method includes detecting hypopigmented areas that are peripheral and/or symmetric, which may be characteristic of benign lesions, such as of a dysplastic nevus. Please also read paragraph [0082-0084, 0086-0093, and 0156-0159]), irregular dots and globules (Fig. 9. Paragraph [0144]-MISHRA discloses determining white area features includes determining white area globule features (e.g., by running a globule feature code), which may include determining a binary feature mask, determining a lesion centroid co-ordinate, and determining lesion area. In certain embodiments, automatically detecting white area within a lesion includes determining an average eccentricity feature, a relative size of all white areas compared to lesion area, a relative size of largest white area compared to lesion area, an absolute size of the largest white area, a number of marked white areas per unit lesion area, an average border irregularity of all white areas, and/or a white area dispersement index. The white area globule features are also computed inside the lesion and outside the lesion separately. Please also read paragraph [0078, 0128, and 0156-0159]), regression patterns, and combinations thereof (Fig. 9. Paragraph [0103]-MISHRA discloses the method includes using the feature vectors of each of the six image processing modules including median split of colors, atypical pigment network, salient points, white color, pink colors, and vascular blush, along with the features from the clinical information module to create seven individual logistic regression models, which may include determining a binary feature mask, determining a lesion centroid co-ordinate, and determining lesion area. Further in paragraph [0152]-MISHRA discloses determining white area features includes applying a logistic regression or logistic model or logit model, such as to help generate a best-f it model to differentiate melanoma and other skin cancers from benign lesions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of NGUYEN in view of MISHRA of having a method of analyzing digital dermoscopy images, with the teachings of MISHRA of having wherein the determination of the potential showing of the pre-malignant or malignant skin lesion or the atypical melanocytic nevus of uncertain malignant potential further comprises feature extraction from the digital dermoscopy image, wherein features are analyzed by the artificial intelligence system or the augmented intelligence system for a presence of parameters selected from the group consisting 1 of asymmetry, atypical network, blue-white-grey-violet structures, radial streams, pseudopods, irregular diffuse pigmentation, irregular dots and globules, regression patterns, and combinations thereof.
Wherein NGUYEN’s method having wherein the determination of the potential showing of the pre-malignant or malignant skin lesion or the atypical melanocytic nevus of uncertain malignant potential further comprises feature extraction from the digital dermoscopy image, wherein features are analyzed by the artificial intelligence system or the augmented intelligence system for a presence of parameters selected from the group consisting 1 of asymmetry, atypical network, blue-white-grey-violet structures, radial streams, pseudopods, irregular diffuse pigmentation, irregular dots and globules, regression patterns, and combinations thereof.
The motivation behind the modification would have been to obtain a method that improves the analysis of skin lesions and detection of skin cancer, since both NGUYEN and MISHRA concern skin lesion detection and dermoscopy image analysis. Wherein NGUYEN’s systems and methods improves the automatic detection of skin cancer using deep learning and image analysis, while MISHRA’s systems and methods improve the identification of melanoma and other skin cancer in a dermoscopy images using machine learning models. Please see NGUYEN et al. (US 12118723 B1), Abstract and MISHRA et al. (US 20210209754 A1), Abstract and Paragraph [0059-0063].
Regarding claim 4, NYUGEN in view of MISHRA explicitly teaches the method of analyzing digital dermoscopy images as recited in claim 3, NYUGEN fails to explicitly teach wherein the irregular dots and globules comprise round and/or oval structures.
However, MISHRA explicitly teaches wherein the irregular dots and globules comprise round and/or oval structures (Fig. 9. Paragraph [0144]-MISHRA discloses determining white area features includes determining white area globule features (e.g., by running a globule feature code), which may include determining a binary feature mask, determining a lesion centroid co-ordinate, and determining lesion area. In certain embodiments, automatically detecting white area within a lesion includes determining an average eccentricity feature, a relative size of all white areas compared to lesion area, a relative size of largest white area compared to lesion area, an absolute size of the largest white area, a number of marked white areas per unit lesion area, an average border irregularity of all white areas, and/or a white area dispersement index. The white area globule features are also computed inside the lesion and outside the lesion separately. Therefore, it would have been obvious to a person of ordinary skill in the art for the irregular dots and globules to comprise round and/or oval structures. Although MISHRA explicitly teaches detecting irregular dots and globules as well as the size, shape, radial symmetry/asymmetry and uniformity of skin lesion structures, MISHRA is silent on globules and irregular dots comprising round and/or oval structures. However, it is well-known that globules and irregular dots are defined as generally round and/or oval structures, including within the field of dermatology. Thus, by further defining irregular dots and globular structures to be in line with conventional understandings, the accuracy of skin lesion detection and diagnosis can be improved. Please also read paragraph [0078, 0128, and 0156-0159]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of NGUYEN in view of MISHRA of having a method of analyzing digital dermoscopy images, with the teachings of MISHRA of having wherein the irregular dots and globules comprise round and/or oval structures.
Wherein NGUYEN’s method having wherein the irregular dots and globules comprise round and/or oval structures.
The motivation behind the modification would have been to obtain a method that improves the analysis of skin lesions and detection of skin cancer, since both NGUYEN and MISHRA concern skin lesion detection and dermoscopy image analysis. Wherein NGUYEN’s systems and methods improves the automatic detection of skin cancer using deep learning and image analysis, while MISHRA’s systems and methods improve the identification of melanoma and other skin cancer in a dermoscopy images using machine learning models. Please see NGUYEN et al. (US 12118723 B1), Abstract and MISHRA et al. (US 20210209754 A1), Abstract and Paragraph [0059-0063].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
OKA et al. (US 20080275315 A1)- A major object of this invention is to provide a remote diagnosis apparatus, a remote diagnosis system, a user terminal, a program, a diagnosis program, and a storage for easily diagnosing a pigmentary deposition portion. A user of the system picks up the skin having benign nevus pigmentosus that might be melanoma through a dermoscope using a portable telephone with a camera to which the dermoscope is attached by means of an adapter, accesses to the Internet using an Internet connection function of the portable telephone, and sends the picked up skin image to the remote diagnosis apparatus. After receiving the skin image, the remote diagnosis apparatus uses the melanoma diagnosis program to examine the skin image for melanoma and a disease stage of melanoma if there is melanoma and then sends a result to the user............................ Please see Fig. 2-3. Abstract.
PATWARDHAN (US 20190298252 A1)- Methods and apparatuses are disclosed for assessing pigmentation of skin based on images thereof. In disclosed implementations, a cross-polarized blue image of skin is obtained and processed to extract pigment distribution information. Useful applications include assessing depigmentation, as in the skin condition vitiligo, as well as assessing re-pigmentation such as due to treatment. In addition to the spatial extent of depigmentation, implementations of the present disclosure can also provide the degree of depigmentation. The degree and area of depigmentation can be measured with better accuracy and sensitivity than known techniques....…....................... Please see Fig. 19 and para. [0067]. Abstract.
Rahman et al. (US 20210118550 A1)- Disclosed is a content-based image retrieval (CBIR) system and related methods that serve as a diagnostic aid for diagnosing whether a dermoscopic image correlates to a skin cancer type. Systems and methods according to aspects of the invention use as a reference a set of images of pathologically confirmed benign or malignant past cases from a collection of different classes that are of high similarity to the unknown new case in question, along with their diagnostic profiles. Systems and methods according to aspects of the invention predict what class of skin cancer is associated with a particular patient skin lesion, and may be employed as a diagnostic aid for general practitioners and dermatologists........................... Please see Fig. 1-5. Abstract.
GAREAU (US 20180235534 A1)- A standardized, quantitative risk assessment method and apparatus for noninvasive melanoma screening. The apparatus and methods generate a melanoma Q-Score which calculates and displays a probability that a skin lesion is melanoma............................ Please see Fig. 1-4. Abstract.
GUYON et al. (US 20120008838 A1)- A system and method are provided for diagnosing diseases or conditions from digital images taken by a remote user with a smart phone or a digital camera and transmitted to an image analysis server in communication with a distributed network. The image analysis server includes a trained learning machine for classification of the images. The user-provided image is pre-processed to extract dimensional, shape and color features then is processed using the trained learning machine to classify the image. The classification result is postprocessed to generate a risk score that is transmitted to the remote user. A database associated with the server may include referral information for geographically matching the remote user with a local physician. An optional operation includes collection of financial information to secure payment for analysis services............................. Please see para. [0028-0034 and 0149]. Abstract.
ABEDINI et al. (US 20190147594 A1)- A method for image analysis comprises receiving one or more images of a plurality of lesions captured from a body of a person, extracting one or more features of the plurality of lesions from the one or more images, analyzing the extracted one or more features, wherein the analyzing comprises determining a distance between at least two lesions with respect to the extracted one or more features, and determining whether any of the plurality of lesions is an outlier based on the analyzing.......................... Please see Fig. 4-5 and para. [0077-0078]. Abstract.
GREENHALGH (US 20250308222 A1)- Qualifying an unqualified dermascope imaging device for use with an image classification algorithm is described. A qualification data set comprising a plurality of pairs of images of skin lesions is accessed, wherein each pair comprises an image of a skin lesion captured by an unqualified dermascope imaging device and an image of the skin lesion captured by a qualified dermascope imaging device. Using the image classification algorithm, a confidence value of classification of each image is computed. A similarity metric is measured between the unqualified and qualified dermascope imaging device using differences in the confidence values between images of each pair. Qualifying the unqualified dermascope imaging device for use with the image classification algorithm is done in response a comparison between the similarity metric and a similarity threshold.......................... Please see Fig. 1-3. Abstract.
Avanaki et al. (US 20200359887 A1)- A system and method of optical coherence tomography includes defining a suspect region-of-interest (SROI) for a suspect lesion in a first OCT B-scan image, defining a healthy region-of-interest (HROI) near the suspect lesion in a second OCT B-scan image, extracting optical properties from the SROI and from the HROI, obtaining an averaged A-line in the SROI and in the HROI, creating a set of normalized optical radiomic features from the averaged A-line in the SROI and in the HROI, and evaluating the set of normalized optical radiomic features to distinguish whether the suspect lesion is consistent with melanoma......................... Please see Fig. 7. Abstract.
DAVIS et al. (US 20170143249 A1)- Reference imagery of dermatological conditions is compiled in a crowd-sourced database (contributed by clinicians and/or the lay public), together with associated diagnosis information. A user later submits a query image to the system (e.g., captured with a smartphone). Image-based derivatives for the query image are determined (e.g., color histograms, FFT-based metrics, etc.), and are compared against similar derivatives computed from the reference imagery. This comparison identifies diseases that are not consistent with the query image, and such information is reported to the user. Depending on the size of the database, and the specificity of the data, 90% or more of candidate conditions may be effectively ruled-out, possibly sparing the user from expensive and painful biopsy procedures, and granting some peace of mind (e.g., knowledge that an emerging pattern of small lesions on a forearm is probably not caused by shingles, bedbugs, malaria or AIDS). A great number of other features and arrangements are also detailed....…....................... Please see para. [0359]. Abstract.
BANDIC et al. (US 20100185064 A1)- This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that extract multiple attributes from an object portrayed in a digital image utilizing a multi-attribute contrastive classification neural network. For example, the disclosed systems utilize a multi-attribute contrastive classification neural network that includes an embedding neural network, a localizer neural network, a multi-attention neural network, and a classifier neural network. In some cases, the disclosed systems train the multi-attribute contrastive classification neural network utilizing a multi-attribute, supervised-contrastive loss. In some embodiments, the disclosed systems generate negative attribute training labels for labeled digital images utilizing positive attribute labels that correspond to the labeled digital images........................ Please see Fig. 13 and para. [0028, 0226-0228, and 0329-0331]. Abstract.
DUNN et al. (US 12040080 B2)- The present disclosure is directed to a deep learning system for differential diagnoses of skin diseases. In particular, the system performs a method that can include obtaining a plurality of images that respectively depict a portion of a patient's skin. The method can include determining, using a machine-learned skin condition classification model, a plurality of embeddings respectively for the plurality of images. The method can include combining the plurality of embeddings to obtain a unified representation associated with the portion of the patient's skin. The method can include determining, using the machine-learned skin condition classification model, a skin condition classification for the portion of the patients skin, the skin condition classification produced by the machine-learned skin condition classification model by processing the unified representation, wherein the skin condition classification identifies one or more skin conditions selected from a plurality of potential skin conditions.......................... Please see Fig. 11. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights