Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED OFFICE ACTION
Status of Claims
Claims 1-20 are pending examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b) (2) (C) for any potential 35 U.S.C. 102(a) (2) prior art against the later invention.
1. Claims 1,2,3,4 and 5 are rejected under 35 U.S.C 103(a) as being unpatentable over Koruga et al. (USPUB 20090245603) in view of Yuan Liu et al. ( NPL Doc: “A deep learning system for differential diagnosis of skin diseases,” 18th May 2020, Nature Medicine,VOL 900 26, June 2020, Pages 900–906).
As per claim 1, Koruga et al. teaches A system( Paragraph [0298]- “ …a skin analysis system 104 may be used to interface with the device 108, store images, deploy algorithms 150, track a skin state 158 by keeping track of images from any number of areas of concern, the interval between image capture, a projected next image capture date, communicate findings to a practitioner, interact with simulation tools 132…”), comprising: one or more processors coupled to a memory ( Paragraph [0359]- “The processor may include memory that stores methods, codes, instructions and programs as described herein and elsewhere. The processor may access a storage medium through an interface that may store methods, codes, and instructions …”) , configured to: receive an image depicting a diseased portion of skin of a user ( Paragraph [0144]- “…The polarized light may result from the reflection on the skin and is not polarized from the light source. The capture and storage of the reflections enables the imaging and analysis of skin lesions, as well as all types of skin diseases, skin problems, and cosmetic concerns and indications. Analysis of polarized reflections may enable obtaining thermal, electrical, and magnetic properties of the imaged skin area....”) ;
access a medical record associated with the user to identify a skin characteristics of the diseased portion of skin represented in the image ( Paragraph [0152]- “… data storage may be in a skin health record 121. The skin health record 121 may be an object or database or repository for an individual that contains information on key medical, non-medical, and cosmetic indications related to a user's skin. This may comprise images, graphics, icons, written history, personal demographic information, levels of cosmetic conditions such as moisture, elasticity, firmness, texture, color level, or non-medical conditions such as inflammation,…”) ; determine a plurality of treatment plans for the diseased portion of skin based at least on the skin characteristics and the image ( Paragraph [0217]- “…The user may use the device in the privacy of their home, work, or any other location to perform remote monitoring 164 and submit images to track progress of their skin's health or medical conditions. A practitioner may be able to remotely guide changes in treatment or guide on prevention factors. Remote diagnosis may greatly increase efficiency of progress monitoring since users will not have to make a physician trip to the provider,…” and Paragraph [0262]) ; select a first treatment plan of the plurality of treatment plans based on the medical record ( Paragraph [0258]- “…Use of the device 108 to capture images enables a user to readily transmit the images to any practitioner for remote assessment, to track progression of a skin condition, rapidly compare images to previous images, other user images or third party images, such as images in a dermascopic database 115, …”) ; receive a second image of the diseased portion of skin of the user ( Paragraph [0031]- “…obtaining a second skin state assessment, comparing the second assessment to the baseline assessment to determine progress towards a skin care goal, and optionally, optimizing the regimen or product in order to improve a skin state. …”) ; determine a treatment progress of the diseased portion of skin of the user based at least on the image and the second image( Paragraph [0262]- “…selecting or capturing a starting image, a user may indicate the kind of simulation they would like to perform. For example, the user may like to perform a simulation of aging only, or a simulation of aging and treatment effects. The simulation tool 132 may return data on overall appearance, wrinkle count, elasticity, luminosity, moisture, product usage simulation, and the like….” And Paragraph [0336-0337]- “…Given the current skin state 158, a new product or regimen 118 may be recommended. For example, the system may recommend specific ingredients to look for in order to increase a user's luminosity given a current skin state 158. Reports may be on-screen, printed, custom, and the like. Reports may be shared with a practitioner for ongoing treatment and consultation. …”) ;
Koruga et al. does not explicitly teach provide the first treatment plan to a computing device of the user; and select a second treatment of the plurality of treatment plans based at least on the medical record and the treatment progress.
However, within analogous art, Yuan Liu et al. teaches provide the first treatment plan to a computing device of the user ( Page 900-Col. 2- “…predicting a single diagnosis, instead of a full differential diagnosis. A differential diagnosis is a ranked list of diagnoses that is used to plan treatments in the common setting of diagnostic ambiguity in dermatology, and can capture a more comprehensive assessment of a clinical case than a single diagnosis…” and Page 901- Fig. 1 ) ; and select a second treatment of the plurality of treatment plans based at least on the medical record and the treatment progress(Different treatment plans based on the diagnosis of the skin disease and progression taught within Page 904- Col. 1 - “…The top-3 sensitivity for malignant lesions is important because the inclusion of a diagnosis of malignancy on the differential diagnosis may prompt a clinician to obtain a specimen for pathology, even if it is not the primary suspected diagnosis. In erythematosquamous and papulosquamous skin disease, these eruptions can be clinically similar to erythema and scaling, although they can have very different etiologies and treatment plans….”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Yuan Liu et al. within the modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. because the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. provides a method and system for implementation of skin image analysis and diagnosis with deep learning algorithm.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. within the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. for implementation of skin image analysis and diagnosis with deep learning algorithm.
As per claim 2, Combination of Karuga et al. and Yuan Liu et al. teach claim 1,
Koruga et al. teaches wherein the one or more processors are further configured ( Paragraph [0359]- “The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The processor may be part of a server, client, network infrastructure, mobile computing platform, stationary computing platform, or other computing platform. A processor may be any kind of computational or processing device capable of executing program instructions …”) to select the first treatment plan further based on at least one of: a frequency of prescription of the first treatment plan, outcome data for the first treatment plan, or a historical diagnosis of the user ( Paragraphs [0019-0020]- “…an imaging device permits a user to take high magnification pictures of the skin in the vicinity of an area of concern and submit those pictures, optionally along with textual and data responses, for medical, non-medical, and cosmetic analysis, diagnosis and treatment recommendation and follow-up….”) .
As per claim 3, Combination of Karuga et al. and Yuan Liu et al. teach claim 1,
Koruga et al. teaches wherein to determine the treatment progress of the diseased portion of skin of the user, the one or more processors are further configured to determine a change in characteristics of the diseased portion of skin between the image and the second image ( Change of characteristic of skin state taught within Fig. 57 and Paragraph [0245-0246]- “Referring now to FIG. 52, it is possible to determine changes in skin state 158 using spectral characteristics of specifically selected light sources based on specific biophysical criteria. FIG. 52 shows a comparison of PB(S-O) signatures showing an example for differences between benign/healthy expected tissues and diseased tissue. Changes, such as in the 462 nm-485 nm range in FIG. 52, such as absorption or emission within the spectral diagram may correspond to additional changes in tissue processes, tissue activity, or presence of other molecules that indicate a changed state of skin. By measuring these changes, it is possible to determine healthy and diseased or disturbed states of the skin….”) .
As per claim 4, Combination of Karuga et al. and Yuan Liu et al. teach claim 1,
Koruga et al. teaches wherein the one or more processors are further configured to transmit the treatment progress of the diseased portion of skin to the computing device of the user ( Paragraph [0217]- ‘…The user may use the device in the privacy of their home, work, or any other location to perform remote monitoring 164 and submit images to track progress of their skin's health or medical conditions. A practitioner may be able to remotely guide changes in treatment or guide on prevention factors. Remote diagnosis may greatly increase efficiency of progress monitoring since users will not have to make a physician trip to the provider, and the provider could conveniently select a time during the day to observe the patients change. The monitored data may be viewed as a recording or in real time….”) .
As per claim 5, Combination of Karuga et al. and Yuan Liu et al. teach claim 1,
Koruga et al. teaches wherein the one or more processors are further configured to store the treatment progress of the diseased portion of skin in the medical record of the user ( Paragraphs [0337-0338]- “…[0337] Referring to FIG. 16, a summary screen of a skin care system is depicted. An overall analysis for a time interval may be shown, current measurements, progress towards reaching a skin care goal, a product assessment, a regimen 118 assessment, advice on continuing, modifying, or terminating a regimen 118 or product usage, and the like. The user may view a step-by-step analysis or obtain a full report….”) .
2. Claims 6,7,8,9,10,11,12,13 and 14 are rejected under 35 U.S.C 103(a) as being unpatentable over Guyon et al. (USPUB 20120008838) in view of Koruga et al. (USPUB 20090245603).
As per claim 6, Guyon et al. teaches A system, comprising: one or more processors, coupled to memory, configured to: receive an image depicting a portion of skin of a user having a skin color( Paragraph [0030-0031]- “…the system comprising: a server in communication with a distributed network for receiving a digital image data set from the remote user, the remote user also in communication with the distributed network; a processor for executing a learning machine, wherein the learning machine is trained using image data sets having known outcomes for skin cancer, the processor further operable for: receiving the digital image data set from the remote user; pre-processing the digital image data set to extract features including contour, dimension and color features;…”) ;
identify a reference color of a reference object in the image (Paragraph [0274]- “…b) surrounding objects, clothes or background were taken either as reference skin color or as mole instead of the real skin or mole causing the contour to be incorrect; c) the lesion was not a single mole but a group of moles, a cluster or a plaque; d) hairy nevi caused the software to confuse background and mole; and e) shading or glare….”) ;access biometric information of the user from a computing device of the user( Paragraph [0224]- “…one for the risk associated with the personal profile (e.g., age, skin color, family history, eye color, use of sunscreen, etc.) and one for the risk associated with the particular mole. To validate the system, data was collected by taking pictures of moles available on the Internet using a smart phone camera, in this case, an iPhone.RTM.. The photographs were taken directly from the computer screen display at a distance of about 40 cm from the screen to avoid aliasing
…”) ;
Guyon et al. does not explicitly teach provide the image, the biometric information and reference color as input to a skin color classifier; and provide to the computing device a classification of the skin color of the user selected from an output of the skin color classifier providing a plurality of skin colors.
However, within analogous art, Koruga et al. teaches provide the image, the biometric information and reference color as input to a skin color classifier( Paragraphs [0272-0274]- “…determining skin characteristics and cosmetic features using color analysis includes a step of generating a phototype of the skin through a decision tree unit responsive to the estimated distribution model parameters colors. The phototype of the skin is generated according to a corrected Fitzpatrick classification, or any other applicable color classifier. In accordance with an exemplary embodiment of the present invention, the step of generating a phototype of the skin according to corrected Fitzpatrick classification includes generating a phototype of the skin according to a skin type scale which ranges from very fair skin to very dark skin….a person's body can be captured by any digital camera. The acquired digital image sample of the person's skin may be analyzed in a pixel by pixel manner in the RGB color system. After the conversion of colors from a device-dependent RGB color system into a device-independent standard RGB color system (sRGB), a table of most frequent sRGB colors which appear on the image may be generated. According to an example, the generated table may consist of 256 most frequent colors which appear on the image of the person's skin…”) ; and provide to the computing device a classification of the skin color of the user selected from an output of the skin color classifier providing a plurality of skin colors ( Paragraphs [0279-0280]- “… FIG. 60, a diagram depicting a pixel view of the acquired digital image of a part of person's skin after quantization is shown. The image of the sample of the person's skin is captured under the white emitting light. The image may be captured by any digital camera and the like under white emitting light. The analyzer coupled to the image capturing device analyzes the acquired digital image in a pixel by pixel manner in the RGB color system. The analysis of acquired digital image in a pixel by pixel manner in the sRGB (after RGB to sRGB color system conversion) is not only limited for determining skin phototype but also may be useful for other purposes like classification of other skin characteristics (e.g. elasticity, melanin, oil concentration etc.)…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Koruga et al. within the modified teaching of the System and method for remote melanoma screening mentioned by Guyon et al. because the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. provides a method and system for implementation of skin care assessment and analysis based on images captured of skin region.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. within the System and method for remote melanoma screening mentioned by Guyon et al. for implementation of skin care assessment and analysis based on images captured of skin region.
As per claim 7, Combination of Guyon et al. and koruga et al. teaches claim 6,
Guyon et al. teaches wherein the one or more processors are further configured to select the classification of the skin color of the user based at least on confidence scores corresponding to the plurality of skin colors ( Paragraph [0034]- “The resulting vote (score) of the ensemble or second level classifier is post-processed to obtain a mapping of the output to probabilities. The output is converted into an alphanumeric and/or graphical display that may be stored in a memory medium and/or transmitted to the remote user to provide an overall probability, i.e., a confidence level, that the lesion in the image is melanoma….”) .
As per claim 8, Combination of Guyon et al. and koruga et al. teaches claim 6,
Guyon et al. teaches wherein the one or more processors are further configured to select the classification of the skin color of the user based at least on probability score corresponding to the plurality of skin colors ( The classification of the skin color within the image and the probability scoring taught within FIG. 6 and FIG7 and FIG.12 and Paragraph [0178]- “0178] The ensemble of classifiers that follow the ABCD diagnosis rules provides a good understanding of the decision made by each classifier. FIG. 12 illustrates an exemplary format for a sample report to the user showing the result of classification of the image that is shown in FIG. 7. At the upper portion 902 of the display window 900, the user name is provided along with a case number. The photograph that was submitted by the user is reproduced as image 904. The result of the ensemble classifier is displayed at 906 in the form of a confidence level that the suspected lesion is melanoma. In this example, the classification is Melanoma with 96% confidence. This number is the post-processed logistic regression probability value of the ensemble classifier. The lower portion of the figure shows the logistic regression probability values for the individual A, B and C classifiers as bar graphs 90A, 908B and 908C, respectively, showing the confidence level for the corresponding feature….”) .
As per claim 9, Combination of Guyon et al. and koruga et al. teaches claim 6,
Guyon et al. teaches wherein the one or more processors are further configured to format the image based at least on the reference color and provide the formatted image as input to the skin classifier (FIG. 15 and Paragraph [0195]- “Using the borders detected in the segmentation step, the color features are extracted from the original (manually pre-processed) image in step 624 and input into Classifier C (626). If the dimensional information can be obtained from the image using the EXIF meta data, or if the user included in the photograph a dimensional reference such as a ruler, or a coin or other object of known dimension, the actual measurements can be extracted using the segmented image and input into Classifier D (614). Note that Classifier D is only indicated by dashed lines within image analysis module 606 because it may not be possible to obtain the dimensional data from the image. ” and Paragraph [0274]- “…clothes or background were taken either as reference skin color or as mole instead of the real skin or mole causing the contour to be incorrect; c) the lesion was not a single mole but a group of moles,…”) .
As per claim 10, Combination of Guyon et al. and koruga et al. teaches claim 6,
Guyon et al. teaches wherein the skin classifier is further configured to identify a color in the image that is closest to the reference color and determine a difference between the color and the reference color ( FIG. 10 and Paragraph [0149-150]- “The classifiers used in step 218 for analysis of the user-provided image and data are pre-trained, i.e., trained and tested, using one or more image datasets having known outcomes. The feature matrix shown in FIG. 10 represents a training dataset with 103 cases of which 50 were images of malignant melanoma and 53 …”) .
As per claim 11, Combination of Guyon et al. and koruga et al. teaches claim 10,
Guyon et al. teaches wherein the skin classifier is further configured to compensate for differences in lighting in the image based at least on the difference ( Classifier for correcting the image features such as color based on differences taught within Table 3 and Paragraphs [0150-0151]- “An ensemble of classifiers was trained using the features extracted from the image, with one classifier trained on each of the feature types A, B, C and D. As is known in the art, an ensemble of classifiers is a set of classifiers whose individual decisions are combined in some way to classify new examples. An ensemble may, but not necessarily, consist of a set of different classifier types. Table 3 lists the features that fall within the 4 feature types:…”) .
As per claim 12, Combination of Guyon et al. and koruga et al. teaches claim 10,
Guyon et al. teaches wherein the skin classifier is further configured to adjust other colors in the image based at least on the difference( Classifier for correcting the image features such as color based on differences taught within Table 3 and Paragraphs [0156-0157] - “ …Each extracted feature is separately processed using a trained base classifier 504A-504D, respectively, that has been optimized for classification of the corresponding ABCD feature, to generate an output that identifies whether the characteristics of the features extracted from the image correspond to a diagnosis of melanoma. The results of the classification of each of classifiers 504A-504D are combined to create the ensemble by overall classifier 506, generating a single result which is output for further processing.…”) .
As per claim 13, Combination of Guyon et al. and koruga et al. teaches claim 6,
Within analogous art, Koruga et al. teaches wherein the one or more processors are further configured to update a model used by the skin color classifier to analyze the image based on the biometric information retrieved from the computing device ( Paragraph [0055]- “…The device may further include a user interface. The device may further include a display surface. The skin assessment data of locations may be overlaid on an image of a larger skin region and displayed on the display surface. The device may further include an access restriction module used for restricting access to authorized users only. The access restriction module may be based on biometric access control. The device may be capable of generating alerts about abnormal skin conditions in real-time….” AND Paragraph [0198]- “… the vertical panel-including skin care device may also have an access restriction module restricting access to private information to authorized users only. The access restriction module may be based on a user name and password feature and/or biometric access control, for example, fingerprint recognition, facial recognition,…”) .
As per claim 14, Combination of Guyon et al. and koruga et al. teaches claim 6,
Within analogous art, Koruga et al. teaches wherein the one or more processors are further configured to update a model used by the skin color classifier to analyze the image based on a user provided classification of the skin color received from the computing device ( Paragraph [0305]- “…Users may be able to upload an image and model various skin parameters (such as moisture level in skin, collagen level, age, and the like.) and observe changes in the image. Additionally, users may be able to model the impact of various products and regimens 118 (skin care, cosmetic, medical, nail care, hair care, and the like) on the image. Simulation tools 132 may enable users to view changes on the entire image or split half of the image to show a comparison of modeled change with current image…”) .
2. Claims 15,19 and 20 are rejected under 35 U.S.C 103(a) as being unpatentable over Koruga et al. (USPUB 20090245603) in view of Oka et al. (USPUB 20080275315).
As per claim 15, Koruga et al. teaches A system ( Paragraph [0298]- “ …a skin analysis system 104 may be used to interface with the device 108, store images, deploy algorithms 150, track a skin state 158 by keeping track of images from any number of areas of concern, the interval between image capture, a projected next image capture date, communicate findings to a practitioner, interact with simulation tools 132…”) , comprising: one or more processors coupled to memory( Paragraph [0359]- “The processor may include memory that stores methods, codes, instructions and programs as described herein and elsewhere. The processor may access a storage medium through an interface that may store methods, codes, and instructions …”), configured to: determine, based on an image depicting a portion of skin of a user ( Paragraph [0277]- “Referring to FIG. 59, a diagram depicting a pixel view of an acquired digital image of a sample of person's skin is shown. The image of a sample of a person's skin is captured under white emitting light. The image may be captured by any digital camera and the like under white emitting light…”) ,a plurality of scores that each correspond to a respective one of a plurality of skin characteristics ( Paragraph [0060]- “…a graphical user interface unit included in the web-enabled computing system for generating a frequently asked questionnaire unit further comprising a self answer guide module, a scoring module coupled to the frequently asked questionnaire unit, a comparison module coupled to the scoring module for comparing: a color parameter; a symmetry parameter; and a border parameter, an automation unit coupled to the graphical user interface for enabling a time-based synchronization of the frequently asked questionnaire unit, the scoring module, and the comparison module…”) ; provide to a computing device of the user a skin characteristic selected from the plurality of skin characteristics based at least on the plurality of scores ( Paragraphs [0308-0309]- “…the user interface 102 may enable ranking and rating 138. Ranking and rating 138 may be performed for various product characteristics as well as on the various raters and rankers. Product experience may be collected from users in simple ranking and rating 138 format as well as textual comment data to be stored in a database. This ranking and rating 138 may be real time, and may be synthesized to show what is most relevant to the user based on like users or peers, such as users with any of the following characteristics: same age, same sex, same skin type, same ethnicity, geography, moisture levels, and the like. These ranking and ratings 138 may be dynamic ranking and ratings 138. The users may be shown either the total number of rankers/raters and/or the weighted percent score ranking or rating 138….”) ;
Koruga et al. does not explicitly teach determine second skin characteristics using a second image depicting a portion of skin of the user at a later time than the first image; and determine a change in characteristics of the portion of skin responsive to a comparison of the second skin characteristics with the skin characteristic.
However, within analogous art, Oka et al. teaches determine second skin characteristics using a second image depicting a portion of skin of the user at a later time than the first image ( Paragraph [0017-0018]- “…receiving a second skin image of the pigmentary deposition portion same as the first skin image picked up by the camera device with dermoscope at a different time point, source information for identifying the pigmentary deposition portion, and second time information relating to a time of sending the second skin image from the user terminal, storing the second skin image in the diagnosis result storage after associating the source information with the second skin image, taking out the stored second skin image to diagnose the second skin image for the skin lesion with the use of the first diagnosis program, diagnosing through a comparison between the diagnosis result for the first skin image and a diagnosis result for the second skin image with the use of the second diagnosis program…”) ; and determine a change in characteristics of the portion of skin responsive to a comparison of the second skin characteristics with the skin characteristic ( Paragraphs [0092-0093]- “…A comparison diagnosis program provided in the remote diagnosis apparatus and serving as the second diagnosis program obtains the diagnosis result for the first skin image and the diagnosis result for the second skin image from the diagnosis result storage to make a further diagnosis on the skin lesion based on a comparison with the diagnosis result for the second skin images (S23). Thus, it is possible to know a change of the skin lesion from the time point of sending the first skin image to the time point of sending the second skin image. …”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Oka et al. within the modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. because the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. provides a method and system for implementation of remote diagnosis apparatus uses the diagnosis program to examine the skin image for disease.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. within the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. for implementation of remote diagnosis apparatus uses the diagnosis program to examine the skin image for disease.
As per claim 19, Combination of Koruga et al. and Oka et al. teaches claim 15,
Karuga et al. teaches wherein the one or more processors are further configured to provide to the computing device identification of the change in characteristics of the portion of skin ( Change on the skin condition and identification of the change taught within Paragraph [0154-0155]- “…selection may be achieved by analyzing and tracking the change of various skin health parameters through the application of various products and ingredients through using the device 108 and tracking the change of the skin health over time through a personalized manufacturing record 172….”) .
As per claim 20, Combination of Koruga et al. and Oka et al. teaches claim 15,
Karuga et al. teaches wherein the one or more processors are further configured to access, from a medical database, a medical record associated with the user and determine the plurality of scores further based on the medical record ( Paragraph [0260-0261]- “…. The skin state 158 may be stored in the device 108 itself, on a PC, in a central server, a salon record, an e-medicine record, a medical repository, a cosmetic clinical studies database 115, a mobile device, and the like. The device 108 may communicate with a user interface 102, an online platform 120, a mobile platform 124, and the like to upload, deliver, share, and/or port images, analysis 154, skin states 158, data, track history, user profiles, and the like, as will be further described herein. For example, a user may use a device 108 embodied in a mobile device to capture an image of the skin and upload it to a mobile platform 124 for analysis 154 to determine a skin state 158….”) .
3. Claims 16 and 17 are rejected under 35 U.S.C 103(a) as being unpatentable over Koruga et al. (USPUB 20090245603) in view of Oka et al. (USPUB 20080275315) in further view of Yuan Liu et al. ( NPL Doc: “A deep learning system for differential diagnosis of skin diseases,” 18th May 2020, Nature Medicine,VOL 900 26, June 2020, Pages 900–906).
As per claim 16, Combination of Koruga et al. and Oka et al. teaches claim 15,
Combination of Koruga et al. and Oka et al. does not explicitly teach wherein the plurality of skin characteristics comprises a diagnosis of one or more skin conditions.
Within analogous art, Yuan Liu et al. teaches wherein the plurality of skin characteristics comprises a diagnosis of one or more skin conditions ( Page 903 – Col. 2- “…a differential diagnosis instead of a single diagnosis is particularly important in dermatology. Because most skin conditions are not verified with pathology, the differential diagnosis is used for decision making around work-up and treatment. If all conditions in the differential diagnosis share the same treatment, a single diagnosis may not be clinically necessary. If the diagnoses on the differential have opposing treatments (for example, treatment for one condition on the differential may aggravate another diagnosis on the differential), a clinician can still consider this group of diagnoses together to determine a work-up or initiate treatment….”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Yuan Liu et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. because the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. provides a method and system for implementation of skin image analysis and diagnosis with deep learning algorithm.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. for implementation of skin image analysis and diagnosis with deep learning algorithm.
As per claim 17, Combination of Koruga et al. and Oka et al. teaches claim 15,
Combination of Koruga et al. and Oka et al. does not explicitly teach wherein the one or more processors are further configured to determine a diagnosis of a skin condition depicted in the image .
Within analogous art, Yuan Liu et al. teaches wherein the one or more processors are further configured to determine a diagnosis of a skin condition depicted in the image ( Page 901 – Fig. 1 and “…During training, the aggregated ranked list of dermatologist-provided diagnoses have an associated aggregated ‘confidence’ score per diagnosis, and these confidences are the target ‘soft’ labels for the DLS. The DLS therefore learns from both the primary (top-ranked) diagnosis as well as the lower-ranked diagnoses. In this way, the DLS was trained to provide a differential diagnosis instead of a single prediction output….”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Yuan Liu et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. because the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. provides a method and system for implementation of skin image analysis and diagnosis with deep learning algorithm.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. for implementation of skin image analysis and diagnosis with deep learning algorithm.
4. Claim 18 is rejected under 35 U.S.C 103(a) as being unpatentable over Koruga et al. (USPUB 20090245603) in view of Oka et al. (USPUB 20080275315) in further view of Yuan Liu et al. ( NPL Doc: “A deep learning system for differential diagnosis of skin diseases,” 18th May 2020, Nature Medicine,VOL 900 26, June 2020, Pages 900–906) and Sumit Majumder et al. ( NPL Doc: “Smartphone Sensors for Health Monitoring and Diagnosis,” 9th May 2019, Sensors 2019, 19, 2164, Pages 1-30).
As per claim 18, Combination of Koruga et al. and Oka et al. and Yuan Liu et al. teaches claim 17,
Combination of Koruga et al. and Oka et al. and Yuan Liu et al. does not explicitly teach wherein the one or more processors are further configured to determine the severity of the skin condition depicted in the image.
Within analogous art, Sumit Majumder et al. teaches wherein the one or more processors are further configured to determine the severity of the skin condition depicted in the image ( Page 16- “…The features were then fed to a support vector machine (SVM) classifier to enable automatic classification of skin lesions. In another work, a multi-layer perceptron (MLP) was employed on a smartphone to analyze skin images captured by its camera, thus enabling skin cancer detection [100]. A similar application was proposed in Reference [101], which upon capturing the image of the skin, can alert the users about a potential sunburn and/or the severity of melanoma. There a novel method to compute the time-to-skin-burn by utilizing the information of burn frequency level and UV index level was introduced. Additionally, for dermoscopic image analysis, a system for the smartphones that incorporates algorithms for image acquisition, hair detection and exclusion, lesion segmentation, feature extraction, and classification was developed….”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Sumit Majumder et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. and the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. because the Smartphone Sensors for Health Monitoring and Diagnosis mentioned by provides Sumit Majumder et al. provides a method and system for implementation of remote monitoring of an individual’s skin health from images captured and analyzed.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to implement the Smartphone Sensors for Health Monitoring and Diagnosis mentioned by provides Sumit Majumder et al. within the combined modified teaching of the System and method for analysis of light-matter interaction based on spectral convolution mentioned by Koruga et al. and the Pigmentary Deposition Portion Remote Diagnosis System mentioned by Oka et al. and the A deep learning system for differential diagnosis of skin diseases mentioned by Yuan Liu et al. for implementation of remote monitoring of an individual’s skin health from images captured and analyzed.
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2123.
Examiner’s Notes
5. The Examiner acknowledges the following prior arts below as pertinent to the current applications claim limitations and inventive concept, although the following prior arts shown below were not relied upon to address the limitations within the claim , they are analogous art mentioning the inventive concept key points on (Skin image analysis, Skin lesion detection, classification of skin color , classifier, machine learning, skin disease detection and treatment plan etc.).
1) Jufeng Yang et al.,"Clinical Skin Lesion Diagnosis using Representations Inspired by Dermatologist Criteria," June 2018, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, Pages 1258-1264.
2) Philipp Tschandl et al. "Data Descriptor: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions," 14th August 2018,SCIENTIFIC DATA , 5:180161,nature, Pages 1-7.
3) LING-FANG LI et al.,"Deep Learning in Skin Disease Image Recognition: A Review," 1st December 2020,IEEEAccess, Volume 2020, Pages 208264-208275.
4) Andre Esteva et al.,"Dermatologist-level classification of skin cancer with deep neural networks," 2nd February 2017, NATURE , Vol. 542, Pages 115-117.
5) Tiago M de Carvalho et al.,"Development of Smartphone Apps for Skin Cancer Risk Assessment: Progress and Promise,"11th July 2019, JMIR Dermatol 2019, vol. 2 ,iss. 1 ,e13376,Pages 1-6.
6) OMAR ABUZAGHLEH et al.,"Noninvasive Real-Time Automated Skin Lesion Analysis System for Melanoma Early Detection and Prevention," 14th April 2015, ONCOLOGY, Volume 3, 2015, 4300212,Pages 1-10.
7) Xinyuan Zhang et al,"Towards improving diagnosis of skin diseases by combining deep neural network and human knowledge,"13th November, 2017,BMC Medical Informatics and Decision Making 2018, 18(Suppl 2):59,Pages 70-75.
8) Ulzii-Orshikh Dorj et al.,"The skin cancer classification using deep convolutional neural network," 22nd February 2018,Multimed Tools Appl (2018) 77,Pages 9910-9917.
9) CHO BYONG JIN (KR 20230065464)
10) Rasochova et al. (USPUB 20210027897)
11) BUDMAN et al. (USPUB 20200234444)
12) Garnavi et al. (USPAT 10517524)
13) MARINKOVICH et al. (USPUB 20170246473 )
14) CHANG TSAI-RONG et al. ( TW 201247168)
15) Westmoreland et al. ( USPUB 20110218428)
16) Bandic et al. (USPUB 20100185064)
17) Bergman (USPUB 20090279760)
18) Patwardhan (USPUB 20090137908)
19) Gulfo et al.( USPUB 20090060304)
20) Gutkowicz-Krusin et al. ( USPUB 20080214907)
21) Kang et al. (USPUB 20080180950)
22) Gizewski (USPUB 20080161661)
23) Momma (USPAT 7233693 )
24) Skladnev ( USPUB 20040264749)
25) Leshem et al. ( USPAT 6215893)
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to OMAR S ISMAIL whose telephone number is (571)272-9799 and Fax # is (571)273-9799. The examiner can normally be reached on M-F 9:00am-6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David C. Payne can be reached on (571) 272-3024. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)? If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OMAR S ISMAIL/
Primary Examiner, Art Unit 2635
Prosecution Insights