DETAILED ACTION
This action is in response to the application filed on December 30th, 2024. Claims 1-5 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over US20250005758 (herein after referred to by its primary author, Park) in view of “Cognitive Support by Smart Phone Human Judgment on Cosmetic Skin Analysis Support” (herein after referred to by its primary author, Nishiyama).
In regards to claim 1, Park teaches a server for diagnosing a disease of a user through wrinkle detection, the server comprising: a photography control unit, which controls a skin photography device so as to capture an image of the skin of the user (Park Figure 2 S21; Paragraph [0058] “obtaining a face image of the subject by photographing a target skin (e.g., by the imaging unit 10) “); an image analysis unit for acquiring the image of the skin of the user from the skin photography device, performing preprocessing on the basis of landmarks based on the image of the skin (Park Figure 2 S 22; Paragraph [0058] “deriving shape or location information of a facial structure by recognizing feature points capable of identifying an individual in the obtained face image (e.g., by the face detection model 11)”), and determining a skin type and a skin characteristic of the user on the basis of the preprocessed image of the skin (Park Figure 1 14-16; Paragraph [0048] “a plurality of artificial neural network models 13, 14, 15, 16 for each of at least one item among diagnoses for wrinkles, pigmentation, pores, and erythema, wherein the plurality of artificial neural network models receive the de-identified face image as input, and visualize and provide skin diagnosis results for items corresponding to the artificial neural network models and symptom locations for each item.”); a disease analysis unit for detecting wrinkles of the user on the basis of the image of the skin, and diagnosing a disease of the user on the basis of the detected wrinkles of the user (Park Figure 2 S24; Figure 1 Part 13; Paragraph [0058] “visualizing and providing skin diagnosis results for items corresponding to the artificial neural network models and symptom locations for each item by inputting the de-identified face images into a plurality of the artificial neural network models, respectively (e.g., by the plurality of artificial neural network models 13 to 17)” Examiner note: This reference teaches that the neural network 13 visualizes and provides a diagnosis result from the de-identified face image. The visualization portion of this reference is considered analogous to the detection of wrinkles and the diagnosis result is analogous to a diagnosis of a disease of the user); and a solution provision unit for providing a customized solution according to the diagnosed disease of the user (Park Figure 2 S25; Paragraph [0058] “recommending a specific product tailored to the subject's skin concerns and lifestyle or providing beauty eating habit, based on the result of skin diagnose”).
Park does not teach an interface control unit which displays an interface for selecting a photographing objective through a display linked with the server, and which acquires, from the user, an input signal indicating the photographing objective.
However, Nishiyama teaches an interface control unit which displays an interface for selecting a photographing objective through a display linked with the server, and which acquires, from the user, an input signal indicating the photographing objective (Nishiyama Figure 1 step (b)).
Nishiyama is considered to be analogous to the claimed invention because they are both in the same field of image-based skin analysis. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Park to include the teachings of Nishiyama, to provide the advantage of user support which provides ease of use (Nishiyama Section IV “In order to realize such cognitive user support, we designed a system (Fig. 3) as user interface based on the performance (refer to paragraph III-A) of a smart phone (Android phone). The final service with this system is the same as that of the service depicted in Fig. 2; however, this system provides user support when the skin is photographed and aims at improving the success rate. The modules in this system have the following roles… The display module provides information to the user by displaying the input information and treats the operation on display as input from the user. This module displays the skin picture and text at the time the skin picture is taken, transmits it to the diagnosis and analysis server, and provides the diagnostic result. Moreover, the area of the picture to be transmitted to the diagnosis and analysis server can be selected with the use of the touch-panel function of the smart phone.”)
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Nishiyama, and further in view of US20190392953 (herein after referred to by its primary author, Steuer).
In regards to claim 2, Park in view of Nishiyama teaches the server of claim 1, wherein the photography control unit photographs a user in a first photography mode through the skin photography device when obtaining an input signal indicating disease analysis (Park Figure 2 S21; Paragraph [0058] “obtaining a face image of the subject by photographing a target skin (e.g., by the imaging unit 10) (S21);”), and the disease analysis unit comprises a wrinkle detection model that is trained by using training data consisting of a training input value corresponding to a skin image of each of multiple users obtained from multiple user terminals (Park Figures 6A-6D; Paragraph [0068] “Referring to FIGS. 6A to 6D, each of the plurality of artificial neural network models can be learned using the plurality of training samples as learning data. In an experimental example, 13,596 images for Korean men and women's research on human body application taken with a full-eye imaging device (VISIA-CR, VISIA, etc.) of a high-resolution were selected and processed, 50,358 image patches were used to learn the artificial neural network model, and 1,150 image patches were used to verify model learning results.”), and a training output value corresponding to a wrinkle image or a wrinkle probability map of the user (Park Paragraph [0068] “The data labeling is a process of displaying final results or information on original data so that the artificial intelligence can learn according to the development goal. For example, in order to learn wrinkle diagnosis, a wrinkle model displays exact location of the wrinkles on the skin image to be diagnosed. “), and generates the wrinkle probability map corresponding to the user on the basis of a deep learning network consisting of a plurality of hidden layers (Park Paragraph [0049] “In this specification, the artificial neural network model may include a deep learning model, wherein the deep learning model may be in the form of artificial neural networks stacked in multiple layers. The deep learning model automatically learns the features of each image by learning a large amount of data in a deep neural network consisting of a network of the multiple layers, and through this, trains the network in the manner of minimizing errors in the objective function, that is, the prediction accuracy.”), inputs the preprocessed skin image of the user into the wrinkle detection model based on a convolutional neural network (CNN) (Park Paragraph [0050] “In this specification, the deep learning model may use, for example, CNN (Convolutional Neural Network),”), and generates a wrinkle image or a wrinkle probability map corresponding to the skin image on the basis of an output of the wrinkle detection model, and detects wrinkles of the user on the basis of the wrinkle image or the wrinkle probability map, which is generated (Park Figure 3 “Wrinkle Model” and “Wrinkle result & image”).
Park in view of Nishiyama does not teach wherein the photography control unit photographs user in a second photography mode through the skin photography device when obtaining an input signal indicating skin analysis.
However, Steuer teaches wherein the photography control unit photographs a user in a first photography mode through the skin photography device when obtaining an input signal indicating disease analysis, and photographs the user in a second photography mode through the skin photography device when obtaining an input signal indicating skin analysis (Steuer Paragraph [0025] “i. acquire via the processor information from the user by means of questionnaire(s)/chatbot: a. each question to the user affects the following questions in terms of content and quantity; b. the answer of the user affects the analysis engine and/or affects the next presented question; ii. acquire at least one lesion image; iii. apply via the processor analysis of the image so to extract features (such as lesion color, elevation, shape, etc.); iv. present the user via the processor a first question about the lesion; v. acquire via the processor the user's response to said first question; vi. process via the processor the user's response, user lesion image and user profile and if a diagnosis cannot be made, decide whether to formulate another question to the user or request an additional image (such as a close-up or an image from another distance, or with different light conditions) of the lesion; vii. repeat steps ii to vi until a diagnosis can be made; and viii. present via the processor the diagnosis to the user via a textual/graphical interface such as a mobile phone screen, monitor, etc” Examiner note: This section teaches that an initial image of a lesion can be captured, which shows an input signal indicating disease analysis, and the capture process shows the first photography mode. Then, when a disease is not diagnosed, the user captures a second photo at another distance, which shows a second input signal indicating skin analysis, and the second capture process, with the second distance, shows the second photography mode).
Steuer is considered to be analogous to the claimed invention because they are both in the same field of image-based skin analysis. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Park in view of Nishiyama to include the teachings of Steuer, to provide the advantage of multiple camera angles of the skin being analyzed, allowing for more robust analysis (Steuer Paragraph [0097] “The application is further configured to instruct the inquirer to photograph the lesion from different distances and/or locations/angles and/or different lighting and/or to slowly move above lesion or abnormality according to application instructions, which may be in a real time mode.”)
In regards to claim 3, Park in view of Nishiyama and Steuer teaches The server of claim 2, wherein the disease analysis unit determines a wrinkle occurrence region of a user, and determines whether the user has a disease on the basis of the wrinkle occurrence region and a degree of wrinkles occurring in the region (Park Figure 2 S24; Figure 8 “Wrinkles Number and Area” Examiner note: Figure 8 shows all the criteria this system uses for determining a disease, which includes the wrinkles).
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Nishiyama and Steuer, and further in view of US20140243651 (herein after referred to by its primary author, Kim).
In regards to claim 4, Park in view of Nishiyama and Steuer teaches the server of claim 3, wherein the disease analysis unit calculates a disease risk on the basis of age of the user, the wrinkle occurrence region, and the degree of the wrinkle (Park Figure 2 S24; Figure 8 Examiner note: S24 shows that the results of each neural network are used for disease diagnosis and figure 8 shows that age and wrinkle region and intensity are used for diagnosis.) and suggests that the average number of wrinkles increases with age (Park Figure 11A).
Park in view of Nishiyama and Steuer fails to teach wherein the disease analysis unit sets a weight for the degree of the wrinkle to be low as the age of the user increases, and sets a weight for the degree of the wrinkle to be high as the age decreases on the basis of the obtained age information.
However, Kim teaches wherein the disease analysis unit sets a weight for the degree of the wrinkle to be low as the age of the user increases, and sets a weight for the degree of the wrinkle to be high as the age decreases on the basis of the obtained age information (Kim Paragraph [0078] “Thus, pathological information unit 400 can increase reliability of diagnosis results by calculating stochastically for disease which can occur depending on gender, age, occupation and body condition using received users' basic information, and updating basic information.” Examiner note: This section shows that diagnosing a patient is done based on the gender, age, occupation, etc of the patient. This section, when considered in combination with figure 11A of Park, suggests to one of ordinary skill in the art that wrinkles are less common at a younger age and should therefore be weighted more heavily with a younger patient than with an older patient when analyzing the skin).
Kim is considered to be analogous to the claimed invention because they are both in the same field of image-based skin analysis. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Park in view of Nishiyama and Steuer to include the teachings of Kim, to provide the advantage of automatic analysis and diagnosis of camera images (Kim Paragraph [0008] “However, prior art is to transmit checkup result including opinion of a medical specialist according to pathological checkup to user, and there was a problem that an extra operator or a specialist having expert knowledge for diagnosis has to monitor and pathological diagnosis is not performed quickly. Besides, currently there is no technology that recognizes body parts automatically, analyzes according to checkup elements for body part, and draws diagnosis results automatically.”)
In regards to claim 5, Park in view of Nishiyama, Steuer, and Kim teaches the server of claim 4, wherein the solution provision unit provides a medical diagnosis service or information on recommended lifestyle habits and recommended eating habits on the basis of the disease risk (Park Figure 2 S25; Paragraph [0058] “recommending a specific product tailored to the subject's skin concerns and lifestyle or providing beauty eating habit, based on the result of skin diagnose (S25);”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Deep Wrinkles in Skin Associated with Higher Cardiovascular Mortality Risk” suggests that wrinkles can be an early indicator for increased cardiovascular risk.
“Wrinkles” gives details of a large study done on wrinkles.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CALEB LOGAN ESQUINO whose telephone number is (703)756-1462. The examiner can normally be reached M-Fr 8:00AM-4:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CALEB L ESQUINO/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights