DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendment Entered
This Office action is responsive to the Amendment filed on March 30th, 2026. The examiner acknowledges the amendments to claims 1, 7, 9, 11, and 15 as well as the cancellation of claims 2, 4, 10, and 12. Claim 16 has been added. Claims 1, 3, 5-9, 11, and 13-16 are pending in the application.
Response to Arguments
Applicant’s arguments, filed March 30th, 2026, with respect to the drawing objections have been fully considered. The drawing objections are withdrawn.
Applicant’s arguments, filed March 30th, 2026, with respect to the claim objections have been fully considered. The claim objections are withdrawn.
Applicant’s arguments and amendments, filed March 30th, 2026, with respect to the claim interpretations under 35 U.S.C. 112(f) have been fully considered. The claim interpretations under 35 U.S.C. 112(f) are withdrawn.
Applicant’s arguments, filed March 30th, 2026, with respect to the rejections under 35 U.S.C. 112(b) have been fully considered. The rejections under 35 U.S.C. 112(b) are withdrawn.
Applicant’s arguments, filed March 30th, 2026, with respect to the rejections under 35 U.S.C. 101 have been fully considered but are not persuasive.
At page 11, Applicant argues that claim 1 is not directed to an abstract idea because claim 1 recites an image-based technical workflow. Examiner respectfully disagrees. Pages 7-9 2 of the previous office action clearly identified the limitations that are considered abstract. “It is essential that the broadest reasonable interpretation (BRI) of the claim be established prior to examining a claim for eligibility.” MPEP 2106 II. Applicant’s specification clearly explains that the claimed wavelength reflectance model is a mathematical relationship. See, for example, [0093-0097]. “A mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words ….” Determining at least one wavelength band based on applying the wavelength reflectance model is “a relationship between variables or numbers” that is “expressed in words.” Id. Further in light of Applicant’s specification, the claim encompasses concentrations of pigments with as few as two or three data points. See, for example, [0095-0097]. “The courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea.” MPEP 2106.04(a)(2) III. The claimed steps can be performed using two or three data points in the human mind or by using a pen and paper. Furthermore, the recited electronic device and skin details detector is a generic device/sensor configured to perform pre-solutional data gathering activity, the electronic device is configured to perform insignificant extra-solution activity, and the memory/non-transitory computer-readable storage medium storing instructions and processor is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
At page 11, Applicant argues that the capturing and converting steps integrate the steps into a practical application because they define how the skin information is technologically obtained. Examiner respectfully disagrees. The limitation of the “capturing a Red, Greed, and Blue (RGB) image of skin of a user” does not add a meaningful limitation to the method as it merely adds data-gathering to perform the abstract ides. With or without the claimed abstract idea, the electronic device gathers data the same. Therefore, it is unclear how there can be an improvement to the technology. Under step 2B, the claim utilizes an electronic device, which is generic and well-known in the industry – as evidenced by the cited non-patent literature herewith. See He et al., "Hyperspectral imaging enabled by an unmodified smartphone for analyzing skin morphological features and monitoring hemodynamics", February 2020, 16 pages). Furthermore, the step of converting, by the electronic device, the RGB image into a hyper spectral image is directed to mathematical concepts (including mathematical relationships, mathematical formulas or equations, and mathematical calculations) and is thus drawn to an Abstract idea.
At page 11, Applicant argues that the claims are directed to a specific improvement in the field of non-invasive skin imaging and analysis that enable skin information to be derived from an RGB image and that the limitations impose meaningful constraints on how the claimed result is achieved. Examiner respectfully disagrees. The improvement cannot be found in the abstract idea itself. “[I]t is important to keep in mind that an improvement in the abstract idea itself ... is not an improvement in technology.” MPEP 2106.05(a) Il. The claims recite steps for an processing of data. The claims do not integrate the processing into a practical application. Rather, the alleged improvement lies solely within the processing steps performed by the processor. “Merely adding generic computer components to perform the method is not sufficient. Thus, the claim must include more than mere instructions to perform the method on a generic component or machinery to qualify as an improvement to an existing technology." Id. Furthermore, the wavelength reflectance model and neural network model are used to generally apply the abstract idea (i.e., perform the mental processes and/or mathematical concepts, “determining, segmenting, extracting, determining”) without placing any limitations on how the wavelength reflectance model and neural network model operates to derive the wavelength band and information of the skin. In addition, the limitations would cover every mode of implementing the recited abstract idea using the wavelength reflectance model and neural network model. The claim omits any details as to how the wavelength reflectance model and neural network model solves a technical problem and instead recites only the idea of a solution or outcome. See MPEP 2106.05(f). Therefore, the limitations “determining, by the electronic device, at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image, wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising concentrations of pigments in the different tissues; determining, by the electronic device, information of the skin by applying a neural network model on the at least one wavelength band, wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model: and outputting a report comprising the information of the skin” represents no more than mere instructions to implement the abstract idea.
At pages 11-12, Applicant argues that the claim must be evaluated based on the particular ordered combination of operations and that the ordered combination is not well-understood, routine, or conventional. Examiner respectfully disagrees. The claims as a whole are analyzed to determine whether any element, or combination of elements, is sufficient to ensure that the claim amounts to significantly more than the exception. Besides the Abstract Idea, the claim recites additional steps of: electronic device comprising at least one processor including processing circuitry using hyperspectral reconstruction; capturing, by the electronic device, a Red, Green, and Blue (RGB) image of skin of a user; wavelength reflectance model; neural network model; outputting a report comprising the information of the skin. Consideration of the additional elements as a combination also adds no other meaningful limitations to the exception not already present when the elements are considered separately. Unlike the eligible claim in Diehr in which the elements limiting the exception are individually conventional, but taken together act in concert to improve a technical field, the claim here does not provide an improvement to the technical field. Even when viewed as a combination, the additional elements fail to transform the exception into a patent-eligible application of that exception. Thus, the claim as a whole does not amount to significantly more than the exception itself. The claim is therefore drawn to non-statutory subject matter. Furthermore, the electronic device recited in the claim is a generic device comprising generic components configured to perform the abstract idea – as evidenced by the cited non-patent literature herewith. See He et al., "Hyperspectral imaging enabled by an unmodified smartphone for analyzing skin morphological features and monitoring hemodynamics", February 2020, 16 pages); Nouri, et al., Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods. Int J CARS 11, 2185–2197 (2016). https://doi.org/10.1007/s11548-016-1449-5; Du et al., "Band selection using independent component analysis for hyperspectral image processing," 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings., Washington, DC, USA, 2003, pp. 93-98, doi: 10.1109/AIPR.2003.1284255; and Gevaux L, Gierschendorf J, I Rengot J, et al. Real-time skin chromophore estimation from hyperspectral images using a neural network. Skin Res Technol. 2021; 27: 163-177. https://doi.org/10.1111/srt.12927. The recited electronic device and skin details detector is a generic device/sensor configured to perform pre-solutional data gathering activity, the electronic device is configured to perform insignificant extra-solution activity, i.e. mere data outputting, and the memory/non-transitory computer-readable storage medium storing instructions and processor/wavelength reflectance model/neural network model is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
Applicant’s arguments, filed March 30th, 2026, with respect to the rejections under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 101
Claims 1, 3, 5-9, 11, and 13-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. A streamlined analysis of claim 1, 9, and 16 follows.
STEP 1
Regarding claims 1, 9, and 16, the claim recites a series of steps or acts and/or a series of structural elements, including a device. Thus, the claims are directed to a process and/or a machine, which is one of the statutory categories of invention.
STEP 2A, PRONG ONE
The claims are then analyzed to determine whether it is directed to any judicial exception. The steps of:
converting, by the electronic device, the RGB image into a hyper spectral image;
determining, by the electronic device, at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image,
wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising concentrations of pigments in the different tissues;
determining, by the electronic device, information of the skin by applying a neural network model on the at least one wavelength band,
wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model: and outputting a report comprising the information of the skin
set forth a judicial exception. These steps describe a concept performed in the human mind (including an observation, evaluation, judgment, opinion) (determining, extracting) and/or mathematical concepts (including mathematical relationships, mathematical formulas or equations, and mathematical calculations) (converting, segmenting, extracting). Thus, the claims are drawn to a Mental Process and/or Mathematical Concepts, which is an Abstract Idea."
STEP 2A, PRONG TWO
Next, the claim as a whole is analyzed to determine whether the claim recites additional elements that integrate the judicial exception into a practical application. The claim fails to recite an additional element or a combination of additional elements to apply, rely on, or use the judicial exception in a manner that imposes a meaningful limitation on the judicial exception. Claims 1, 9, and 16 recite an electronic device configured to output a report to a user, which is merely adding insignificant extra-solution activity to the judicial exception (MPEP 2106.05(g)). The outputting a report does not provide an improvement to the technological field, the method does not effect a particular treatment or effect a particular change based on the presented intrinsic frequency, nor does the method use a particular machine to perform the Abstract Idea. The wavelength reflectance model and neural network model are used to generally apply the abstract idea (i.e., perform the mental processes and/or mathematical concepts, “determining, segmenting, extracting, determining”) without placing any limitations on how the wavelength reflectance model and neural network model operates to derive the wavelength bands and information of the skin. In addition, the limitations would cover every mode of implementing the recited abstract idea using the wavelength reflectance model and neural network model. The claim omits any details as to how the wavelength reflectance model and neural network model solves a technical problem and instead recites only the idea of a solution or outcome. See MPEP 2106.05(f). Therefore, the limitations represent no more than mere instructions to implement the abstract idea.
STEP 2B
Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure that the claim amounts to significantly more than the exception. Besides the Abstract Idea, the claim recites additional steps of:
electronic device comprising at least one processor including processing circuitry using hyperspectral reconstruction;
capturing, by the electronic device, a Red, Green, and Blue (RGB) image of skin of a user;
wavelength reflectance model;
neural network model;
outputting a report comprising the information of the skin.
The capturing and outputting steps are well-understood, routine and conventional activities for those in the field of medical diagnostics. Further, the capturing and outputting steps are each recited at a high level of generality such that it amounts to insignificant pre-solution activity and insignificant extra-solution activity, e.g., mere data gathering and mere data outputting steps necessary to perform the Abstract Idea. When recited at this high level of generality, there is no meaningful limitation, such as a particular or unconventional step that distinguishes it from well-understood, routine, and conventional data gathering and comparing activity engaged in by medical professionals prior to Applicant's invention. Furthermore, it is well established that the mere physical or tangible nature of additional elements such as the capturing and outputting steps do not automatically confer eligibility on a claim directed to an abstract idea (see, e.g., Alice Corp. v. CLS Bank Int'l, 134 S.Ct. 2347, 2358-59 (2014)).
Consideration of the additional elements as a combination also adds no other meaningful limitations to the exception not already present when the elements are considered separately. Unlike the eligible claim in Diehr in which the elements limiting the exception are individually conventional, but taken together act in concert to improve a technical field, the claim here does not provide an improvement to the technical field. Even when viewed as a combination, the additional elements fail to transform the exception into a patent-eligible application of that exception. Thus, the claim as a whole does not amount to significantly more than the exception itself. The claim is therefore drawn to non-statutory subject matter.
Regarding claims 1, 9, and 16, the electronic device recited in the claim is a generic device comprising generic components configured to perform the abstract idea. The recited electronic device and skin details detector is a generic device/sensor configured to perform pre-solutional data gathering activity, the electronic device is configured to perform insignificant extra-solution activity, and the memory/non-transitory computer-readable storage medium storing instructions and processor is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
The dependent claims also fail to add something more to the abstract independent claims. Claims 3, 5-8, 11, and 13-15 are directed to more abstract ideas, which does not add anything significantly more. The steps recited in the independent claims maintain a high level of generality even when considered in combination with the dependent claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 5-6, 8-9, 11, 13-14, and 16 are under 35 U.S.C. 103 as being unpatentable over He (He et al., "Hyperspectral imaging enabled by an unmodified smartphone for analyzing skin morphological features and monitoring hemodynamics", February 2020, 16 pages) in view of Nouri (Nouri, et al., Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods. Int J CARS 11, 2185–2197 (2016). https://doi.org/10.1007/s11548-016-1449-5), and further in view of Bandic (US 20100185064 A1).
Regarding claim 1, He discloses a method for determining information of skin by an electronic device comprising at least one processor including processing circuitry using hyper spectral reconstruction (pages 895-896, Introduction, “unmodified smartphone … RGB images … reconstruct … melanin absorption etc., within the skin” (Examiner note: a smartphone comprises a memory and a processor/processing circuitry)), wherein the method comprises: capturing, by the electronic device (“smartphone”, page 896, Introduction), a Red, Green, and Blue (RGB) image of a skin of a user (page 896, Introduction, “RGB images captured by the built-in camera … smartphone”); converting, by the electronic device, the RGB image into a hyper spectral image (pages 896-897, 2.1., Reconstruction principle from RGB images to hyperspectral images, “Wiener estimation algorithm to perform hyperspectral reconstruction from RGB images”); determining at least one wavelength band (page 898-899, 2.3. Hyperspectral reconstruction and post-processing, “extracted spatial absorption information of skin chromophores, e.g. melanin and hemoglobin, through a series of processing steps on images representing different wavebands … weighted subtraction … red light wavebands”, see equation 9), and the at least one wavelength band comprising concentrations of pigments in the different tissues (extraction of blood and melanin information content from hyperspectral reconstruction with the RGB images captured by a smartphone under the fluorescent lamp illumination … Blood and melanin absorption maps, fig. 5 & 3.2. Skin morphological feature analysis).
He does not expressly disclose determining, by the electronic device, at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image; wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising concentrations of pigments in the different tissues.
However, Nouri directed to enhanced tissue visualization and discrimination combining band selection methods discloses determining, by the electronic device, at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image (pages 2190-2191, Dimension reduction, Band selection & Performance evaluation, “adaptation of PCA, called segmented PCA … supervised PCA … Optimum Index Factor (OIF) method … Sheffield index (SI) criterion … constrained band selection (CBS) … three most relevant bands … selected bands”, fig. 1); wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image (page 2186, “tissue segmentation and classification using neighboring pixels” & page 2190, Dimension reduction, “segmented PCA … divide the HS cube into compact groups that contain the most correlated contiguous spectral bands” & page 2192, Tissue Discrimination, “target tissue and its surrounding tissues”, fig. 5), extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image (page 2186, “tissue segmentation and classification using neighboring pixels” & pages 2190-2191, Dimension reduction, “extracting the spectral signatures of the targeted anatomical tissues and their bounding tissues”), and determining, from the extracted spectra, the at least one wavelength band information in the different tissues (page 2186, “spectral bands to investigate … blood and water absorption bands, valuable information … spectral band …depending on layer thickness and hemoglobin concentration” & page 2191-2191, Dimension reduction, Band Selection, “assign spectral bands according to their information content … three most relevant bands … are selected”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He such that the method comprises determining, by the electronic device, at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image; wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising information in the different tissues, in view of the teachings of Nouri, as this would aid in providing the best performance with rich information, high tissue separability and short computational time by incorporating the combined band selection methods of Nouri.
He, as modified by Nouri hereinabove, further discloses the at least one wavelength band comprising concentrations of pigments in the different tissues (He, pages 903-905, 3.2. Skin morphological feature analysis & figs. 5 & 7, “Extraction of blood and melanin information content from hyperspectral reconstruction … blood absorption map … melanin absorption map”).
He, as modified by Nouri, does not disclose determining, by the electronic device, information of the skin by applying a neural network model on the at least one wavelength band, wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model: and outputting a report comprising the information of the skin.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 00749]) discloses determining, by the electronic device, information of the skin (“make a quantitative assessment of clinical, medical, non-medical, and cosmetic indications, such as moisture level, firmness, … skin color, psoriasis …”; “determining a skin state 158 … UV damage may be assessed”, para. [0312, 0315-0316, 0342]) by applying a neural network model on the at least one wavelength band (“algorithm 150 may be based on artificial neural networks”; “ targeted wavelength or wavelengths may be employed for specific endpoint measurements”, para. [0312, 0315-0316]), wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model (“input data taken from collected images and a biophysical skin state 158”; “measure … concentration … melanin, hemoglobin; make quantitative assessment of clinical, medical, non-medical, and cosmetic indications … skin color, psoriasis, allergies, red areas, general skin disorders”; “determine firmness/tightness, an algorithm 150 may combine an assessment of collagen and elastin concentrations”, para. [0026, 0312, 0315-0316, 0342]); and outputting a report comprising the information of the skin (“user interface 102 … receive a personalized regimen 118 for sun protection given the user's skin state 158”, para. [0344], fig. 14).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri hereinabove, such that the method comprises determining, by the electronic device, information of the skin by applying a neural network model on the at least one wavelength band, wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model: and outputting a report comprising the information of the skin, in view of the teachings of Bandic, as this would aid in making quantitative assessment of clinical, medical, non-medical, and cosmetic indications.
Regarding claim 3, He, as modified by Nouri and Bandic hereinabove, discloses the method as claimed in claim 1, wherein the concentrations of the pigments in the different tissues under the skin comprises information related to at least one of thickness of the skin, melanin concentration in the skin, bilirubin concentration, hair thickness under the skin, blood vessel thickness under the skin, a hemoglobin (Hb) concentration under the skin, and oxygenated hemoglobin (HBO2) concentration under the skin (He, page 901-902 & 906-907, 3.2. Skin morphological feature analysis, “hemoglobin absorption information”; 4. Discussions, “blood, melanin absorption maps and oxygen saturation”).
Regrading claim 5, He, as modified by Nouri and Bandic hereinabove, discloses the method as claimed in claim 1. He, as modified by Nouri and Bandic hereinabove, does not disclose wherein the information of the skin includes at least one of skin tone, ultraviolet exposure risk, pigmentation, psoriasis, eczema and skin abnormalities.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 00749]) and an algorithm 150/neural network to perform the determining of a skin state (para. [0035, 0312, 0315]) discloses wherein the information of the skin includes at least one of skin tone, ultraviolet exposure risk, pigmentation, psoriasis, eczema and skin abnormalities (“properties … psoriasis … pigmentation, tone”; “abnormal skin condition”, “sun damage”, para. [0028, 0256, 0315, 0416, 0440, 0752]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri and Bandic hereinabove, such that the information of the skin includes at least one of skin tone, ultraviolet exposure risk, pigmentation, psoriasis, eczema and skin abnormalities, in view of the teachings of Bandic, as this would aid in making quantitative assessment of clinical, medical, non-medical, and cosmetic indications.
Regarding claim 6, He, as modified by Nouri and Bandic hereinabove, discloses the method as claimed in 1, wherein the RGB image is captured by at least one of an imaging apparatus with limited spectral resolution (Abstract, “built-in RGB camera … unmodified smartphone”).
Regrading claim 8, He, as modified by Nouri and Bandic hereinabove, discloses the method as claimed in claim 1 He, as modified by Nouri and Bandic hereinabove, does not disclose wherein the method comprises: generating, by the electronic device, a skin health and disorder report by applying the wavelength reflectance model and neural network model; and performing, by the electronic device, at least one of: displaying changes in health of the skin based on the skin health and disorders report; and recommending products specific to the skin based on the skin health and disorder report.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 00749]) and an algorithm 150/neural network to perform the determining of a skin state (para. [0035, 0312, 0315]) discloses wherein the method comprises: generating, by the electronic device, a skin health and disorder report (“objective skin health assessment report”; pre-diagnosis 162 & skin state 185”, para. [0060, 0305-0307, 0315]) by applying the algorithm (“algorithms … analysis”, para. [0298-0300, 0305-0307]); and performing, by the electronic device (fig. 1), at least one of: displaying changes in health of the skin based on the skin health and disorders report (“skin condition … tracked … displayed”; “report of the images and skin state 158”; “observe measurable changes in skin health”, para. [0266, 0311, 0333, 0386], figs. 5-7); and recommending products specific to the skin based on the skin health and disorder report (figs. 5-7, “regimen recommendation … personalized”; “recommendations for skin care … products” para. [0262, 0411-0413]). Bandic further discloses that the skin care regiment recommendation is personalized to the skin condition of each person (para. [0262]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri and Bandic hereinabove, such that the method comprises: generating, by the electronic device, a skin health and disorder report by applying the wavelength reflectance model and neural network model; and performing, by the electronic device, at least one of: displaying changes in health of the skin based on the skin health and disorders report; and recommending products specific to the skin based on the skin health and disorder report, in view of the teachings of Bandic, as this would aid in providing a personalized skin care regimen and product recommendations based on the skin condition.
Regarding claim 9, He discloses an electronic device configured to determine information of skin using hyper spectral reconstruction (pages 895-896, Introduction, “unmodified smartphone … RGB images … reconstruct … melanin absorption etc., within the skin”), the electronic device (Abstract, unmodified smartphone), the electronic device comprising: a memory; a processor (Abstract, “smartphone” (Examiner note: a smartphone comprises a memory and a processor)); and a skin details detector comprising circuitry (Abstract, “smartphone … built-in RGB camera”), operably coupled to the memory and the processor (Abstract), configured to: capture a Red, Green, and Blue (RGB) image of skin of a user (page 896, Introduction, “RGB images captured by the built-in camera … smartphone”); convert the RGB image into a hyper spectral image (pages 896-897, 2.1., Reconstruction principle from RGB images to hyperspectral images, “Wiener estimation algorithm to perform hyperspectral reconstruction from RGB images”); determine at least one wavelength band (page 898-899, 2.3. Hyperspectral reconstruction and post-processing, “extracted spatial absorption information of skin chromophores, e.g. melanin and hemoglobin, through a series of processing steps on images representing different wavebands … weighted subtraction … red light wavebands”, see equation 9), and the at least one wavelength band comprising concentrations of pigments in the different tissues (extraction of blood and melanin information content from hyperspectral reconstruction with the RGB images captured by a smartphone under the fluorescent lamp illumination … Blood and melanin absorption maps, fig. 5 & 3.2. Skin morphological feature analysis).
He does not disclose the processor configured to determine at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image, wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising concentrations of pigments in the different tissues.
However, Nouri directed to enhanced tissue visualization and discrimination combining band selection methods discloses a processor (page 2188, “computer”) configured to determine at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image (pages 2190-2191, Dimension reduction, Band selection & Performance evaluation, “adaptation of PCA, called segmented PCA … supervised PCA … Optimum Index Factor (OIF) method … Sheffield index (SI) criterion … constrained band selection (CBS) … three most relevant bands … selected bands”, fig. 1), wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image (page 2186, “tissue segmentation and classification using neighboring pixels” & page 2190, Dimension reduction, “segmented PCA … divide the HS cube into compact groups that contain the most correlated contiguous spectral bands” & page 2192, Tissue Discrimination, “target tissue and its surrounding tissues”, fig. 5), extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image (page 2186, “tissue segmentation and classification using neighboring pixels” & pages 2190-2191, Dimension reduction, “extracting the spectral signatures of the targeted anatomical tissues and their bounding tissues”), and determining, from the extracted spectra, the at least one wavelength band information in the different tissues (page 2186, “spectral bands to investigate … blood and water absorption bands, valuable information … spectral band …depending on layer thickness and hemoglobin concentration” & page 2191-2191, Dimension reduction, Band Selection, “assign spectral bands according to their information content … three most relevant bands … are selected”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He such that the processor is configured to determine at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image, wherein applying the wavelength reflectance model comprises segmenting different tissues under the skin from the hyper spectral image, extracting spectra of each individual tissue of the different tissues by analyzing multiple pixels on the hyper spectral image, and determining, from the extracted spectra, the at least one wavelength band comprising information in the different tissues, in view of the teachings of Nouri, as this would aid in providing the best performance with rich information, high tissue separability and short computational time by incorporating the combined band selection methods of Nouri.
He, as modified by Nouri hereinabove, further discloses the at least one wavelength band comprising concentrations of pigments in the different tissues (He, pages 903-905, 3.2. Skin morphological feature analysis & figs. 5 & 7, “Extraction of blood and melanin information content from hyperspectral reconstruction … blood absorption map … melanin absorption map”).
He, as modified by Nouri hereinabove, does not disclose the processor configured to determine information of the skin by applying a neural network model on the at least one wavelength band, wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model; and output a report comprising the information of the skin.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 00749]) discloses a processor (“processor”, para. [0802]) configured to determine information of the skin (“make a quantitative assessment of clinical, medical, non-medical, and cosmetic indications, such as moisture level, firmness, … skin color, psoriasis …”; “determining a skin state 158 … UV damage may be assessed”, para. [0312, 0315-0316, 0342]) by applying a neural network model on the at least one wavelength band (“algorithm 150 may be based on artificial neural networks”; “ targeted wavelength or wavelengths may be employed for specific endpoint measurements”, para. [0312, 0315-0316]), wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model (“input data taken from collected images and a biophysical skin state 158”; “measure … concentration … melanin, hemoglobin; make quantitative assessment of clinical, medical, non-medical, and cosmetic indications … skin color, psoriasis, allergies, red areas, general skin disorders”; “determine firmness/tightness, an algorithm 150 may combine an assessment of collagen and elastin concentrations”, para. [0026, 0312, 0315-0316, 0342]); and output a report comprising the information of the skin (“user interface 102 … receive a personalized regimen 118 for sun protection given the user's skin state 158”, para. [0344], fig. 14).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri hereinabove, such that the process is configured to determine, by the electronic device, information of the skin by applying a neural network model on the at least one wavelength band, wherein applying the neural network model comprises inputting the concentrations of pigments in the different tissues to the neural network model and obtaining the information of the skin from the neural network model: and outputting a report comprising the information of the skin, in view of the teachings of Bandic, as this would aid in making quantitative assessment of clinical, medical, non-medical, and cosmetic indications.
Regarding claim 11, He, as modified by Nouri and Bandic hereinabove, discloses the electronic device as claimed in claim 9, wherein the concentrations of the pigments in the different tissues under the skin comprises information related to at least one of a thickness of the skin, a Melanin concentration in the skin, a Bilirubin concentration, a hair thickness under the skin, a Blood vessel thickness under the skin, a hemoglobin (Hb) concentration under the skin, and a oxygenated hemoglobin (HBO2) concentration under the skin (He, page 901-902 & 906-907, 3.2. Skin morphological feature analysis, “hemoglobin absorption information”; 4. Discussions, “blood, melanin absorption maps and oxygen saturation”).
Regrading claim 13, He, as modified by Nouri and Bandic hereinabove, discloses the electronic device as claimed in claim 9. He, as modified by Nouri and Bandic hereinabove, does not disclose wherein the information of the skin includes at least one of a skin tone, an ultraviolet exposure risk, a pigmentation, a psoriasis, an eczema and a skin abnormalities.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 0749]) and an algorithm 150/neural network to perform the determining of a skin state (para. [0035, 0312, 0315]) discloses wherein the information of the skin includes at least one of a skin tone, an ultraviolet exposure risk, a pigmentation, a psoriasis, an eczema and a skin abnormalities (“properties … psoriasis … pigmentation, tone”; “abnormal skin condition”, “sun damage”, para. [0028, 0256, 0315, 0416, 0440, 0752]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri and Bandic hereinabove, such that the information of the skin includes at least one of a skin tone, an ultraviolet exposure risk, a pigmentation, a psoriasis, an eczema and a skin abnormalities, in view of the teachings of Bandic, as this would aid in making quantitative assessment of clinical, medical, non-medical, and cosmetic indications.
Regarding claim 14, He, as modified by Nouri and Bandic hereinabove, discloses the electronic device as claimed in claim 9, wherein the RGB image is captured by at least one of an imaging apparatus with limited spectral resolution (Abstract, “built-in RGB camera … unmodified smartphone”).
Regarding claim 16, He, as modified by Nouri and Bandic hereinabove, teaches the non-transitory computer-readable storage medium storing instructions executed by at least one processor comprising processing circuitry of an electronic device (Abstract, “smartphone” (Examiner note: a smartphone comprises a non-transitory computer-readable storage medium storing instructions and a processor for executing instructions)), as the subject matter of claim 16 is analogous to the subject matter of claims 1 and 9.
Claims 7 and 15 are under 35 U.S.C. 103 as being unpatentable over He in view of Nouri and Bandic, as applied to claims 1 and 9 above, further in view of Demirli (US 20080212894 A1), and further in view of Bandic.
Regarding claim 7, He, as modified by Nouri and Bandic hereinabove, discloses the method as claimed in claim 1. He, as modified by Nouri and Bandic hereinabove, does not disclose wherein the method further comprises: generating, by the electronic device, a hyper pigmentation report by applying the wavelength reflectance model and neural network model containing information of an extent of pigmentation; determining, by the electronic device, whether the extent of pigmentation is improving based on the hyper pigmentation report.
However, Demirli directed to the generation of images depicting the simulated aging or de-aging of skin discloses generating, by the electronic device (para. [0035], fig. 11), a hyper pigmentation report (“severity score”, para. [0057]) by applying the information of an extent of pigmentation (“degree of hyperpigmentation”, para. [0057]); determining, by the electronic device (para. [0035], fig. 11), whether the extent of pigmentation is improving based on the hyper pigmentation report (“score … used to monitor worsening or improvement of pigmentation”, para. [0057]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri and Bandic hereinabove, such that the method further comprises: generating, by the electronic device, a hyper pigmentation report by applying the wavelength reflectance model and neural network model containing information of an extent of pigmentation; determining, by the electronic device, whether the extent of pigmentation is improving based on the hyper pigmentation report, in view of the teachings of Demirli, as this would aid in monitoring the worsening or improvement of pigmentation based on a severity score associated with the degree of hyperpigmentation.
He, as modified by Nouri, Bandic, and Demirli hereinabove, does not disclose performing, by the electronic device, at least one of: recommending to the user not to change a prescription in response to determining that the extent of pigmentation is improving; recommending to the user to change the prescription in response to determining that the extent of pigmentation is not improving; and recommending to the user to stop medication and consult a doctor in response to determining that the extent of pigmentation is declining.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 0749]) and an algorithm 150/neural network to perform the determining of a skin state/pigmentation (para. [0035, 0312, 0315]) discloses performing, by the electronic device (fig. 1), at least one of: recommending to the user not to change a prescription in response to determining that the skin state is improving; recommending to the user to change the prescription in response to determining that the skin state is not improving; and recommending to the user to stop medication and consult a doctor in response to determining that the skin state is declining (“skin state … pigmentation”; “skin health assessment … comparing … advice on continuing, modifying, or terminating a regimen 118 … skin state 158 changed over time … healthier … shared with a practitioner … consultation”, para. [0315, 0420-0422], figs. 16-17).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri, Bandic, and Demirli hereinabove, such that the method further comprises performing, by the electronic device, at least one of: recommending to the user not to change a prescription in response to determining that the extent of pigmentation is improving; recommending to the user to change the prescription in response to determining that the extent of pigmentation is not improving; and recommending to the user to stop medication and consult a doctor in response to determining that the extent of pigmentation is declining, in view of the teachings of Bandic, as this would aid in tracking the effectiveness of a skin care product or regimen based on the skin state/pigmentation.
Regrading claim 15, He, as modified by Nouri and Bandic hereinabove, discloses the electronic device as claimed in claim 9. He, as modified by Nouri and Bandic hereinabove, does not disclose wherein the electronic device is configured to: generate a hyper pigmentation report by applying the wavelength reflectance model and neural network model containing information of an extent of pigmentation; and determine whether the extent of pigmentation is improving or in optimal range based on the hyper pigmentation report.
However, Demirli directed to the generation of images depicting the simulated aging or de-aging of skin discloses wherein the electronic device (para. [0035], fig. 11) is configured to generate a hyper pigmentation report (“severity score”, para. [0057]) by applying the information of an extent of pigmentation (“degree of hyperpigmentation”, para. [0057]); and determine whether the extent of pigmentation is improving or in optimal range based on the hyper pigmentation report (“score … used to monitor worsening or improvement of pigmentation”, para. [0057]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri and Bandic hereinabove, such that the method further comprises: generating, by the electronic device, a hyper pigmentation report by applying the wavelength reflectance model and neural network model containing information of a extent of pigmentation; determining, by the electronic device, whether the extent of pigmentation is improving based on the hyper pigmentation report, in view of the teachings of Demirli, as this would aid in monitoring the worsening or improvement of pigmentation based on a severity score associated with the degree of hyperpigmentation.
He, as modified by Nouri, Bandic, and Demirli hereinabove, does not disclose the electronic device configured to perform at least one of: recommend not changing a prescription in response to determining that the extent of pigmentation is improving or in optimal range; recommend changing the prescription in response to determining that the extent of pigmentation is not improving; and recommend stopping medication and consulting a doctor in response to determining that the extent of pigmentation is declining.
However, Bandic directed to an image processing technique for determining a skin photo type of a captured image in a Red Green Blue (RGB) color imaging system and classification of other skin characteristics (para. [0007, 0749]) and an algorithm 150/neural network to perform the determining of a skin state/pigmentation (para. [0035, 0312, 0315]) discloses the electronic device (fig. 1) configured to perform at least one of: recommend not changing a prescription in response to determining that the skin state is improving or in optimal range; recommend changing the prescription in response to determining that the skin state is not improving; and recommend stopping medication and consulting a doctor in response to determining that the skin state is declining (“skin state … pigmentation”; “skin health assessment … comparing … advice on continuing, modifying, or terminating a regimen 118 … skin state 158 changed over time … healthier … shared with a practitioner … consultation”, para. [0315, 0420-0422], figs. 16-17).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He, as modified by Nouri, Bandic, and Demirli hereinabove, such that the electronic device is configured to perform at least one of: recommend not changing a prescription in response to determining that the extent of pigmentation is improving or in optimal range; recommend changing the prescription in response to determining that the extent of pigmentation is not improving; and recommend stopping medication and consulting a doctor in response to determining that the extent of pigmentation is declining, in view of the teachings of Bandic, as this would aid in tracking the effectiveness of a skin care product or regimen based on the skin state/pigmentation.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Gareau (WO 2020146489 A1) directed to hyperspectral dermoscopy and segmenting a digital image automatically into epidermal and dermal regions (para. [0057], fig. 5); Du et al., "Band selection using independent component analysis for hyperspectral image processing," 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings., Washington, DC, USA, 2003, pp. 93-98, doi: 10.1109/AIPR.2003.1284255; Ito (US 20210256280 A1) directed to an information processing apparatus that includes an acquisition section that acquires a multi-spectral image; Akoho et al., Nonlinear Estimation of Chromophore Concentrations and Shading from Hyperspectral Images. In: Mansouri et al., Image and Signal Processing. ICISP 2016. Lecture Notes in Computer Science(), vol 9680. pages 101-108. Springer, Cham. https://doi.org/10.1007/978-3-319-33618-3_11.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW ELI HOFFPAUIR whose telephone number is (571)272-4522. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Marmor II can be reached at (571) 272-4730. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.E.H./Examiner, Art Unit 3791
/AURELIE H TU/Primary Examiner, Art Unit 3791
Prosecution Insights