DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
Information Disclosure Statement
The information disclosure statement (IDS) submitted on June 6, 2024 has been considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
[0014] and [0045] recite "obtaining hyperspectral data of at original facial position…" (emphasis added). This appears to be a typographical error and should be “of an” (emphasis added).
Appropriate correction is required.
Claim Objections
Claim 4 is objected to because of the following informalities:
Claim 4 line 4 recites "obtaining hyperspectral data of at original facial position…" (emphasis added). This appears to be a typographical error and should be “of an” (emphasis added).
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“hyperspectral image collecting unit” in claim 11 line 3,
“hyperspectral image processing unit” in claim 11 line 6,
“hemoglobin component distribution image obtaining unit” in claim 11 line 9,
“melanin distribution obtaining unit” in claim 11 line 14, and
“image enhancement processing unit” in claim 12 lines 2-3.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Regarding claim 11, a “hyperspectral image collecting unit” can be a hyperspectral imaging camera with a Full Width at Half Maximum (FWHM) less than 50nm ([0012] and [0060]).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 and 14-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Relative Terminology
Regarding claims 1 and 11, the term “obvious” in lines 4-5 of claim 1 and line 5 of claim 11 is a relative term which renders the claim indefinite. The term “obvious” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. What is considered “obvious” to one person may not be considered “obvious” by another person. Therefore, applicant has failed to particularly point out and distinctly claim the subject matter which the inventor or joint inventor regards as the invention.
Additionally, claims 2-10 and 13-17 depend on claim 1 and fail to clarify what the term “obvious” is defined as, so they are likewise rejected.
Antecedent Basis
Claim 3 recites the limitation "the white balance" in line 4. Although “white balance processing” is recited in claim 1 line 6 and claim 3 line 3, there is no mention of “white balance” prior to claim 3. It is unclear what “the white balance” is referring to. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Claim 4 recites the limitation "the facial hyperspectral images" in line 4. “Facial hyperspectral image data” is recited in claim 1 line 3, and there are many images obtained by image processing algorithms in steps S3 and S4 such as “original hemoglobin component distribution image” in claim 1 line 8-9, “melanin content difference distribution image” in claim 1 line 13, or “melanin distribution image” in claim 1 line 14. In particular, it is unclear if “the facial hyperspectral images” is referring to the “facial hyperspectral image data” of claim 1 or to the “original hemoglobin component distribution image”, “melanin content difference distribution image”, or “melanin distribution image” in claim 1. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Claims 3 and 4 recite the limitation “the facial hyperspectral image data of three different preset wavebands" in claim 3 lines 2-3 and claim 4 lines 2-3. Although the collection of “facial hyperspectral image data in different preset wavebands” is recited in claim 1 line 3, there is no mention of “facial hyperspectral image data of three different preset wavebands" prior to claims 3 and 4. In particular, it is unclear if the "three different present wavebands" are part of the "different preset wavebands" of claim 1 or if they are entirely different from the "different preset wavebands" of claim 1. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Additionally, claims 5, 6, 8, 9 depend on claim 3, and claims 15, 16, and 17 depend claim 4, so they are likewise rejected.
Clarity
Claim 10 recites the “separation method for image with facial skin component according to claim 1, wherein further comprises image enhancement processing to…” in lines 1-2. It is unclear whether the applicant meant to say “wherein the method further comprises” or “further comprising” since an image enhancement processing is being introduced, or if there are words missing after the word “wherein”. Therefore, applicant has failed to particularly point out and distinctly claim the subject matter which the inventor or joint inventor regards as the invention.
Written Description
Claim limitation “hyperspectral image processing unit” in claim 11 line 6, “hemoglobin component distribution image obtaining unit” in claim 11 line 9, “melanin distribution obtaining unit” in claim 11 line 14, and “image enhancement processing unit” in claim 12 lines 2-3 invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function.
Regarding the “hyperspectral image processing unit"” in claim 11 line 6, the applicant describes the “hyperspectral image processing unit” as performing white balance processing or absolute reflectance processing on the face hyperspectral image data in the different wavebands in [0024] and [0045]. For white balance processing, it can obtain three frames of facial hyperspectral images I1(x, y), I2(x, y), and I3(x, y) by using the Gray World Algorithm to achieve the white balance [0044]. For absolute reflectance processing, it can obtain hyperspectral data of at original facial position of a reference white board in the corresponding waveband, divide the facial hyperspectral images by the hyperspectral data of the reference white board, and then obtain three frames of facial hyperspectral images I1(x, y), I2(x, y), and I3(x, y) after the absolute reflectance processing [0045]. However, none of these paragraphs describe a structure for the “hyperspectral image processing unit”. Although there are mentions of a central processing unit (CPU) performing processing based on programs in [0065] referencing Fig. 8 and executing the functions defined in the method in [0067], there is no clear link between the “hyperspectral image processing unit” and the CPU. Therefore, in this instance, “hyperspectral image processing unit” is interpreted as a 112(f) limitation and the specification fails to disclose a specific structure for the “hyperspectral image processing unit”.
Regarding the “hemoglobin component distribution image obtaining unit” in claim 11 line 9, the applicant describes the algorithm for the “hemoglobin component distribution image obtaining unit” as obtaining the original hemoglobin component distribution image O(x, y) by subtracting the facial hyperspectral image I2(x, y) by I3(x, y), or dividing I2(x, y) by I3(x, y) in [0017] and [0047]. However, none of these paragraphs describe a structure for the “hemoglobin component distribution image obtaining unit”. Although there are mentions of a central processing unit (CPU) performing processing based on programs in [0065] referencing Fig. 8 and executing the functions defined in the method in [0067], there is no clear link between the “hemoglobin component distribution image obtaining unit” and the CPU. Therefore, in this instance, “hemoglobin component distribution image obtaining unit” is interpreted as a 112(f) limitation and the specification fails to disclose a specific structure for the “hemoglobin component distribution image obtaining unit”.
Regarding the “melanin distribution obtaining unit” in claim 11 line 14, the applicant describes the algorithm for the “melanin distribution obtaining unit” as obtaining the melanin distribution image M(x, y) based on the image processing algorithm, M(x, y)= I3(x, y)+ ΔM(x, y), or obtaining the melanin distribution image M(x, y) based on the skin reflection model, M(x, y) = M'(x, y) + ΔM(x, y), where M'(x, y) is the content distribution of melanin component obtained by the linear regression in [0020], [0049], and [0059]. However, none of these paragraphs describe a structure for the “melanin distribution obtaining unit”. Although there are mentions of a central processing unit (CPU) performing processing based on programs in [0065] referencing Fig. 8 and executing the functions defined in the method in [0067], there is no clear link between the “melanin distribution obtaining unit” and the CPU. Therefore, in this instance, “melanin distribution obtaining unit” is interpreted as a 112(f) limitation and the specification fails to disclose a specific structure for the “melanin distribution obtaining unit”.
Regarding the “image enhancement processing unit” in claim 12 lines 2-3, the applicant describes the algorithm for the “image enhancement processing unit” as performing maximum-minimum normalization, contrast enhancement, and histogram equalization on gray-scale images of contents of hemoglobin and melanin in [0021], [0027], [0050], [0060], and [0063]. However, none of these paragraphs describe a structure for the “image enhancement processing unit”. Although there are mentions of a central processing unit (CPU) performing processing based on programs in [0065] referencing Fig. 8 and executing the functions defined in the method in [0067], there is no clear link between the “image enhancement processing unit” and the CPU. Therefore, in this instance, “image enhancement processing unit” is interpreted as a 112(f) limitation and the specification fails to disclose a specific structure for the “image enhancement processing unit”.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Additionally, claim 12 depends on claim 11 and is likewise rejected.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 11-12 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 11, applicant claims “hyperspectral image processing unit: configured for performing white balance processing or absolute reflectance processing on the face hyperspectral image data in the different wavebands”, “hemoglobin component distribution image obtaining unit: configured for using an image processing algorithm to obtain an original hemoglobin component distribution image, constructing a skin reflection model, and obtaining content distributions of melanin and hemoglobin component by making linear regression on pixels comprising concentrations of melanin and hemoglobin component”, and “melanin distribution image obtaining unit: configured for using an image processing algorithm to obtain a melanin content difference distribution image, and obtaining a melanin distribution image based on the content distribution of melanin component, wherein the image processing algorithm comprises image subtraction or image division.”
As per MPEP § 2181(IV), “A means- (or step-) plus-function limitation that is found to be indefinite under 35 U.S.C. 112(b) based on failure of the specification to disclose corresponding structure, material or act that performs the entire claimed function also lacks adequate written description” (emphasis added). Furthermore, as per MPEP § 2163.03(VI), “(s)uch a limitation also lacks an adequate written description as required by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C 112, first paragraph, because an indefinite, unbounded functional limitation would cover all ways of performing a function and indicate that the inventor has not provided sufficient disclosure to show possession of the invention.”
Since applicant has not defined any particular structure for the “hyperspectral image processing unit” in claim 11 line 6, “hemoglobin component distribution image obtaining unit” in claim 11 line 9, and “melanin distribution obtaining unit” in claim 11 line 14, the inventor has not provided sufficient disclosure to show possession of the invention. Applicant has not provided any specific definition for the structure that carry out the functions disclosed in claim 11. Additionally, the claimed invention as a whole may not be adequately described if the claims require an essential or critical feature which is not adequately described in the specification and which is not conventional in the art or known to one of ordinary skill in the art. It appears that these components and/or features are essential and critical features of the applicant’s invention because without them, the applicant’s invention would not work. In particular, the structure of the “hyperspectral image processing unit” in claim 11 line 6, “hemoglobin component distribution image obtaining unit” in claim 11 line 9, and “melanin distribution obtaining unit” in claim 11 line 14 are not described in any detail. Therefore, since applicant has not described a particular structure for performing each of the functions, a person skilled in the art at the time of the invention was filed would not have recognized that the inventor was in possession of the invention as claimed.
Regarding claim 12, applicant claims “an image enhancement processing unit configured for performing image enhancement processing on gray- scale images of contents of hemoglobin and melanin, and the image enhancement processing comprises maximum-minimum normalization, contrast enhancement, and histogram equalization.”
As per MPEP § 2181(IV), “A means- (or step-) plus-function limitation that is found to be indefinite under 35 U.S.C. 112(b) based on failure of the specification to disclose corresponding structure, material or act that performs the entire claimed function also lacks adequate written description” (emphasis added). Furthermore, as per MPEP § 2163.03(VI), “(s)uch a limitation also lacks an adequate written description as required by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C 112, first paragraph, because an indefinite, unbounded functional limitation would cover all ways of performing a function and indicate that the inventor has not provided sufficient disclosure to show possession of the invention.”
Since applicant has not defined any particular structure for the “image enhancement processing unit” in claim 12 line 1, the inventor has not provided sufficient disclosure to show possession of the invention. Applicant has not provided any specific definition for the structure that carry out the function disclosed in claim 12. Additionally, the claimed invention as a whole may not be adequately described if the claims require an essential or critical feature which is not adequately described in the specification and which is not conventional in the art or known to one of ordinary skill in the art. It appears that the components and/or features are essential and critical features of the applicant’s invention because without them, the applicant’s invention would not work. In particular, the structure of the “image enhancement processing unit” in claim 12 line 1 is not described in any detail. Therefore, since applicant has not described a particular structure for performing the function, a person skilled in the art at the time of the invention was filed would not have recognized that the inventor was in possession of the invention as claimed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Patwardhan (U.S. Patent No. US 8849380 B2) ("Patwardhan") in view of Jang et al. (Korean Publication No. KR 20170100717 A) ("Jang") and further in view of Barker et al. (International Publication No. WO 2016/183676 A1) ("Barker").
Regarding claim 1, Patwardhan discloses a separation method for image with facial skin component (Abstract lines 1-3; column 5 lines 46-48; column 17 lines 46-49), comprising the following steps of:
S1: collecting facial hyperspectral image data in different preset wavebands, which wavebands have obvious relative changes of hemoglobin component and melanin (column 12 lines 11-20 and 27-34, wherein hyperspectral imaging is a more specific type of multi-spectral imaging utilizing narrow, contiguous spectral bands, which Patwardhan also uses, and images (i.e. facial hyperspectral image data) is collected at different wavebands where melanin or hemoglobin have higher absorbance levels);
S2: performing white balance processing or absolute reflectance processing on the face hyperspectral image data in the different wavebands (column 13 lines 6-15, wherein white balance processing is performed on the captured images);
S3: using an image processing algorithm to obtain an original hemoglobin component distribution image (column 11 lines 58-60, wherein hemoglobin is listed as a chromophore of interest; column 13 lines 25-35, wherein the process of obtaining a 2D distribution of melanin is described as an example of obtaining a 2D distribution of a chromophore (i.e. original hemoglobin component distribution image)), and obtaining content distributions of melanin and hemoglobin component (column 11 lines 58-60, wherein hemoglobin is listed as a chromophore of interest; column 13 lines 25-35, wherein the process of obtaining a 2D distribution of melanin (i.e. content distribution of melanin component) is described as an example of obtaining a 2D distribution of a chromophore (i.e. content distribution of hemoglobin component))
S4: using an image processing algorithm to obtain a melanin content difference distribution image, and obtaining a melanin distribution image based on the content distribution of melanin component, wherein the image processing algorithm comprises image subtraction or image division (column 13 lines 25-35, wherein obtaining a 2D distribution of melanin (i.e. melanin content difference distribution image which shows the content distribution of melanin component) includes dividing the 360 to 370 nm image by the 650 to 1200 nm image).
Although, Patwardhan teaches a mathematical model of light/tissue interaction based upon the 2D or 3D chromophore absorption maps (column 14 lines 17-18), Patwardhan does not teach constructing a skin reflection model (emphasis added). Jang, on the other hand, teaches c. More specifically and as it relates to the applicant’s claims, Jang discloses constructing a skin reflectance model (Eq. 3; [0030] and [0032] (please see attached translation), wherein melanin and hemoglobin indicators are estimated using a skin spectral reflectance model (i.e. skin reflection model)).
Jang is combinable with Patwardhan because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to more efficiently and accurately analyze the skin of the entire face (Jang, [0004-0005] (please see attached translation)).
Additionally, Patwardhan and Jang fail to teach obtaining content distributions of melanin and hemoglobin component by making linear regression, specifically, on pixels comprising concentrations of melanin and hemoglobin component. Barker, on the other hand, teaches using a multilinear regression model on the images to separate the melanin and hemoglobin from the images. More specifically and as it relates to the applicant’s claims, Barker discloses obtaining content distributions of melanin and hemoglobin component by making linear regression on pixels comprising concentrations of melanin and hemoglobin component (p. 6 lines 14-16, wherein concentrations of melanin/hemoglobin at the pixel level of an image are utilized (i.e. pixels comprising concentrations of melanin and hemoglobin); Eq. 5 and 6; p. 13 lines 12-29 and p. 14 lines 6-20, wherein a multilinear regression (i.e. linear regression) model is made to obtain the distribution of hemoglobin and melanin by compensating for the hemoglobin and melanin covariance).
Barker is combinable with Patwardhan and Jang because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to more accurately image the concentration and distribution of biomolecules of interest (Barker, p. 6 lines 14-16).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate constructing a skin reflection model, as taught by Jang, and obtaining content distributions of melanin and hemoglobin component by making linear regression on pixels comprising concentrations of melanin and hemoglobin component, as taught by Barker, into the separation method, as taught by Patwardhan, to obtain the invention as specified in claim 1.
Claim 14 has a limitation that is substantially similar to claim 1. Therefore, the rejection applied to claim 1, please see above, also applies equally to claim 14. Furthermore, Jang discloses a non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed ([0070] (please see attached translation), wherein a computer-readable media including non-volatile media stores computer-readable instructions for a method).
Jang is combinable with Padwarthan, Jang, and Barker because they are from the same art of image processing. It is well known in the art to use a computer to execute instructions regarding a method and that storing a program with executable instructions in a non-transitory computer-readable medium is required in order for the processing to be executed on a computer.
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed, as taught by Jang, into the separation method, as taught by Patwardhan, Jang, and Barker, to obtain the invention as specified in claim 14.
Regarding claim 2, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 1. Additionally, Patwardhan discloses wherein the facial hyperspectral image data is taken by a hyperspectral imaging camera with a Full Width at Half Maximum (FWHM) less than 50nm (column 9 lines 52-68, wherein a camera is used to capture multi-spectral images (i.e. hyperspectral imaging camera); column 12 lines 4-9, wherein reflectance images (i.e. hyperspectral image data) is captured with a FWHM of approximately 10 to 40 nm (i.e. less than 50nm)).
Regarding claim 4, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 1. Patwardhan additionally teaches wherein the facial hyperspectral image data of three different preset wavebands is collected (column 12 lines 11-20 and 27-34, wherein hyperspectral imaging is a more specific type of multi-spectral imaging utilizing narrow, contiguous spectral bands, which Patwardhan also uses, and images (i.e. facial hyperspectral image data) is collected at different wavebands)
Since the claim language of claim 1 only requires either white balance processing or absolute reflectance processing, and Patwardhan teaches white balance processing (see the rejection of claim 1), then Patwardhan also reads on any limitation directed towards absolute reflectance processing since the limitations of absolute reflectance processing do not have to be met.
Regarding claim 16, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 4. Additionally, Patwardhan discloses wherein the using an image processing algorithm to obtain an original hemoglobin component distribution image, comprises: subtracting the facial hyperspectral image I2(x, y) by I3(x, y), or dividing I2(x, y) by I3(x, y) to obtain the original hemoglobin component distribution image O(x, y) (column 11 lines 58-60, wherein hemoglobin is listed as a chromophore of interest; column 13 lines 25-35, wherein obtaining a 2D distribution of a chromophore (i.e. hemoglobin) includes dividing the image taken at a wavelength corresponding to the chromophore (i.e. facial hyperspectral image I2(x, y)) by the 650 to 1200 nm image (i.e. I3(x, y)).
Claims 3 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Patwardhan (U.S. Patent No. US 8849380 B2) ("Patwardhan") in view of Jang et al. (Korean Publication No. KR 20170100717 A) ("Jang") and (International Publication No. WO 2016/183676 A1) ("Barker"), and further in view of Yuan (U.S. Publication No. 2019/0019312 A1) ("Yuan").
Regarding claim 3, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 1.
Patwardhan additionally teaches wherein the facial hyperspectral image data of three different
preset wavebands is collected, and the white balance processing in step S2 comprises: obtaining the white balance and obtaining three frames of facial hyperspectral images I1(x, y), I2(x, y), and I3(x, y) after the white balance. However, Patwardhan fails to teach obtaining the white balance by using the Gray World Algorithm, specifically. Yuan, on the other hand, teaches using the gray world algorithm to perform white balance processing on an image. More specifically and as it relates to the applicant’s claims, Yuan discloses obtaining the white balance by using the Gray World Algorithm ([0044]).
Yuan is combinable with Patwardhan, Jang, and Barker because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to achieve a better white balance effect (Yuan, [0049]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate obtaining the white balance by using the Gray World Algorithm as taught by Yuan, into the separation method, as taught by Patwardhan, Jang, and Barker, to obtain the invention as specified in claim 3.
Regarding claim 6, Patwardhan, Jang, Barker, and Yuan disclose the separation method for image with facial skin component according to claim 3. Additionally, Patwardhan discloses wherein the using an image processing algorithm to obtain an original hemoglobin component distribution image, comprises: subtracting the facial hyperspectral image I2(x, y) by I3(x, y), or dividing I2(x, y) by I3(x, y) to obtain the original hemoglobin component distribution image O(x, y) (column 11 lines 58-60, wherein hemoglobin is listed as a chromophore of interest; column 13 lines 25-35, wherein obtaining a 2D distribution of a chromophore (i.e. hemoglobin) includes dividing the image taken at a wavelength corresponding to the chromophore (i.e. facial hyperspectral image I2(x, y)) by the 650 to 1200 nm image (i.e. I3(x, y)).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Patwardhan (U.S. Patent No. US 8849380 B2) ("Patwardhan") in view of Jang et al. (Korean Publication No. KR 20170100717 A) ("Jang"), Barker et al. (International Publication No. WO 2016/183676 A1) ("Barker"), and Yuan (U.S. Publication No. US 2019/0019312 A1) ("Yuan") and further in view of Wang et al. (U.S. Publication No. 2022/0329767 A1) ("Wang") and Mir et al. (U.S. Publication No. US 2012/0253224 A1) ("Mir").
Regarding claim 8, Patwardhan, Jang, Barker, and Yuan disclose the separation method for image with facial skin component according to claim 3.
Although Patwardhan teaches obtaining the melanin content difference distribution image in the step S4 (column 13 lines 25-35, wherein obtaining a 2D distribution of melanin (i.e. melanin content difference distribution image which shows the content distribution of melanin component) includes dividing the 360 to 370 nm image by the 650 to 1200 nm image), Patwardhan, Jang, Barker, and Yuan fails to teach obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y). Wang, on the other hand, teaches subtracting the absorption values of hemoglobin from absorption values of melanin. More specifically and as it relates to the applicant’s claims, Wang discloses obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y) ([0074], wherein values of absorption caused by hemoglobin (i.e. hyperspectral image I2(x, y)) are subtracted from values of absorption caused by melanin (i.e. hyperspectral image I1(x, y)) to emphasize the melanin concentration (i.e. melanin content difference distribution image)).
Wang is combinable with Patwardhan, Jang, Barker, and Yuan because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to emphasize the contribution of reflections from desired wavelength bands while minimizing contributions of reflections from undesired wavelength bands (Wang, [0056]).
Patwardhan, Jang, Barker, Yuan, and Wang fail to teach obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y) which is processed with Gaussian blur, or by dividing the I1(x, y) by the I2(x, y) which is processed with Gaussian blur, specifically (emphasis added). Mir, on the other hand, teaches using Gaussian blur for subtracting images. More specifically, and as it relates to the applicant’s claims, Mir discloses obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y) which is processed with Gaussian blur (Fig. 17A; [0120], wherein Gaussian blur is applied to a baseline image (i.e. I2(x, y)), which is subtracted from an assessment image (i.e. I1(x, y)) to create an interim difference image (i.e. melanin content difference distribution image)), or by dividing the I1(x, y) by the I2(x, y) which is processed with Gaussian blur
Mir is combinable with Patwardhan, Jang, Barker, Yuan, and Wang because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable improved skin test image analysis ([Mir, [0057]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y), as taught by Wang, and processing with Gaussian blur, as taught by Mir, into the separation method, as taught by Patwardhan, Jang, Barker, and Yuan, to obtain the invention as specified in claim 8.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Patwardhan (U.S. Patent No. US 8849380 B2) ("Patwardhan") in view of Jang et al. (Korean Publication No. KR 20170100717 A) ("Jang") and Barker et al. (International Publication No. WO 2016/183676 A1) ("Barker"), and further in view of Hyde et al. (U.S. Publication No. US 2023/0346296 A1) ("Hyde").
Regarding claim 10, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 1.
However, Patwardhan, Jang, and Barker fail to teach wherein further comprises image enhancement processing on gray-scale images of contents of hemoglobin and melanin, and the image enhancement processing comprises maximum-minimum normalization, contrast enhancement, and histogram equalization. Hyde, on the other hand, teaches processing images using min-max normalization and enhancing contrast by using histogram equalization. More specifically and as it relates to the applicant’s claims, Hyde discloses wherein further comprises image enhancement processing on gray-scale images ([0059], wherein grayscale images are used) of contents of hemoglobin and melanin, and the image enhancement processing comprises maximum-minimum normalization ([0047]), contrast enhancement, and histogram equalization ([0049-0050], wherein contrast enhancement is done through histogram equalization).
Hyde is combinable with Patwardhan, Jang, and Barker because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve the overall image quality and overall confidence of the results (Hyde, [0047] and [0053]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein further comprises image enhancement processing on gray-scale images of contents of hemoglobin and melanin, and the image enhancement processing comprises maximum-minimum normalization, contrast enhancement, and histogram equalization, as taught by Hyde, into the separation method, as taught by Patwardhan, Jang, and Barker, to obtain the invention as specified in claim 10.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Patwardhan (U.S. Patent No. US 8849380 B2) ("Patwardhan") in view of Jang et al. (Korean Publication No. KR 20170100717 A) ("Jang") and Barker et al. (International Publication No. WO 2016/183676 A1) ("Barker"), and further in view of Wang et al. (U.S. Publication No. 2022/0329767 A1) ("Wang") and Mir et al. (U.S. Publication No. US 2012/0253224 A1) ("Mir").
Regarding claim 17, Patwardhan, Jang, and Barker disclose the separation method for image with facial skin component according to claim 4.
Although Patwardhan teaches obtaining the melanin content difference distribution image in the step S4 (column 13 lines 25-35, wherein obtaining a 2D distribution of melanin (i.e. melanin content difference distribution image which shows the content distribution of melanin component) includes dividing the 360 to 370 nm image by the 650 to 1200 nm image), Patwardhan, Jang, Barker, and Yuan fails to teach obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y). Wang, on the other hand, teaches subtracting the absorption values of hemoglobin from absorption values of melanin. More specifically and as it relates to the applicant’s claims, Wang discloses obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y) ([0074], wherein values of absorption caused by hemoglobin (i.e. hyperspectral image I2(x, y)) are subtracted from values of absorption caused by melanin (i.e. hyperspectral image I1(x, y)) to emphasize the melanin concentration (i.e. melanin content difference distribution image)).
Wang is combinable with Patwardhan, Jang, and Barker because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to emphasize the contribution of reflections from desired wavelength bands while minimizing contributions of reflections from undesired wavelength bands (Wang, [0056]).
Patwardhan, Jang, Barker, and Wang fail to teach obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y) which is processed with Gaussian blur, or by dividing the I1(x, y) by the I2(x, y) which is processed with Gaussian blur, specifically (emphasis added). Mir, on the other hand, teaches using Gaussian blur for subtracting images. More specifically, and as it relates to the applicant’s claims, Mir discloses obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y) which is processed with Gaussian blur (Fig. 17A; [0120], wherein Gaussian blur is applied to a baseline image (i.e. I2(x, y)), which is subtracted from an assessment image (i.e. I1(x, y)) to create an interim difference image (i.e. melanin content difference distribution image)), or by dividing the I1(x, y) by the I2(x, y) which is processed with Gaussian blur
Mir is combinable with Patwardhan, Jang, Barker, and Wang because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable improved skin test image analysis ([Mir, [0057]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate obtaining the melanin content difference distribution image ΔM(x, y) by subtracting the facial hyperspectral image I1 (x, y) by the I2(x, y), or by dividing the I1(x, y) by the I2(x, y), as taught by Wang, and processing with Gaussian blur, as taught by Mir, into the separation method, as taught by Patwardhan, Jang, and Barker, to obtain the invention as specified in claim 17.
Allowable Subject Matter
Claims 5, 7, 9, and 15 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 5, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest wherein the three preset different wavebands comprise 530-560nm, 575-585nm and 600-630nm, and the three frames of facial hyperspectral images in the corresponding three wavebands are obtained, in combination with the other elements of the claim.
The closest prior art, Patwardhan (U.S. Patent No. US 8849380 B2) discloses obtaining facial hyperspectral images in corresponding wavebands including 530 to 550 nm, 570 to 590 nm, and 600 to 630 nm but fails to disclose the three preset different wavebands being 530-560nm, 575-585nm and 600-630nm, specifically.
Regarding claim 7, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest wherein the constructing a skin reflection model comprises: based on the Lambert-Beer law, linear regression is performed on each of the pixels by combining light absorbance values of hemoglobin and melanin in two wavebands and the hyperspectral image data captured: -Log(R) =COO+ CMM, image information in two wavebands is used to obtain linear equation in two unknowns of CO and CM, and the concentration or content of hemoglobin and melanin at each pixel position is obtained, where R represents the reflectance of facial skin, O and M represent the light absorbance coefficients of hemoglobin and melanin respectively, CO and CM represent the corresponding concentration or content of hemoglobin and melanin, in combination with the other elements of the claim.
The closest prior arts disclose the following:
Jang et al. (Korean Publication No. KR 20170100717 A) discloses using a skin spectral reflectance model to estimate melanin and hemoglobin indicators. Barker et al. (International Publication No. WO 2016/183676 A1) discloses using a multilinear regression model on the images to separate the melanin and hemoglobin from the images with respect to their reflectance values. These prior arts fail to disclose wherein the constructing a skin reflection model comprises: based on the Lambert-Beer law, linear regression is performed on each of the pixels by combining light absorbance values of hemoglobin and melanin in two wavebands and the hyperspectral image data captured: -Log(R) =COO+ CMM, image information in two wavebands is used to obtain linear equation in two unknowns of CO and CM, and the concentration or content of hemoglobin and melanin at each pixel position is obtained, where R represents the reflectance of facial skin, O and M represent the light absorbance coefficients of hemoglobin and melanin respectively, CO and CM represent the corresponding concentration or content of hemoglobin and melanin.
Regarding claim 9, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest wherein the obtaining the melanin distribution image comprises: obtaining the melanin distribution image M(x, y) based on the image processing algorithm, M(x, y) =I3(x, y) + ΔM(x, y), or obtaining the melanin distribution image M(x, y) based on the skin reflection model, M(x, y) = M'(x, y) +ΔM(x, y), where M'(x, y) is the content distribution of melanin component obtained by the linear regression, in combination with the other elements of the claim.
The closest prior art, Barker et al. (International Publication No. WO 2016/183676 A1) discloses using a multilinear regression model on the images to remove covariance from the hemoglobin and image the concentration and distribution of melanin but fails to teach the skin reflection model or its equation.
Regarding claim 15, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest wherein the three preset different wavebands comprise 530-560nm, 575-585nm and 600-630nm, and the three frames of facial hyperspectral images in the corresponding three wavebands are obtained, in combination with the other elements of the claim.
The closest prior art, Patwardhan (U.S. Patent No. US 8849380 B2) discloses obtaining facial hyperspectral images in corresponding wavebands including 530 to 550 nm, 570 to 590 nm, and 600 to 630 nm but fails to disclose the three preset different wavebands being 530-560nm, 575-585nm and 600-630nm, specifically.
Claim 11 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, and 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), 1st paragraph, set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 11, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest a melanin distribution image obtaining unit: configured for using an image processing algorithm to obtain a melanin content difference distribution image, and obtaining a melanin distribution image based on the content distribution of melanin component, wherein the image processing algorithm comprises image subtraction or image division, in combination with the other elements of the claim.
The closest prior art, Barker et al. (International Publication No. WO 2016/183676 A1) discloses using a multilinear regression model on the images to remove covariance from the hemoglobin and image the concentration and distribution of melanin but fails to teach the skin reflection model or its equation. However, more specifically, Barker does not teach the algorithm of “obtaining the melanin distribution image M(x, y) based on the image processing algorithm, M(x, y)= I3(x, y)+ ΔM(x, y), or obtaining the melanin distribution image M(x, y) based on the skin reflection model, M(x, y) = M'(x, y) + ΔM(x, y), where M'(x, y) is the content distribution of melanin component obtained by the linear regression” (See [0020], [0049], and [0059] of the specification) associated with the 112(f) limitations for a “melanin distribution image obtaining unit”.
Further, it should be noted that claim 11 is interpreted under 35 U.S.C 112(f) thus, “Therefore, the broadest reasonable interpretation of a claim limitation that invokes 35 U.S.C 112(f) is the structure, material, or acts described in the specification as performing the entire claimed function and equivalents to the disclosed structure, material or act. As a result, section 112(f) limitations will, in some cases, be afforded a more narrow interpretation than a limitation that is not crafted in “means plus function” format (See MPEP 2181).”
Claim 12 is dependent on claim 11 and would likewise be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, and 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), 1st paragraph, set forth in this Office action.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL Y DANG whose telephone number is (571)438-9519. The examiner can normally be reached Monday - Thursday: 7am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL Y DANG/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights