DETAILED ACTION
Notice of AIA Status
The present application is being examined under the AIA the first inventor to file provisions.
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “130” has been used to designate both Weakly supervised loss function computation module and Supervised loss function computation module in Fig. 3 and Fig. 4, Respectively.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claims 1-2, 7, and 12 are objected to because of the following informalities:
In claim 1, Page 1 Line 12 the term “from fewer than a predetermined” should be changed to “from fewer than the predetermined” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
In claim 1, Page 1 Line 13 the term “the texture map as inputs” should be changed to “the texture map as the inputs” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
In claim 2, Page 1 Line 25 the term “an MSE calculated” should be changed to “aMean Squared error (MSE) calculated” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
In claim 7, Page 1 Line 3 the term “an MSE calculated” should be changed to “aMean Squared error (MSE) calculated” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
In claim 12, Page 5 Line 18 the term “converting each of predetermined number” should be changed to “converting each of a predetermined number” for typographical/grammar issues to avoid clarity issues.
In claim 12, Page 5 Line 25 the term “an MSE calculated” should be changed to “aMean Squared error (MSE) calculated” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
In claim 12, Page 6 Line 3 the term “fewer than a predetermined number” should be changed to “fewer than the predetermined number” for typographical/grammar issues to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1-2, 4, 6, 8, and 12, recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 1; recites the limitation, “a weakly supervised learning device that converts” Page 1 [Line 4].
Claim 1; recites the limitation, “a supervised learning device that estimates” Page 1 [Line 9].
Claim 1; recites the limitation, “on the basis of a weakly supervised learning device” Page 1 [Line 10-11].
Claim 2; recites the limitation, “a preprocessing module that converts” Page 1 [Line 17].
Claim 2; recites the limitation, “a weakly supervised learning module that trains” Page 1 [Line 22].
Claim 2; recites the limitation, “weakly supervised loss function computation module that trains” Page 1 [Line 24].
Claim 4; recites the limitation, “a wrinkle region derivation module that derives” Page 2 [Line 8].
Claim 4; recites the limitation, “determined by at least one annotator” Page 2 [Line 13].
Claim 4; recites the limitation, “a supervised learning module that estimates” Page 2 [Line 17].
Claim 4; recites the limitation, “on the basis of the weakly supervised learning device” Page 2 [Line 19].
Claim 4; recites the limitation, “a supervised loss function computation module that fine-tunes” Page 2 [Line 19].
Claim 4; recites the limitation, “the supervised learning module is provided to output” Page 2 [Line 21].
Claim 6; recites the limitation, “pre-trained on the basis of the weakly supervised learning device” Page 3 [Line 9].
Claim 8; recites the limitation, “determined by at least one annotator” Page 4 [Line 13].
Claim 8; recites the limitation, “on the basis of the weakly supervised learning device” Page 4 [Line 18].
Claim 12; recites the limitation, “determined by at least one annotator” Page 6 [Line 6].
Claim 12; recites the limitation, “on the basis of a weakly supervised learning device” Page 6 [Line 11].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1-2, 4, 6, 8, and 12:
“a weakly supervised learning device” (Fig. 1, #100. Page 8 Lines [0010-19]- The weakly supervised learning device 100 is configured to convert predetermined number or more of collected facial images into RGB data, then to learn the extracted facial RGB data as inputs through the deep neural network to estimate each texture map, and to train the deep neural network by updating weights based on an MSE calculated from the difference between the estimated texture map and the ground truth texture map. Accordingly, referring to FIG. 3, the weakly supervised learning device 100 may include a preprocessing module 110, a weakly supervised learning module 120, and a weakly supervised loss function computation module 130. (Wherein Fig. 1, shows the weakly supervised learning device as a black box thus the weakly supervised learning device does not have sufficient structure associated with it.).).
“a supervised learning device” (Fig. 1, #200. Paragraph Page 10 Lines [0009-25]- Meanwhile, the supervised learning device 200 converts fewer than a predetermined number of input images into RGB data, extracts wrinkle RGB data by removing false positives such as teeth and hair from the converted RGB data, merges texture maps derived by a Gaussian filter for the extracted wrinkle RGB data on the basis of a channel-wise concatenation operation to output combined data, combines binary wrinkle data derived by wrinkle masks predefined on the basis of at least one of annotators A to C for the input images fewer than a predetermined number through a majority voting algorithm to generate and output a consolidated ground truth wrinkle data, estimates wrinkle data through transfer learning of the deep neural network pre-trained in the weakly supervised learning module 120 with the derived combined data as inputs, and fine-tunes the pre-trained deep neural network by fine-tuning the weight of the pre-trained deep neural network based on the soft dice loss calculated from the difference between the estimated wrinkle data of the supervised learning module and the ground truth wrinkle data. Accordingly, referring to FIG. 4, the supervised learning device 200 may include a wrinkle region derivation module 210, a supervised learning module 220, and a supervised loss function computation module 230. (Wherein Fig. 1, shows the supervised learning device as a black box thus the supervised learning device does not have sufficient structure associated with it.).).
“a preprocessing module” (Fig. 3, #110. Page 8, Lines [0020-25]- Here, the preprocessing module 110 converts a predetermined number or more of original images into RGB data by using a digital image technique, then extracts facial RGB data from the RGB data, and generates the ground truth texture map for the extracted facial RGB data through a Gaussian filter. In this case, the ground truth texture map T(x, y) that is output may be expressed by the following equation 1. (Wherein Fig. 3, shows the preprocessing module as a black box thus the preprocessing module does not have sufficient structure associated with it.).).
“a weakly supervised learning module” (Fig. 3, #120. Page 8, Lines [0020-25]- In addition, the extracted facial RGB data is provided to the weakly supervised learning module 120. The weakly supervised learning module 120 inputs the facial RGB data and learns the facial RGB data through the deep module neural network to output the estimated texture map including information about the contour, curvature, and skin texture of each face. (Wherein Fig. 3, shows the weakly supervised learning module as a black box thus the weakly supervised learning module does not have sufficient structure associated with it.).).
“weakly supervised loss function computation module” (Fig. 1, #130. Page 8, Lines [0021-27]- Next, the estimated texture map of the weakly supervised learning module 120 and the ground truth texture map of the preprocessing module 110 are provided to the weakly supervised loss function computation module 130, and the weakly supervised loss function computation module 130 trains the deep neural network by updating weights of the deep neural network based on the MSE calculated from the difference between the estimated texture map of the deep neural network and the ground truth texture map, and outputs the optimal texture map. (Wherein Fig. 3, shows the weakly supervised loss function computation module as a black box thus the weakly supervised loss function computation module does not have sufficient structure associated with it.).).
“a wrinkle region derivation module” (Fig. 4, #230. Page 10 Lines [0003-9]- The wrinkle region derivation module 210 extracts the wrinkle RGB data by removing the false positives such as teeth and hair from the RGB data of the input images fewer than a predetermined number, and outputs the combined data by merging the texture maps derived by a Gaussian filter for the extracted wrinkle RGB data through a channel- wise concatenation operation, and the output combined data is provided to the supervised learning module 220. (Wherein Fig. 4, shows the wrinkle region derivation module as a black box thus the wrinkle region derivation module does not have sufficient structure associated with it.).).
“annotator” (Fig. 5. Page 11 Lines [0010-14]- Meanwhile, the wrinkle region derivation module 210 generates the consolidated ground truth wrinkle data by combining each binary wrinkle data extracted by a plurality of wrinkle masks predefined on the basis of the annotators A to C for the collected images fewer than a predetermined number through a majority voting algorithm. (Wherein Fig. 4, shows the annotator as a black box thus the annotator does not have sufficient structure associated with it.).).
“a supervised learning module” (Fig. 4. #220. Page 11 Lines [0020-25]- The supervised learning module 220 estimates wrinkle data by fine- tuning a pre-trained deep neural network through the transfer learning of the pre-trained deep neural network in the weakly supervised learning muddle 120 by inputting the derived combined data. (Wherein Fig. 4, shows the supervised learning module as a black box thus the supervised learning module does not have sufficient structure associated with it.).).
“a supervised loss function computation module” (Fig. 4. #130 in Fig and #230 in Specification see objection above. Page 10 Lines [0002-8]- Subsequently, the supervised loss function computation module 230 fine-tunes the pre-trained deep learning neural network by fine-tuning the weight of the pre-trained deep neural network based on the soft dice loss calculated from the difference between the estimated wrinkle data of the supervised learning module and the ground truth wrinkle data of the wrinkle region derivation module 210. (Wherein Fig. 4, shows the supervised loss function computation module as a black box thus the supervised loss function computation module does not have sufficient structure associated with it.).).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2, 4, 6, 8, and 12 along with their dependent claims, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claims 1-2, 4, 6, 8, and 12 recites limitations:
Claim 1; recites the limitation, “a weakly supervised learning device that converts” Page 1 [Line 4].
Claim 1; recites the limitation, “a supervised learning device that estimates” Page 1 [Line 9].
Claim 1; recites the limitation, “on the basis of a weakly supervised learning device” Page 1 [Line 10-11].
Claim 2; recites the limitation, “a preprocessing module that converts” Page 1 [Line 17].
Claim 2; recites the limitation, “a weakly supervised learning module that trains” Page 1 [Line 22].
Claim 2; recites the limitation, “weakly supervised loss function computation module that trains” Page 1 [Line 24].
Claim 4; recites the limitation, “a wrinkle region derivation module that derives” Page 2 [Line 8].
Claim 4; recites the limitation, “determined by at least one annotator” Page 2 [Line 13].
Claim 4; recites the limitation, “a supervised learning module that estimates” Page 2 [Line 17].
Claim 4; recites the limitation, “on the basis of the weakly supervised learning device” Page 2 [Line 19].
Claim 4; recites the limitation, “a supervised loss function computation module that fine-tunes” Page 2 [Line 19].
Claim 4; recites the limitation, “the supervised learning module is provided to output” Page 2 [Line 21].
Claim 6; recites the limitation, “pre-trained on the basis of the weakly supervised learning device” Page 3 [Line 9].
Claim 8; recites the limitation, “determined by at least one annotator” Page 4 [Line 13].
Claim 8; recites the limitation, “on the basis of the weakly supervised learning device” Page 4 [Line 18].
Claim 12; recites the limitation, “determined by at least one annotator” Page 6 [Line 6].
Claim 12; recites the limitation, “on the basis of a weakly supervised learning device” Page 6 [Line 11].
Claims 1-2, 4, 6, 8, and 12 respectively invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed functions. The specification does not provide sufficient details such that one of the ordinary skill in the art would understand which structure performed(s) the claimed function.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-2, 4, 6, 8, and 12 along with their dependent claims, are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
As described above, the disclosure does not provide adequate structure to perform the claimed function in the recited limitation Claims 1-2, 4, 6, 8, and 12 recites limitations:
Claim 1; recites the limitation, “a weakly supervised learning device that converts” Page 1 [Line 4].
Claim 1; recites the limitation, “a supervised learning device that estimates” Page 1 [Line 9].
Claim 1; recites the limitation, “on the basis of a weakly supervised learning device” Page 1 [Line 10-11].
Claim 2; recites the limitation, “a preprocessing module that converts” Page 1 [Line 17].
Claim 2; recites the limitation, “a weakly supervised learning module that trains” Page 1 [Line 22].
Claim 2; recites the limitation, “weakly supervised loss function computation module that trains” Page 1 [Line 24].
Claim 4; recites the limitation, “a wrinkle region derivation module that derives” Page 2 [Line 8].
Claim 4; recites the limitation, “determined by at least one annotator” Page 2 [Line 13].
Claim 4; recites the limitation, “a supervised learning module that estimates” Page 2 [Line 17].
Claim 4; recites the limitation, “on the basis of the weakly supervised learning device” Page 2 [Line 19].
Claim 4; recites the limitation, “a supervised loss function computation module that fine-tunes” Page 2 [Line 19].
Claim 4; recites the limitation, “the supervised learning module is provided to output” Page 2 [Line 21].
Claim 6; recites the limitation, “pre-trained on the basis of the weakly supervised learning device” Page 3 [Line 9].
Claim 8; recites the limitation, “determined by at least one annotator” Page 4 [Line 13].
Claim 8; recites the limitation, “on the basis of the weakly supervised learning device” Page 4 [Line 18].
Claim 12; recites the limitation, “determined by at least one annotator” Page 6 [Line 6].
Claim 12; recites the limitation, “on the basis of a weakly supervised learning device” Page 6 [Line 11].
The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 9-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 9 is drawn to a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 6 on a computer as defined in the specification in Page 15, Lines [0015-25]- “The computer-readable medium may include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory.” thus explicitly defined to encompass both transitory and non-transitory, can be a signal or carrier wave etc; therefore, fail(s) to fall within at least one of the four categories of patent eligible subject matter. It has been understood by the office that the at least one computer readable media is the same as "computer program medium" and "computer usable medium". Therefore claim 9 does not fit within the recognized categories of statutory subject matter. See MPEP 2106.
The office respectfully recommends the applicant to amend claim 9 limitation “a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 6 on a computer” to reflect the limitation “a non-transitory computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 6 on a computer”.
Claim 10 is drawn to a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 7 on a computer as defined in the specification in Page 15, Lines [0015-25]- “The computer-readable medium may include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory.” thus explicitly defined to encompass both transitory and non-transitory, can be a signal or carrier wave etc; therefore, fail(s) to fall within at least one of the four categories of patent eligible subject matter. It has been understood by the office that the at least one computer readable media is the same as "computer program medium" and "computer usable medium". Therefore claim 10 does not fit within the recognized categories of statutory subject matter. See MPEP 2106.
The office respectfully recommends the applicant to amend claim 10 limitation “a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 7 on a computer” to reflect the limitation “a non-transitory computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 7 on a computer”.
Claim 11 is drawn to a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 8 on a computer as defined in the specification in Page 15, Lines [0015-25]- “The computer-readable medium may include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory.” thus explicitly defined to encompass both transitory and non-transitory, can be a signal or carrier wave etc; therefore, fail(s) to fall within at least one of the four categories of patent eligible subject matter. It has been understood by the office that the at least one computer readable media is the same as "computer program medium" and "computer usable medium". Therefore claim 11 does not fit within the recognized categories of statutory subject matter. See MPEP 2106.
The office respectfully recommends the applicant to amend claim 11 limitation “a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 8 on a computer” to reflect the limitation “a non-transitory computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 8 on a computer”.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 12 is drawn to a “computer program” per se, therefore, fail(s) to fall within a statutory category of invention, since applicant`s specification do not define the term “operating program”.
A claim directed to a computer program itself is non-statutory because it is not:
A process occurring as a result of executing the program, or
A machine programmed to operate in accordance with the program, or
A manufacture structurally and functionally interconnected with the program in a manner which enable the program to act as a computer component and realize its functionality, or
A composition of matter.
See MPEP § 2106.01. Data structures not claimed as embodied in computer readable media are descriptive material per se and are not statutory because they are not capable of causing functional change in the computer. See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data structure per se held non-statutory). Such claimed data structures do not define any structural and functional interrelationships between the data structure and other claimed aspects of the invention, which permit the data structure's functionality to be realized. In contrast, a claimed computer readable medium encoded with a data structure defines structural and functional interrelationships between the data structure and the computer software and hardware components which permit the data structure's functionality to be realized, and is thus statutory. Similarly, computer programs claimed as computer listings per se, i.e., the descriptions or expressions of the programs are not physical “things.” They are neither computer components nor statutory processes, as they are not “acts” being performed. Such claimed computer programs do not define any structural and functional interrelationships between the computer program and other claimed elements of a computer, which permit the computer program's functionality to be realized.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, and 9-10 are rejected under 35 U.S.C 103 as being unpatentable over Kim et al. (US 20260004608 A1) hereafter referenced as Kim in view of Dhawan et al. (US 20240293073 A1) hereafter referenced as Dhawan.
Regarding claim 1, Kim teaches a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
extracts facial RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs (Fig. 1, Paragraph [0075]- Kim discloses the generating of the labeling data in S100 may include generating a texture map T corresponding to a training facial image I by using a Gaussian filter.);
by using combined data of preprocessed wrinkle RGB data from fewer than a predetermined number of input images (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.)
and the texture map as inputs (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter).
Kim fails to explicitly teach a supervised learning device that estimates wrinkle data through transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device
However, Dhawan explicitly teaches a supervised learning device that estimates wrinkle data through transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim of having a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Dhawan a supervised learning device that estimates wrinkle data through transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device.
Wherein having Kim’s system for detecting facial wrinkles wherein a supervised learning device that estimates wrinkle data through transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device.
The motivation behind the modification would have been to allow for greater accuracy without the need of a large training dataset, since both Kim and Dhawan are both systems that determine information about facial wrinkles. Wherein Kim’s system wherein improved accuracy of data labeling, while Dhawan’s system allows for an accurate system without the need for a large training set. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Dhawan et al. (US 20240293073 A1) Paragraph [0070].
Regarding claim 2, Kim in view of Dhawan teaches the facial wrinkle detection system of claim 1, Kim further teaches wherein the weakly supervised learning device comprises: a preprocessing module that converts each of the predetermined number or more of the collected images to the RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
extracts RGB data of a facial region from the converted RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
and then derives a ground truth texture map for the facial RGB data through a Gaussian filter (Fig. 1, Paragraph [0075]- Kim discloses specifically, the generating of the labeling data in S100 may include generating a texture map T corresponding to a training facial image I by using a Gaussian filter; generating a binary mask corresponding to a rough wrinkle-labeled image RA obtained by primarily pre-labeling wrinkles from the training facial image so as to correspond to the training facial image; removing a non-wrinkle texture from the texture map T by using the binary mask M; and generating the labeling data GT by performing adaptive thresholding on a corrected texture map T′ obtained by removing the non-wrinkle texture from the texture map T.);
a weakly supervised learning module that trains the deep neural network with the facial RGB data and estimates a texture map (Fig. 1, Paragraph [0070]- Kim discloses the inputting of the facial image of a user into the wrinkle detection model trained through supervised learning in S120 may include inputting the facial image of a user and the texture map corresponding to the facial image of a user into the wrinkle detection model trained through supervised learning.);
and a weakly supervised loss function computation module that trains the deep neural network by updating weights based on an MSE calculated from the difference between the estimated texture map and the ground truth texture map (Fig. 4, Paragraph [0061]- Kim discloses the training of the wrinkle detection model MD through the supervised learning in S110 may include repeatedly performing of inputting a training facial image used as a training data set and a texture map corresponding to the training facial image into the wrinkle detection model MD, comparing a wrinkle detection image (a predicted output (PO)) obtained as the output of the wrinkle detection model MD with the labeling data GT on the basis of a loss function, and adjusting parameters constituting the wrinkle detection model MD on the basis of the comparison result while changing the training facial image. Further in Fig. 4, Paragraph [0063]- Kim discloses the loss function (Loss) may use mean squared error (MSE) and cross-entropy error. Meanwhile, since the number of pixels corresponding to wrinkles in a facial image is relatively very small among pixels constituting an entire facial image, applying the mean squared error or cross-entropy error may cause a data imbalance, and thus the adjusting of parameters using the loss function may not converge quickly.).
Regarding claim 3, Kim in view of Dhawan teaches the facial wrinkle detection system of claim 1, Kim further teaches wherein the texture map comprises facial contours (Fig.4, Paragraph [0109]- Kim discloses as shown in Table 1 above, it could be seen that the wrinkle detection model MD proposed in the present disclosure showed slightly higher performance when using an input image generated by concatenating the facial image I of a user and the corresponding texture map T than when performing wrinkle detection by simply using the facial image I of a user as an input image. (wherein Fig. 4 shows the texture map T includes facial contours of the forehead).),
curves (Fig.4, Paragraph [0109]- Kim discloses as shown in Table 1 above, it could be seen that the wrinkle detection model MD proposed in the present disclosure showed slightly higher performance when using an input image generated by concatenating the facial image I of a user and the corresponding texture map T than when performing wrinkle detection by simply using the facial image I of a user as an input image. (wherein Fig. 4 shows the texture map T includes curves located on the forehead).),
and skin texture features (Fig.4, Paragraph [0109]- Kim discloses as shown in Table 1 above, it could be seen that the wrinkle detection model MD proposed in the present disclosure showed slightly higher performance when using an input image generated by concatenating the facial image I of a user and the corresponding texture map T than when performing wrinkle detection by simply using the facial image I of a user as an input image. (wherein Fig. 4 shows the texture map T includes skin texture features of the forehead).).
Regarding claim 5, Kim in view of Dhawan teaches the facial wrinkle detection system of claim 1, Kim further teaches wherein the wrinkle data comprises label information comprising wrinkle presence and background (Fig. 4, Paragraph [0080]- Kim discloses the generating of the binary mask M corresponding to the rough wrinkle-labeled image RA may include generating the binary mask M by setting a pixel value corresponding to a position at which wrinkles are labeled and a pixel value corresponding to a position at which wrinkles are not labeled in the rough wrinkle-labeled image RA to a pixel maximum value (e.g., 255 or white) and a pixel minimum value (e.g., 0 or black), respectively.).
Regarding claim 6, Kim in view of Dhawan teaches a facial wrinkle detection method performed on the basis of the facial wrinkle detection system of claim 1, Kim further teaches wherein at least one processor comprised in the facial wrinkle detection system comprises (Fig. 8, Paragraph [0115]- Kim discloses Referring to FIG. 8, the device 100 for detecting facial wrinkles may store at least one processor 110 and instructions for instructing the at least one processor 110 to perform at least one operation.):
a weakly supervised learning stage for converting each of the predetermined number or more of the collected images into the RGB data (Fig. 5, Paragraph [0088]- Kim discloses in FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
extracting the facial RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD (wherein the color information would be of the facial region as it is a facial image).),
and then estimating the texture map through the training of the deep neural network by using the extracted facial RGB data as the inputs (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter);
by using the combined data of the preprocessed wrinkle RGB data from fewer than the predetermined number of the input images (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.)
and the texture map as the inputs (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter).
Kim fails to explicitly teach a supervised learning stage for estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device.
However, Dhawan explicitly teaches a supervised learning stage for estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Dhawan of having a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Dhawan a supervised learning stage for estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device.
Wherein having Kim’s system for detecting facial wrinkles wherein a supervised learning stage for estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device.
The motivation behind the modification would have been to allow for greater accuracy without the need of a large training dataset, since both Kim and Dhawan are both systems that determine information about facial wrinkles. Wherein Kim’s system wherein improved accuracy of data labeling, while Dhawan’s system allows for an accurate system without the need for a large training set. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Dhawan et al. (US 20240293073 A1) Paragraph [0070].
Regarding claim 7, Kim in view of Dhawan teaches the facial wrinkle detection method of claim 6, Kim further teaches wherein the weakly supervised learning comprises: converting each of the predetermined number or more of the collected images into the RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
extracting RGB data of a facial region from the converted RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD (wherein the color information would be of the facial region as it is a facial image).),
and then deriving a ground truth texture map for the facial RGB data through a Gaussian filter (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter);
training the deep neural network with the facial RGB data and estimating a texture map (Fig. 1, Paragraph [0070]- Kim discloses the inputting of the facial image of a user into the wrinkle detection model trained through supervised learning in S120 may include inputting the facial image of a user and the texture map corresponding to the facial image of a user into the wrinkle detection model trained through supervised learning.);
and training the deep neural network by updating weights based on an MSE calculated from the difference between the estimated texture map and the ground truth texture map (Fig. 4, Paragraph [0061]- Kim discloses the training of the wrinkle detection model MD through the supervised learning in S110 may include repeatedly performing of inputting a training facial image used as a training data set and a texture map corresponding to the training facial image into the wrinkle detection model MD, comparing a wrinkle detection image (a predicted output (PO)) obtained as the output of the wrinkle detection model MD with the labeling data GT on the basis of a loss function, and adjusting parameters constituting the wrinkle detection model MD on the basis of the comparison result while changing the training facial image. Further in Fig. 4, Paragraph [0063]- Kim discloses the loss function (Loss) may use mean squared error (MSE) and cross-entropy error. Meanwhile, since the number of pixels corresponding to wrinkles in a facial image is relatively very small among pixels constituting an entire facial image, applying the mean squared error or cross-entropy error may cause a data imbalance, and thus the adjusting of parameters using the loss function may not converge quickly.),
and outputting an optimal texture map (Fig. 1, Paragraph [0087]- Kim discloses the wrinkle detection model MD generates an input image by concatenating the training facial image I and the texture map T corresponding thereto, and outputs the wrinkle detection image PO by receiving the generated input image).
Regarding claim 9, Kim in view of Dhawan teaches the method of claim 6, Kim further teaches a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 6 on a computer (Fig. 1 Paragraph [0122]- Kim discloses the methods according to the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and may be recorded in a computer-readable medium.).
Regarding claim 10, Kim in view of Dhawan teaches the method of claim 7, Kim further teaches a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 7 on a computer (Fig. 1 Paragraph [0122]- Kim discloses the methods according to the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and may be recorded in a computer-readable medium.).
Claims 4, 8, and 11 are rejected under 35 U.S.C 103 as being unpatentable over Kim et al. (US 20260004608 A1) hereafter referenced as Kim in view of Dhawan et al. (US 20240293073 A1) hereafter referenced as Dhawan and Rusko et al. (US 20210125707 A1) hereafter referenced as Rusko.
Regarding claim 4, Kim in view of Dhawan teaches the facial wrinkle detection system of claim 2, Kim further teaches wherein the supervised learning device comprises: a wrinkle region derivation module that derives combined data by combining the preprocessed wrinkle RGB data from fewer than the predetermined number of the input images (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.)
and a texture map derived from the wrinkle RGB data through the Gaussian filter (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter),
based on a channel-wise concatenation operation (Fig. 5, Paragraph [0088]- Kim discloses for example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
derives each of binary wrinkle data with a mask determined by at least one annotator for fewer than the predetermined number of the input images (Fig. 1, Paragraph [0080]- Kim discloses the generating of the binary mask M corresponding to the rough wrinkle-labeled image RA may include generating the binary mask M by setting a pixel value corresponding to a position at which wrinkles are labeled and a pixel value corresponding to a position at which wrinkles are not labeled in the rough wrinkle-labeled image RA to a pixel maximum value (e.g., 255 or white) and a pixel minimum value (e.g., 0 or black), respectively. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.),
and a supervised loss function computation module that fine-tunes a weight of the pre-trained deep neural network based on the soft dice loss calculated from the difference between the estimated wrinkle data and the ground truth wrinkle data (Fig. 5, Paragraph [0065]- Kim discloses In Mathematical expression 1, p.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the wrinkle detection image PO obtained as the output of the wrinkle detection model MD, g.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the labeling data GT, and the sigma operation may represent a sum of all pixel values for the x-coordinate or y-coordinate depending on the subscript notation (wherein the mathematical expression 1 shows a soft dice loss).),
Kim fails to explicitly teach a supervised learning module that estimates the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning module is provided to output optimal wrinkle data as a result of the transfer learning of the fine- tuned deep neural network.
However, Dhawan explicitly teaches a supervised learning module that estimates the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network);
wherein the supervised learning module is provided to output optimal wrinkle data as a result of the transfer learning of the fine- tuned deep neural network (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network. Fig. 1, Paragraph [0067]- Dhawan discloses some examples disclosed herein involve image classification using a set of training images associated with multiple classes. For the glabellar lines case, for example, those classes are mapped to a set of discrete scores (e.g., FIGS. 12-19) related to the severity of glabellar lines, namely zero (normal, no wrinkles) to three (severe).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Dhawan of having a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Dhawan a supervised learning module that estimates the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning module is provided to output optimal wrinkle data as a result of the transfer learning of the fine- tuned deep neural network.
Wherein having Kim’s system for detecting facial wrinkles wherein a supervised learning module that estimates the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning module is provided to output optimal wrinkle data as a result of the transfer learning of the fine- tuned deep neural network.
The motivation behind the modification would have been to allow for greater accuracy without the need of a large training dataset, since both Kim and Dhawan are both systems that determine information about facial wrinkles. Wherein Kim’s system wherein improved accuracy of data labeling, while Dhawan’s system allows for an accurate system without the need for a large training set. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Dhawan et al. (US 20240293073 A1) Paragraph [0070].
Kim in view of Dhawan fails to explicitly teach outputs a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
However, Rusko explicitly teaches outputs a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm (Fig. 1, Paragraph [0046]- Rusko discloses binarizing prediction values with a predetermined threshold value and determining voxels in which at least two of the corresponding three binary values represent presence of the object (‘majority vote’).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Dhawan of a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Rusko outputs a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
Wherein having Kim’s system for detecting facial wrinkles wherein outputs a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
The motivation behind the modification would have been to allow for greater accuracy of automatic segmentation, since both Kim and Rusko are both systems that determine binary data about the location of objects on a human. Wherein Kim’s system wherein improved accuracy of data labeling, while Rusko’s system more accurate automatic segmentation. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Rusko et al. (US 20210125707 A1) Paragraph [0016].
Regarding claim 8, Kim in view of Dhawan The facial wrinkle detection method of claim 6, Kim further teaches wherein the supervised learning comprises: deriving combined data by combining the preprocessed wrinkle RGB data from fewer than the predetermined number of the input images and a texture map derived from the wrinkle RGB data through the Gaussian filter (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.),
based on a channel-wise concatenation operation (Fig. 5, Paragraph [0088]- Kim discloses for example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
deriving each of binary wrinkle data with a mask determined by at least one annotator for fewer than the predetermined number of the input images (Fig. 1, Paragraph [0080]- Kim discloses the generating of the binary mask M corresponding to the rough wrinkle-labeled image RA may include generating the binary mask M by setting a pixel value corresponding to a position at which wrinkles are labeled and a pixel value corresponding to a position at which wrinkles are not labeled in the rough wrinkle-labeled image RA to a pixel maximum value (e.g., 255 or white) and a pixel minimum value (e.g., 0 or black), respectively. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.),
and fine-tuning a weight of the pre-trained deep neural network based on a soft dice loss calculated from the difference between the estimated wrinkle data of the supervised learning module and the ground truth wrinkle data (Fig. 5, Paragraph [0065]- Kim discloses In Mathematical expression 1, p.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the wrinkle detection image PO obtained as the output of the wrinkle detection model MD, g.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the labeling data GT, and the sigma operation may represent a sum of all pixel values for the x-coordinate or y-coordinate depending on the subscript notation (wherein the mathematical expression 1 shows a soft dice loss).),
Kim fails to explicitly teach estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
However, Dhawan explicitly teaches estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network);
wherein the supervised learning further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network. Fig. 1, Paragraph [0067]- Dhawan discloses some examples disclosed herein involve image classification using a set of training images associated with multiple classes. For the glabellar lines case, for example, those classes are mapped to a set of discrete scores (e.g., FIGS. 12-19) related to the severity of glabellar lines, namely zero (normal, no wrinkles) to three (severe).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Dhawan of having a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Dhawan estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
Wherein having Kim’s system for detecting facial wrinkles wherein estimating the wrinkle data through the transfer learning of the deep neural network pre-trained on the basis of the weakly supervised learning device with the combined data as the inputs; wherein the supervised learning further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
The motivation behind the modification would have been to allow for greater accuracy without the need of a large training dataset, since both Kim and Dhawan are both systems that determine information about facial wrinkles. Wherein Kim’s system wherein improved accuracy of data labeling, while Dhawan’s system allows for an accurate system without the need for a large training set. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Dhawan et al. (US 20240293073 A1) Paragraph [0070].
Kim in view of Dhawan fails to explicitly teach outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
However, Rusko explicitly teaches outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm (Fig. 1, Paragraph [0046]- Rusko discloses binarizing prediction values with a predetermined threshold value and determining voxels in which at least two of the corresponding three binary values represent presence of the object (‘majority vote’).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Dhawan of a facial wrinkle detection system comprising: a weakly supervised learning device that converts each of a predetermined number or more of collected images into RGB data, extracts facial RGB data, and then estimates a texture map through training of a deep neural network by using the extracted facial RGB data as inputs with the teachings of Rusko outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
Wherein having Kim’s system for detecting facial wrinkles wherein outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
The motivation behind the modification would have been to allow for greater accuracy of automatic segmentation, since both Kim and Rusko are both systems that determine binary data about the location of objects on a human. Wherein Kim’s system wherein improved accuracy of data labeling, while Rusko’s system more accurate automatic segmentation. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Rusko et al. (US 20210125707 A1) Paragraph [0016].
Regarding claim 11, Kim in view of Dhawan and Rusko teaches the method of claim 8, Kim further teaches a computer-readable recording medium having a program recorded for executing the facial wrinkle detection method of claim 8 on a computer (Fig. 1 Paragraph [0122]- Kim discloses the methods according to the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and may be recorded in a computer-readable medium.).
Claim 12 is rejected under 35 U.S.C 103 as being unpatentable over Kim et al. (US 20260004608 A1) hereafter referenced as Kim in view of Rusko et al. (US 20210125707 A1) hereafter referenced as Rusko and Dhawan et al. (US 20240293073 A1) hereafter referenced as Dhawan.
Regarding claim 12, Kim teaches an operating program of a facial wrinkle detection system (Fig. 1 Paragraph [0122]- Kim discloses the methods according to the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and may be recorded in a computer-readable medium.),
which is a computer program stored in a computer-readable recording medium for executing a facial wrinkle detection method on a computer by being coupled with the computer (Fig. 1 Paragraph [0123]- Kim discloses examples of computer-readable media may include a hardware device specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program instructions may include high-level language codes that can be executed by a computer by using an interpreter, etc.),
wherein the facial wrinkle detection method comprises: converting each of predetermined number or more of collected images into RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
extracting RGB data of a facial region from the converted RGB data (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD (wherein the color information would be of the facial region as it is a facial image).),
and then deriving a correct texture map for the facial RGB data through a Gaussian filter (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter);
training a deep neural network with the facial RGB data and estimating a texture map (Fig. 1, Paragraph [0070]- Kim discloses the inputting of the facial image of a user into the wrinkle detection model trained through supervised learning in S120 may include inputting the facial image of a user and the texture map corresponding to the facial image of a user into the wrinkle detection model trained through supervised learning.);
and training the deep neural network by changing a weight of the deep learning neural network by updating weights based on an MSE calculated from the difference between the estimated texture map and the ground truth texture map (Fig. 4, Paragraph [0061]- Kim discloses the training of the wrinkle detection model MD through the supervised learning in S110 may include repeatedly performing of inputting a training facial image used as a training data set and a texture map corresponding to the training facial image into the wrinkle detection model MD, comparing a wrinkle detection image (a predicted output (PO)) obtained as the output of the wrinkle detection model MD with the labeling data GT on the basis of a loss function, and adjusting parameters constituting the wrinkle detection model MD on the basis of the comparison result while changing the training facial image. Further in Fig. 4, Paragraph [0063]- Kim discloses the loss function (Loss) may use mean squared error (MSE) and cross-entropy error. Meanwhile, since the number of pixels corresponding to wrinkles in a facial image is relatively very small among pixels constituting an entire facial image, applying the mean squared error or cross-entropy error may cause a data imbalance, and thus the adjusting of parameters using the loss function may not converge quickly.),
wherein supervised learning stage comprises: deriving combined data by combining preprocessed wrinkle RGB data from fewer than a predetermined number of input images (Fig. 5, Paragraph [0088]- Kim discloses In FIG. 5, numbers displayed at the top of each layer represent a channel (also referred to as a depth). For example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.)
and a texture map derived from the wrinkle RGB data through the Gaussian filter (Fig. 1, Paragraph [0021]- Kim discloses the generating of the labeling data may include: generating the texture map corresponding to the training facial image by using a Gaussian filter),
based on a channel-wise concatenation operation (Fig. 5, Paragraph [0088]- Kim discloses for example, the training facial image I may be composed of three channels for each of R, G, and B, and the texture map T may be composed of one channel in grayscale, and an input image composed of four channels formed by concatenating these to each other is input to the wrinkle detection model MD.),
deriving each of binary wrinkle data with a mask determined by at least one annotator for fewer than the predetermined number of the input images (Fig. 1, Paragraph [0080]- Kim discloses the generating of the binary mask M corresponding to the rough wrinkle-labeled image RA may include generating the binary mask M by setting a pixel value corresponding to a position at which wrinkles are labeled and a pixel value corresponding to a position at which wrinkles are not labeled in the rough wrinkle-labeled image RA to a pixel maximum value (e.g., 255 or white) and a pixel minimum value (e.g., 0 or black), respectively. Further in Paragraph [0078]- Kim discloses in a Gaussian filter, the size of a Gaussian kernel may be 21×21, and the sigma (σ) value of the Gaussian filter may be set to 5, but this is an example, and the size of the Gaussian kernel and the sigma value may be set to be different depending on those skilled in the art. Further in Fig. 6, paragraph [0103]- Kim discloses to obtain the experimental results of FIGS. 6 and 7, 300 facial images were collected, and 250 facial images among them were used as the training facial images corresponding to the training data set to train the wrinkle detection model MD through supervised learning, and the remaining 50 facial images were used as data for performance verification.),
and fine-tuning a weight of the pre-trained deep neural network based on a soft dice loss calculated from the difference between the estimated wrinkle data of a supervised learning module and the ground truth wrinkle data (Fig. 5, Paragraph [0065]- Kim discloses In Mathematical expression 1, p.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the wrinkle detection image PO obtained as the output of the wrinkle detection model MD, g.sub.x,y may represent a pixel value for the x-coordinate or y-coordinate of the labeling data GT, and the sigma operation may represent a sum of all pixel values for the x-coordinate or y-coordinate depending on the subscript notation (wherein the mathematical expression 1 shows a soft dice loss).),
Kim fails to explicitly teach outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
However, Rusko explicitly teaches outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm (Fig. 1, Paragraph [0046]- Rusko discloses binarizing prediction values with a predetermined threshold value and determining voxels in which at least two of the corresponding three binary values represent presence of the object (‘majority vote’).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim of having an operating program of a facial wrinkle detection system, which is a computer program stored in a computer-readable recording medium for executing a facial wrinkle detection method on a computer by being coupled with the computer, wherein the facial wrinkle detection method comprises: converting each of predetermined number or more of collected images into RGB data, extracting RGB data of a facial region from the converted RGB data, and then deriving a correct texture map for the facial RGB data through a Gaussian filter; training a deep neural network with the facial RGB data and estimating a texture map with the teachings of Rusko outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
Wherein having Kim’s system for detecting facial wrinkles wherein outputting a consolidated ground truth wrinkle data by combining each of the binary wrinkle data through a majority voting algorithm.
The motivation behind the modification would have been to allow for greater accuracy of automatic segmentation, since both Kim and Rusko are both systems that determine binary data about the location of objects on a human. Wherein Kim’s system wherein improved accuracy of data labeling, while Rusko’s system more accurate automatic segmentation. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Rusko et al. (US 20210125707 A1) Paragraph [0016].
Kim in view of Rusko fails to explicitly teach estimating the wrinkle data through transfer learning of the deep neural network pre-trained on the basis of a weakly supervised learning device with the combined data as the inputs; wherein the supervised learning stage further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
However, Dhawan explicitly teaches estimating the wrinkle data through transfer learning of the deep neural network pre-trained on the basis of a weakly supervised learning device with the combined data as the inputs (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network);
wherein the supervised learning stage further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network (Fig. 1, paragraph [0070]- Dhawan discloses the method includes transfer learning, namely fine-tuning of a pre-trained model to the given domain (e.g. glabellar lines) to compensate for a lack of millions of annotated images required for a from-scratch end-to-end training of a deep convolutional network. Fig. 1, Paragraph [0067]- Dhawan discloses some examples disclosed herein involve image classification using a set of training images associated with multiple classes. For the glabellar lines case, for example, those classes are mapped to a set of discrete scores (e.g., FIGS. 12-19) related to the severity of glabellar lines, namely zero (normal, no wrinkles) to three (severe).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kim in view of Rusko of having an operating program of a facial wrinkle detection system, which is a computer program stored in a computer-readable recording medium for executing a facial wrinkle detection method on a computer by being coupled with the computer, wherein the facial wrinkle detection method comprises: converting each of predetermined number or more of collected images into RGB data, extracting RGB data of a facial region from the converted RGB data, and then deriving a correct texture map for the facial RGB data through a Gaussian filter; training a deep neural network with the facial RGB data and estimating a texture map with the teachings of Dhawan estimating the wrinkle data through transfer learning of the deep neural network pre-trained on the basis of a weakly supervised learning device with the combined data as the inputs; wherein the supervised learning stage further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
Wherein having Kim’s system for detecting facial wrinkles wherein estimating the wrinkle data through transfer learning of the deep neural network pre-trained on the basis of a weakly supervised learning device with the combined data as the inputs; wherein the supervised learning stage further comprises outputting optimal wrinkle data as a result of the transfer learning of the fine-tuned deep neural network.
The motivation behind the modification would have been to allow for greater accuracy without the need of a large training dataset, since both Kim and Dhawan are both systems that determine information about facial wrinkles. Wherein Kim’s system wherein improved accuracy of data labeling, while Dhawan’s system allows for an accurate system without the need for a large training set. Please see Kim et al. (US 20260004608 A1), Paragraph [0028] and Dhawan et al. (US 20240293073 A1) Paragraph [0070].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered
pertinent to applicant`s disclosure.
Chopde et al. (US 20230419170 A1)- Systems and methods employ knowledge distillation for efficient machine learning. Systems and methods integrate self-supervised learning, supervised learning, semi-supervised learning and active learning, each of which learning is executed in an iterative fashion. The system comprises three main components: a database server, a data analytics system and a standard dashboard. The database server contains real-time inventory images as well as historical images of each product type. The data analytics system is executed by a computer processor configured to apply a multi-head self-supervised learning-based deep neural network. The standard dashboard is configured to output a report regarding the object information....................Please see Fig. 1. Abstract.
Park et al. (US 20250005758 A1)- Embodiments relate to a skin diagnosis system and method based on image analysis using deep learning, containing: a face recognition model that derives shape or location information of a facial structure by recognizing feature points capable of identifying an individual in an acquired face image; a de-identification model that de-identifies a face image on the basis of the shape or location information of the facial structure such that personal information of an analysis target cannot be identified; and a plurality of artificial neural network models for each of at least one item among diagnoses for wrinkles, pigmentation, pores, erythema, and aging....................Please see Fig. 1. Abstract.
Miller et al. (US 20240355065 A1)- Described is a system for dynamically applying model adaptations customized for individual users by detecting an image of a first real-world object from a camera feed, detecting landmarks on the first real-world object, and processing the landmarks on the first real-world object using a generative machine learning model to generate a first custom image template for the first real-world object where portions of the first custom image template are populated with visual content placed based on the first custom image template. The system then applies a content augmentation based on the first custom image template to the camera feed....................Please see Fig. 1. Abstract.
Shon et al. (US 11521639 B1)- The present disclosure describes a system, method, and computer program for predicting sentiment labels for audio speech utterances using an audio speech sentiment classifier pretrained with pseudo sentiment labels. A speech sentiment classifier for audio speech (“a speech sentiment classifier”) is pretrained in an unsupervised manner by leveraging a pseudo labeler previously trained to predict sentiments for text. Specifically, a text-trained pseudo labeler is used to autogenerate pseudo sentiment labels for the audio speech utterances using transcriptions of the utterances, and the speech sentiment classifier is trained to predict the pseudo sentiment labels given corresponding embeddings of the audio speech utterances. The speech sentiment classifier is then subsequently fine tuned using a sentiment-annotated dataset of audio speech utterances, which may be significantly smaller than the unannotated dataset used in the unsupervised pretraining phase.....................Please see Fig. 1. Abstract.
Senn et al. (US 20240177288 A1)- Methods and systems for training a model for automated defect detection of a product during manufacturing are provided herein. Such methods utilize a combination of supervised transfer learning through auxiliary tasks and a combination of supervised and unsupervised learning. The methods can utilize supervised transfer learning with expert labels on a generalized auxiliary task, such as product classification, which is transferred to more specific auxiliary tasks, such as identification of specific product features and/or anomaly detection, where additional expert labels are then applied to the anomalies, and another iteration of supervised learning further improves the model. The anomalies can correspond to features associated with defects, which can be induced experimentally to improve efficiency of the training procedure. The product can be a sample cartridge such that the model allows detection of faulty cartridges based on sample cartridge and/or manufacturing process data......................Please see Fig. 1. Abstract.
Wang et al. (US 20200019938 A1)- The disclosed computer-implemented method for artificial-intelligence-based automated surface inspection can include receiving customer data, a request for a targeted model, and compensation for the requested targeted model. The compensation can include an agreement to contribute the customer data and/or targeted model to be available for other third-party entities. The method can also include retrieving the pre-trained model from a pre-trained model pool. The pre-trained model can be related to objects in a second industry. The method can include generating the targeted model from the pre-trained model and the customer data. The targeted model can be related to mapping sensor data to surface anomalies. The method can also include providing the targeted model to the third-party entity. The method can further include updating a distributed blockchain structure to include the at least one of the customer data and the targeted model.......................Please see Fig. 1. Abstract.
Benyoub et al. (US 7057595 B1)- The method for transforming a digital image (A) having several gray levels into a binary image (F) in which each pixel is coded over one bit, consists in applying, to each current pixel (P) of the digital image having several gray levels, several different parallel binarization processes (T1, T2, T3) each delivering as output a binary value for this current pixel and in combining (T4) the binary values delivered by the various binarization processes for each current pixel of the digital image having several gray levels so as to obtain a resultant binary value constituting the corresponding pixel of the binary image.......................Please see Fig. 1. Abstract.
GOKARN et al. (US 20210174228 A1)- The present invention relates to a method for processing a plurality of candidate annotations of a given instance of an image, each candidate annotation being defined as a closed shape matching the instance, characterized in that the method comprises performing, by a processing unit (21) of a server (2), steps of: (a) segregating said candidate annotations into a set of separate groups of at least overlapping candidate annotations; (b) selecting a subset of said groups as a function of the number of candidate annotations in each group; (c) building a final annotation of the given instance of said image as a combination of regions of the candidate annotations of said selected groups where at least a second predetermined number of the candidate annotations of said selected groups overlap.......................Please see Fig. 1. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUCIUS C.G. ALLEN whose telephone number is (703)756-5987. The examiner can normally be reached Mon - Fri 8-5pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571)272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUCIUS CAMERON GREEN ALLEN/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights