Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status
Claim 13 has been amended. Claims 1-8 have been previously withdrawn.
Claims 9-19 are rejected as set forth below.
Priority
This application claims the following priority:
FR2303587 4/11/2023
63/435603 12/28/2022
Note: Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. FR2303587, filed on 4/11/2023.
IDS
The following IDS(s) have been previously considered by the Examiner: 12/26/23, 4/9/24, 7/17/24, 11/12/24
Response to Arguments
Applicant traverses the rejection of claims 9-19 under 35 U.S.C. 101 as allegedly being directed to an abstract idea without “significantly more.”
The Applicant admits that the claims recite techniques that may be used in a virtual try on application, but argues that the claims themselves are directed to solving a technical problem (e.g., rendering a glitter effect in an output image) and are not directed to organizing human activity.
The Examiner notes, the claims as written, do not recite a specific technological improvement to computer functionality or graphics processing. The claim states results, without specifying concrete algorithmic steps, parameters, architectural constraints, or processing changes that improve operations.
Accordingly, although the claim frames the work in a graphics processing context, it does so via results-oriented and field-of-use limitations rather than articulating a specific practical application that improves the technology itself.
The Applicant notes, Applicant's specification explains that computing position and light reflection for glitter particles "may require too much computational power to be run in live mode on a mobile device." In an illustrative embodiment, "to off-load the computation during live mode rendering, texture maps (position, and particle properties) are pre-generated before the rendering occurs." (See applicant's published application at paragraphs [0069] and [0070].
The Examiner notes, the purported improvement is not reflected in the claim language. The specification can and should describe the improvement, but it cannot, by itself, confer eligibility if the claims recite only results or generic implementation.
The Applicant argues that Sartori does not teach or suggest pre-computed texture and light environment maps. The applicant argues that Sartori Odizzio uses a simplistic approach that ignores the lighting environment, and does not use pre-computed texture and light environment maps.
The Examiner disagrees, Sartori Odizzio does not ignore the lighting environment. For example, paragraph 0054, discloses a light focus filter, and generally operates as a mask filter and serves to add non-uniform illumination to a masked region of interest in the underlying base image. In other words, a light focus filter can be used to simulate the effect of reflected light on a portion of the base image. This has particular application for simulating the effect of a high gloss makeup product (e.g., lip gloss) when applied. In an embodiment, the filter includes a reference curve (e.g., the contour of an eye) that can be defined by user input and/or automatically using facial feature detection. Image processing (e.g., lightening/darkening) is applied according to the reference curve such that the light decays as the position moves away from the reference curve. The resulting effect simulates light falling on a three-dimensional surface even where the process ed image is in two dimensions.
This referenced lighting curve, is functionally equivalent to the light environment map as disclosed by Applicant.
The Applicant argues that Sartori Odizzio does not address a reflectance angle.
The Examiner notes, paragraph 0054 discusses the direction of the simulated light which is functionally equivalent to the reflectance angle.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 9-19 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more.
Step 1 (Statutory Categories)
Claim(s) 9-19: method/process.
Accordingly, claim(s) 9-19 pass step 1.
Step 2A, Prong 1 (Does the claim recite an abstract idea, law of nature, or natural phenomena?)
As seen in MPEP 2106.04(a) Abstract Ideas [R-07.2022], the enumerated groupings of abstract ideas are as follows:
1) Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), subsection I);
2) Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (see MPEP § 2106.04(a)(2), subsection II); and
3) Mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III).
Regarding independent claim(s) 9 and 15, the claims are directed to an abstract idea. Representative claim 9 recites:
rendering a makeup effect in association with the facial feature, the makeup effect including a glitter effect defined from pre-computed texture and light environment maps for locating and lighting the glitter effect; and
providing an output image defined from the input image and the makeup effect for
The claim as a whole is directed to providing a virtual try on experience, which is an abstract idea because it is a method of organizing human activity consistent with MPEP 2106.04(a) Abstract Ideas [R-07.2022].
**For the purposes of the analysis under 35 USC 101 (STEP2A – Prong 1&2, and STEP 2B): Strikethrough(s) are used to designate aspects of the claim(s) that are being interpreted as additional elements and not part of the claimed abstraction.
Dependent claims
Subsequent dependent further recite at least the abstract idea of:
… wherein the makeup effect comprises a lip makeup effect, an eye region makeup effect, or a cheek region makeup.
… wherein the step of rendering is responsive to physical based rendering techniques to define the glitter effect.
….a pre-computed texture map defines, for each of a plurality of glitter particles, a glitter location and a glitter reflectance angle;
….a light environment map simulates a light source for determining a specular brightness of a particular glitter particle in accordance with the reflectance angle of the particular glitter particle;
…the rendering defines pixel values for pixels of the makeup effect, adjusting a lighting up of a current pixel in accordance with a distance of the current pixel to a glitter location of a particular particle, the specular brightness of the particular glitter particle, and a size of the particular glitter particle.
…the specular brilliance is further determined in response to a three dimensional shape of the makeup effect and a rotation of the face; or
…a size of the particular glitter particle is randomly assigned within a normalized range; or
…the lighting up is further responsive to a glow alpha of the particular glitter particle, defining a light fade from a center; or
…the rendering further colours the makeup effect responsive to a glitter colour of the particular glitter particle; or
…the plurality of glitter particles are randomly spaced in the texture map without overlapping and wherein the rendering repeats the defining of the pixel values using the texture map in a plurality of overlapping texture map layers to overlap glitter particles in the glitter effect, and, wherein the rendering varies a glitter particle density of the glitter effect, randomly determining whether a particular particle in the texture map exists in any one of the overlapping texture map layers; or
…anormal to the reflectance angle is used with the light environment map for determining spectral brilliance; or
…each of the glitter locations of the plurality of glitter particles are randomly assigned using Poisson disk sampling.
…wherein rendering is responsive to any one or more rendering effects parameters comprising color, reflection, color variation, density, intensity, size, or size variation parameters.
…pre-computing texture and light environment maps for locating and lighting a glitter effect as a component of a makeup effect to be rendered in association with a facial feature localized
…providing the pre-computed texture and light environment maps to a rendering pipeline of the make-up virtual try on application to render to an output image for display the makeup effect having the glitter effect.
…wherein the pre-computed texture and light environment maps are defined for generating the glitter effect in accordance with physical based rendering techniques that model the particles in a lit environment.
…wherein a texture map is precomputed in accordance with a Voronoi Diagram technique, providing random locations for a plurality of glitter particles about the glitter effect, and the texture map is further computed to comprise respectively a reflectance angle for each of the plurality of glitter particles.
…wherein the light environment map is defined to model a source of light, such that at a time of rendering, a specular brilliance is provided respectively for each of the plurality of glitter particles in the glitter effect, the specular brilliance responsive to the reflectance angle.
…wherein the locations of the glitter particles are assigned using Poisson disk sampling distribution.
The dependent claims recite the same additional elements as the parent claim(s) and/or fail to recite any additional elements. The dependent claims further reiterate the same abstract idea with further embellishments. There are aspects of the dependent claims that are directed towards Mathematical concepts, for example the recitation of Poisson disk sampling.
Therefore, the identified claims fall within the subject matter groupings of abstract ideas enumerated in MPEP 2106.04(a)(2).
Step 2A, Prong 2 (Does the claim recite additional elements that integrates the recited judicial exception into a practical application?)
The Supreme Court and Federal Circuit have identified a number of considerations as relevant to the evaluation of whether the claimed additional elements demonstrate that a claim is directed to patent-eligible subject matter.
Regarding the instant application, the judicial exception is not integrated into a practical application. In particular, the language above that was designated as additional elements are recited at a high level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components or merely uses a computer as a tool to perform an abstract idea. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Step 2B (Do the claim recite additional elements that amount to an inventive concept (aka “significantly more”) then the recited judicial exception?)
Regarding independent claim(s) 9 and 15, the claims are directed to an abstract idea without significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) (individually and in combination) are merely being used to apply the abstract idea using generic computer components or merely uses a computer as a tool to perform an abstract idea. In conclusion, merely “applying” the exception using generic computer components cannot provide an inventive concept. Therefor the independent claims are not patent eligible under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 9-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Sartori Odizzio US 2018/0075524.
Referring to claim 9, Sartori discloses a computer implemented method comprising executing by one or more processors the steps of: processing a face of an input image to determine a location of a facial feature (Sartori: FIG. 5 is a flow chart illustrating an example process for facial feature detection using the virtual makeup platform described with respect to FIGS. 2A-2B.), the input image processed by a face tracking engine comprising at least one deep neural network to localize facial features (Sartori 0077: “For example, an embodiment is contemplated in which transparent layers are composed over a displayed base video wherein generated makeup images rendered in the transparent layer(s) over the base video are continually updated to track the motion (i.e., orientation and/or position) of a subject captured in the video.”);
rendering a makeup effect in association with the facial feature, the makeup effect including a glitter effect defined from pre-computed texture and light environment maps for locating and lighting the glitter effect (Sartori 0082: “Without fully modeling the face in three dimensions, curvature lines may be defined that dictate how some filters are applied. For example, consider a virtual eyeshadow product with glitter. When the virtual product is applied, the glitter specs in the generated makeup image may follow a distribution pattern (i.e., a non-uniform distribution pattern) that approximates application of the real world product over contours of the face.”); and
providing an output image defined from the input image and the makeup effect for presenting via a user interface as a component of a virtual try on (Sartori: “FIGS. 9A-9L show a series of screen captures of an example graphical user interface for virtual makeup application.”).
Referring to claim 10, Sartori discloses a method wherein the makeup effect comprises a lip makeup effect, an eye region makeup effect, or a cheek region makeup (Sartori: “FIG. 7A shows example regions of interest in a face that may be defined based on the facial feature detection process described with respect to FIG. 5.”).
Referring to claim 11, Sartori discloses a method wherein the step of rendering is responsive to physical based rendering techniques to define the glitter effect (Sartori 0049).
Referring to claim 12, Sartori discloses a method wherein: a pre-computed texture map defines, for each of a plurality of glitter particles, a glitter location and a glitter reflectance angle (Sartori 0055-0056);
a light environment map simulates a light source for determining a specular brightness of a particular glitter particle in accordance with the reflectance angle of the particular glitter particle (Sartori 0055-0056); and
the rendering defines pixel values for pixels of the makeup effect, adjusting a lighting up of a current pixel in accordance with a distance of the current pixel to a glitter location of a particular particle, the specular brightness of the particular glitter particle, and a size of the particular glitter particle (Sartori 0055-0056).
Referring to claim 13, Sartori discloses a method , wherein one or more of:
the specular brilliance is further determined in response to a three dimensional shape of the makeup effect and a rotation of the face (Sartori 0054, “The resulting effect simulates light falling on a three dimensional surface even where the process ed image is in two dimensions.”); or
a size of the particular glitter particle is randomly assigned within a normalized range (Sartori 0054-0056); or
the lighting up is further responsive to a glow alpha of the particular glitter particle, defining a light fade from a center (Sartori 0054-0056); or
the rendering further colours the makeup effect responsive to a glitter colour of the particular glitter particle (Sartori 0054-0056); or
the plurality of glitter particles are randomly spaced in the texture map without overlapping and wherein the rendering repeats the defining of the pixel values using the texture map in a plurality of overlapping texture map layers to overlap glitter particles in the glitter effect, and, preferably, wherein the rendering varies a glitter particle density of the glitter effect, randomly determining whether a particular particle in the texture map exists in any one of the overlapping texture map layers (Sartori 0054-0056); or
anormal to the reflectance angle is used with the light environment map for determining spectral brilliance (Sartori 0054-0056); or
each of the glitter locations of the plurality of glitter particles are randomly assigned using Poisson disk sampling (Sartori 0079 “If the facial features are detected and accurate, process 500 continues to step 512 where data associated with the detected facial features (i.e., region(s) of interest) is output for use in applying virtual makeup effects. In some embodiments, the accuracy of the detected facial features may be determined based on a user input indicative of a confirmation that the detected features are accurate.”).
Referring to claim 14, Sartori discloses a method of claim wherein rendering is responsive to any one or more rendering effects parameters comprising color, reflection, color variation, density, intensity, size, or size variation parameters (Sartori 0077).
Claims 15-19 are rejected under the same rationale as set forth above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW S GART whose telephone number is (571)272-3955. The examiner can normally be reached M-F 8:30AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Deborah Reynolds can be reached at 571-272-0734. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW S GART/Supervisory Patent Examiner, Art Unit 3696