DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: extracting, by a reference analysis engine; extracting, by a user image analysis engine in; generating, by a virtual try-on engine claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (Publication: US 2021/0406996 A1) in view of Ivie et al. (Patent: US 10,395,297 B1).
Regarding claim 1, Yu discloses a non-transitory computer-readable medium having stored thereon instructions configured to, when executed by one or more computing devices of a computer system, cause the computer system to perform operations comprising ([0048], [0049] , Fig. 4 - System 40 is a computer device 402, It is known that a computer has instructions stored in the memory and processed by CPU to perform the following methods: ):
extracting by a reference analysis engine, one or more reference attributes of a reference image or graphical avatar ([0139] FIG. 17, a facial-attribute unit 1702 “a reference analysis engine” including processing circuitry configured to extract a plurality of facial attributes from a source image of a face.
In digital environments, an avatar is simply a graphical representation of a person.);
executing a machine learning model using the one or more reference attributes and the one or more target attributes as input to generate target image modification data as output ([0075] processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used.
[0081], [0085] - FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. In the present interface, the source image is presented as modified (420A) to present (e.g. overlay) information 802 about facial attributes. Facial attribute information 803 is presented contextually and in association with regions of the face 420 related to the attributes. the facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Control 808 (e.g. “Continue to my Make-up Looks”) advances to a recommendation interface, “one or more reference attributes and the one or more target attributes as input”.
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. The looks are keyed (stored in association) with facial attribute information such as where a product or technique to apply it is associated with (e.g. recommended for) one or more particular facial attributes. For example, to achieve a particular look for a face having: an oval face, almond eyes, thick brows, brown brows, red undertone, blond hair, etc.; then applicable products for face, eyes, brows, and lips are determined. After selecting the recommendation, “generate target image modification data as output”, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D); and
generating, by a virtual try-on engine, a modified version of the target user image based on the target image modification data generated by the machine learning model ([0075] - An augmented reality (AR) virtual try on method “virtual try-on engine” configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image, giving a virtual try user experience. processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used, “machine learning model”.
By way of summary, a camera frame or photo (e.g. as a source image) as well as rendering values (such as a red, green blue (RGB) color and color opacity) which represent makeup products are received as input to the method. The source image is processed, using face tracking, to determine key landmarks around a user's face in the source image. Using these landmarks, areas of the face relative to a makeup product, are masked out such as the user's lips for a lip related product. Next, the rendering values are used to draw the makeup onto the user's face in the masked out area, “generating, by a virtual try-on engine, a modified version of the target user image”.
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “modifying”.).
Yu does not disclose; however, Ivie discloses
extracting, by a user image analysis engine one or more target attributes of a target user image (column 9 line 51 - the presence of faces and other features within a digital photograph may also be extracted from the image file through standard recognition algorithms and procedures.
Column 15 line 46 - As is also shown in FIG. 5A, the outlines or silhouettes of the individuals 510A, 520A may be extracted from the photograph 502A and analyzed in order to identify data, attributes and/or other information regarding the personal tendencies of the person 530A who made the posting 500A. For example, the individual 510A is determined to be wearing a light blue wool hat 512A and a white ski jacket 516A, and having blonde hair 514A.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Yu with extracting, by a user image analysis engine one or more target attributes of a target user image as taught by Ivie. The motivation for doing is to be able to customized to the member’s interest thus provide smooth, easy-to-use and functional interfaces.
Regarding claim 2, see rejection on claim 13.
Regarding claim 3, see rejection on claim 13.
Regarding claim 4, see rejection on claim 14.
Regarding claim 5, see rejection on claim 15.
Regarding claim 6, see rejection on claim 16.
Regarding claim 7, see rejection on claim 16.
Regarding claim 8, see rejection on claim 17.
Regarding claim 9, see rejection on claim 17.
Regarding claim 10, see rejection on claim 18.
Regarding claim 11, see rejection on claim 19.
Regarding claim 12, see rejection on claim 1.
Regarding claim 13, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the machine learning model is trained on a training set of modified user images using a supervised learning approach (
[0056] Processes the source image of a face using a network model performing deep learning and supervised regression “supervised learning approach” to output a prediction for each of the facial attributes. ), and
wherein the training set of modified user images is labeled with corresponding avatar attributes ( [0087] FIG. 9B shows a region interface 910 to highlight visually the region 912 related to the facial attributes of the brows and provide further information in portion 914 such as brow shape, brow color (e.g. “Brows Shape & Colour Information Make-Up Tips”, “labeled with corresponding avatar attributes”).
[0057] - the network model comprises a convolutional neural network (CNN) model comprising residual blocks performing deep learning to produce a feature vector of shared features for classification by respective classifiers to predict the facial attributes. Duplicates of the feature vector are made to be processed by a respective classifier for a respective one of the facial attributes. In an embodiment, the network model comprises a plurality of respective classifiers, each of the respective classifiers comprising one or more fully connected linear layers, wherein each of the respective classifiers providing as output a prediction of a respective one of the facial attributes “training set of modified user images”. The plurality of respective classifiers, in an embodiment, perform in parallel to provide the facial attributes.
In digital environments, an avatar is simply a graphical representation of a person.).
Regarding claim 14, Yu in view of Ivie disclose all the limitation of claim 13.
Yu discloses wherein the one or more reference attributes comprise a first set of color values, and wherein the target image modification data generated by the machine learning model comprises a second set of color values (
[0087] FIG. 9B shows a region interface 910 to highlight visually the region 912 related to the facial attributes of the brows and provide further information in portion 914 such as brow shape, brow color (e.g. “Brows Shape & Colour Information Make-Up Tips”.). ).
[0043] Table 2 shows outputs and evaluations for a model for predicting color attributes as trained. Attribute outputs annotated with an “*” represent outputs that are rarely predicted by the model. In an embodiment, additional training data is utilized to improve such predictions. Another approach includes combining or bundling these granular predictions together or with more common (and related) predictions, reducing granularity or fineness of the predictions, “a first set of color values; a second set of color values” .
PNG
media_image1.png
290
366
media_image1.png
Greyscale
).
Ivie discloses first set of hexadecimal color values and a second set of hexadecimal color values (column 10 line 5 - the colors white and black are expressed in RGB codes as 255, 255, 255 and 0, 0, 0, respectively, while the color National Flag Blue is expressed as 0, 38, 100. Colors may also be expressed according to a six-character hexadecimal model).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Yu in view Ivie with first set of hexadecimal color values and a second set of hexadecimal color values as taught by Ivie. The motivation for doing is to be able to customized to the member’s interest thus provide smooth, easy-to-use and functional interfaces.
Regarding claim 15, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the one or more reference attributes comprise one or more hair color values of [[a graphical avatar]], wherein the machine learning model uses the one or more hair color values of the [[graphical avatar]] as input to generate the target image modification data (
[0028], [0029] – hair attributes were trained.
PNG
media_image2.png
478
378
media_image2.png
Greyscale
[0100] - On an applicable user interface. An application and its interface is configurable to perform any one or more of: Use the hair color detection to recommend achievable colors; Use the hair color detection to extract hair colors from trending images to recommend related colors;
In digital environments, an avatar is simply a graphical representation of a person. ), and
wherein the target image modification data comprises one or more modified hair color values for the target user image ( [0028] – the hair color attributes are the attributes. During inferencing , the prediction (for each respective attribute) are aggregated at the end.
[0100] - On an applicable user interface. An application and its interface is configurable to perform any one or more of: Use the hair color detection to recommend achievable colors; Use the hair color detection to extract hair colors from trending images to recommend related colors; Use the facial/hair features to predict lifestyle.
In digital environments, an avatar is simply a graphical representation of a person. ).
Ivie discloses
hair color values of a graphical avatar (Column 15 line 46 - As is also shown in FIG. 5A, the outlines or silhouettes of the individuals 510A, 520A may be extracted from the photograph 502A and analyzed in order to identify data, attributes and/or other information regarding the personal tendencies of the person 530A who made the posting 500A. For example, the individual 510A is determined to be wearing a light blue wool hat 512A and a white ski jacket 516A, and having blonde hair 514A.
In digital environments, an avatar is simply a graphical representation of a person.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Yu in view Ivie with hair color values of a graphical avatar as taught by Ivie. The motivation for doing is to be able to customized to the member’s interest thus provide smooth, easy-to-use and functional interfaces.
Regarding claim 16, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the one or more reference attributes comprise a face color value of the reference image or graphical avatar ( [0062] - a first network model (e.g. a sub model) processes the source image for color-based facial attributes to produce a color-based feature vector for processing by respective ones of the plurality of classifiers configured to predict color-based facial attributes ) ,
wherein generating the modified version of the target user image comprises modifying a face area of the target user image based on the face color value of the reference image or graphical avatar ([0086] FIG. 9A shows a region interface 900 to highlight visually the region 902 related to the facial attributes of the face overall and provide further information (e.g. in portion 904) such as face shape, skin tone, undertone (e.g. “Shape, Skintone,& Undertone Information Make-Up Tips). Additional information in portion 904 relates to associated products for the region 902 and comprises graphics and/or text, color swatch images, etc.
PNG
media_image3.png
598
540
media_image3.png
Greyscale
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “generating the modified version of the target user image”.),
wherein the modified version of the target user image includes a depiction of a virtual cosmetic applied to the modified face area of the target user image, and wherein a color of the virtual cosmetic corresponds to the face color value of the reference image or graphical avatar (
[0087] - A control 906 is provided (e.g. “See My Recommendations”) to advance to product recommendation interface (e.g. FIG. 10A). A control 918 is provided to advance to another region interface such as for eyes (region interface for eyes not shown). Controls 920 are provided for advancing to specific region interfaces or the “My Results” interface (e.g. FIG. 8). Advancing to specific region interfaces may be navigated by swiping, or taping one of controls 920.
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “the modified version of the target user image”.).
Regarding claim 17, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the one or more reference attributes comprise a texture value of the reference image or graphical avatar, the method further comprising (
[0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
):
executing a machine learning model using the texture value of the reference image or graphical avatar as input to generate target image texture modification data as output ([0075] processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. an augmented reality (AR) virtual try on method configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image.
[0081], [0085] - FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. In the present interface, the source image is presented as modified (420A) to present (e.g. overlay) information 802 about facial attributes. Facial attribute information 803 is presented contextually and in association with regions of the face 420 related to the attributes. the facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Control 808 (e.g. “Continue to my Make-up Looks”) advances to a recommendation interface
PNG
media_image4.png
480
376
media_image4.png
Greyscale
) ; and
by the virtual try-on engine, further modifying the target user image based on the target image texture modification data by modifying a face area of the target user image based on the target image texture modification data, wherein the modified face area of the target user image includes a depiction of a virtual cosmetic applied to the modified face area of the target user image, the depicted virtual cosmetic having a texture or finish based on the target image texture modification data (
[0075] - processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. An augmented reality (AR) virtual try on method “virtual try-on engine” configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image, giving a virtual try user experience. By way of summary, a camera frame or photo (e.g. as a source image) as well as rendering values (such as a red, green blue (RGB) color and color opacity) which represent makeup products are received as input to the method. The source image is processed, using face tracking, to determine key landmarks around a user's face in the source image, “the modified face area of the target user image”. Using these landmarks, areas of the face relative to a makeup product, are masked out such as the user's lips for a lip related product. Next, the rendering values are used to draw the makeup onto the user's face in the masked out area, “a virtual cosmetic applied to the modified face area of the target user image”.
[0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture, “texture”.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “generating the modified version of the target user image”.).
Regarding claim 18, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the one or more reference attributes of the reference image or graphical avatar comprise length, texture, or style of hair of the reference image or graphical avatar, the method further comprising (
[0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture, “texture”.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
):
executing a machine learning model using the length, texture, or style of the hair of the reference image or graphical avatar as input to generate target image hair modification data as output ([0075] processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. an augmented reality (AR) virtual try on method configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image.
[0081], [0085] - FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. In the present interface, the source image is presented as modified (420A) to present (e.g. overlay) information 802 about facial attributes. Facial attribute information 803 is presented contextually and in association with regions of the face 420 related to the attributes. the facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Control 808 (e.g. “Continue to my Make-up Looks”) advances to a recommendation interface
PNG
media_image4.png
480
376
media_image4.png
Greyscale
); and
by the virtual try-on engine, further modifying the target user image based on the target image hair modification data ([0075] - processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. An augmented reality (AR) virtual try on method “virtual try-on engine” configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image, giving a virtual try user experience. By way of summary, a camera frame or photo (e.g. as a source image) as well as rendering values (such as a red, green blue (RGB) color and color opacity) which represent makeup products are received as input to the method. The source image is processed, using face tracking, to determine key landmarks around a user's face in the source image. Using these landmarks, areas of the face relative to a makeup product, are masked out such as the user's lips for a lip related product. Next, the rendering values are used to draw the makeup onto the user's face in the masked out area, “a virtual cosmetic applied to the modified face area of the target user image”.
[0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture, “texture”.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “generating the modified version of the target user image”.).
Regarding claim 19, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses wherein the one or more reference attributes of the reference image or graphical avatar comprise length, density, color, or texture of eyelashes of the reference image or graphical avatar, the method further comprising [0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture, “texture”.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
):
executing a machine learning model using the length, density, color, or texture of eyelashes of the reference image or graphical avatar as input to generate target image eyelash modification data as output ([0075] processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. an augmented reality (AR) virtual try on method configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image.
[0081], [0085] - FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. In the present interface, the source image is presented as modified (420A) to present (e.g. overlay) information 802 about facial attributes. Facial attribute information 803 is presented contextually and in association with regions of the face 420 related to the attributes. the facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Control 808 (e.g. “Continue to my Make-up Looks”) advances to a recommendation interface. Table 1 includes “eyelashes length”
PNG
media_image4.png
480
376
media_image4.png
Greyscale
); and
by the virtual try-on engine, further modifying the target user image based on the target image eyelash modification data ([0075] - processing the source image to apply the at least one facial effect comprises using a deep learning network (e.g. GANs-based) configured to apply the facial effect or other deep-learning models or other image processing techniques are used. An augmented reality (AR) virtual try on method “virtual try-on engine” configured in a (client-side) computing device is used to simulate an effect associated with a recommended product or service on a source image, giving a virtual try user experience. By way of summary, a camera frame or photo (e.g. as a source image) as well as rendering values (such as a red, green blue (RGB) color and color opacity) which represent makeup products are received as input to the method. The source image is processed, using face tracking, to determine key landmarks around a user's face in the source image. Using these landmarks, areas of the face relative to a makeup product, are masked out such as the user's lips for a lip related product. Next, the rendering values are used to draw the makeup onto the user's face in the masked out area, “a virtual cosmetic applied to the modified face area of the target user image”.
[0081] FIG. 8 shows an interface (e.g. a screen 800) for face analysis for presenting facial attributes determined from the source image 420. The facial attribute information identifies the attribute and associated value from Table 1 as determined for the source image 420. Table 1 shows Hair texture, “texture”.
PNG
media_image4.png
480
376
media_image4.png
Greyscale
[0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D, “modifying”.).
Regarding claim 20, Yu in view of Ivie disclose all the limitation of claim 12.
Yu discloses presenting in a user interface a product or style recommendation based on the one or more reference attributes, the target image modification data, the modified version of the target user image, or a combination thereof ([0090], [0093] FIGS. 10A-10D illustrates initial screens 1000, 1010, 1020 and 1030 of a recommendation interface. The looks are keyed (stored in association) with facial attribute information such as where a product or technique to apply it is associated with (e.g. recommended for) one or more particular facial attributes. For example, to achieve a particular look for a face having: an oval face, almond eyes, thick brows, brown brows, red undertone, blond hair, etc.; then applicable products for face, eyes, brows, and lips are determined. After selecting the recommendation, FIGS. 11A and 11B show a product recommendation interface (e.g. screen 1100 shown in two parts as 1100A and 1100B) in which reality is simulated. The selected look with product recommendations determined to match the facial attributes are simulated on source image 420 to present image 420D).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ming Wu whose telephone number is (571)270-0724. The examiner can normally be reached on Monday - Friday: 9:30am - 6:00pm EST .
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MING WU/
Primary Examiner, Art Unit 2618
Prosecution Insights