DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claim 4,5,11,13,14 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Potentially Allowable Subject Matter
Claim 15 is objected to as being dependent upon a rejected base, but would be allowable if rewritten in independent form including all of the limitations of the base claim, any intervening claims and correcting the minor informalities discussed below.
Claim Objections
Claim 15 objected to because of the following informalities: claim 15 recites “and generates navigation information generates navigation information” on lines 4-5. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1,2,3,6,7,9,16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff(US-20230101374-A1) in view of Amini(US-20130018494-A1) and Chang(US-20180133551-A1).
As per claim 1, “An apparatus comprising a processor configured to”, Kosecoff teaches a processor-based cosmetic AR/tutorial apparatus implemented using a smartphone/client computing device, camera, memory, and processors (Kosecoff teaches at (para. [0020]) “the example system 100 includes one or more client computing devices 130, a camera 140, one or more remote computer systems 160, also referred to as server(s), and one or more user devices 190.” This teaches an apparatus/system including computing devices for performing the cosmetic-routine functions. Kosecoff further teaches at (para. [0078]) “the computer-readable medium 730 stores computer-executable instructions that, in response to execution by one or more processors 715, cause the client computing device 130 to implement a control engine 731.” This expressly teaches processor-executed instructions.); “acquire a user video including a beauty motion of a user’s hand on each beauty target part”, Kosecoff teaches acquiring a smartphone video of a makeup/beauty routine and detecting hand motion during application to facial/body target regions (Kosecoff teaches at (para. [0019]) “a user of a smartphone creates a video of their makeup routine that produces a particular aesthetic effect.” This teaches acquiring a user video including a beauty routine. Kosecoff further teaches at (para. [0019]) “the smartphone captures images and a surface mapping of the user’s face.” This maps to acquiring visual data of the beauty target part. Kosecoff further teaches at (para. [0019]) “recognizes new applications by detecting motion of the user’s hand, identifying a new tool or formulation, and/or detecting a color shift on the user’s face.” This teaches that the video includes the user’s hand motion during cosmetic application. Kosecoff further teaches at (para. [0073]) “the biological surface 180 can be or include a face, a hand, a portion of a face, or other surface to which a cosmetic formulation is applied,” and at (para. [0042]) “the operations can similarly be applied to other surfaces. For example, the cosmetic design 110 can describe applications 215 of cosmetic formulations 220 to additional/alternative surfaces including, but not limited to, lips, nose, cheeks, forehead, or hands.” These disclosures teach beauty target parts such as facial parts and hands.); “identify a motion difference between an exemplary motion and the beauty motion by comparing the exemplary motion with the beauty motion”, Kosecoff teaches stored cosmetic design traces and dynamic tutorial motions corresponding to an exemplary cosmetic routine (Kosecoff teaches at (para. [0021]) “the cosmetic design 110 is a numerical representation of a cosmetic routine including a set of textures, mapping data, surface information, and metainformation that is stored in memory.” This teaches an exemplary/stored cosmetic routine. Kosecoff further teaches at (para. [0021]) “the cosmetic design 110 includes one or more traces 115 that are referenced to contour maps of relevant facial features 120 and/or to baseline features 125 of the target region.” This teaches exemplary motion/application traces referenced to target parts. Kosecoff further teaches at (para. [0055]) “the cosmetic design 110 can be deployed to a user device 190 as an animated augmented reality filter reproducing, step-by-step, the layer-wise application of the cosmetic design 110.” This teaches using the stored routine as an exemplary motion/tutorial.); “the motion difference including a position difference which is a motion difference related to a position of the beauty motion”, Kosecoff teaches position-based cosmetic application traces and applicator-position tracking relative to the body surface (Kosecoff teaches at (para. [0002]) “the numerical representation describing position information relative to the baseline description.” This teaches position information for the cosmetic application. Kosecoff further teaches at (para. [0006]) “identifying the application of the cosmetic formulation includes detecting an applicator of the cosmetic formulation, estimating a position of the applicator of the cosmetic formulation relative to the biological surface, tracking a motion of the applicator relative to the biological surface, and generating a numerical representation of the motion relative to the baseline description.” This teaches tracking applicator position and motion relative to the beauty target surface.); “and a velocity difference which is a motion difference related to a velocity of the beauty motion”, Kosecoff teaches motion tracking of the cosmetic application but does not expressly disclose identifying a velocity difference between an exemplary motion and a user beauty motion (Kosecoff teaches at (para. [0003]) “generating the trace includes tracking a motion of the application relative to the biological surface and generating a numerical representation of the motion relative to the baseline description.” This teaches motion tracking but does not expressly recite velocity-difference comparison.); “and generate navigation information corresponding to the motion difference for each beauty target part”, Kosecoff teaches tutorial/navigation guidance for cosmetic routines and visual/audio prompts on the user device (Kosecoff teaches at (para. [0019]) “The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone.” This teaches navigation/tutorial information corresponding to a cosmetic routine and target face shape. Kosecoff further teaches at (para. [0031]) “the cosmetic design 110 can be presented dynamically, in the form of a tutorial visualization, whereby both layers and traces are animated to demonstrate layers, motions, colors, and other cosmetic aspects of the cosmetic design 110.” This teaches visual navigation information for the beauty routine. Kosecoff further teaches at (para. [0086]) “the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.) to guide a user.” This teaches navigation information in the form of guides, targets, progress indicators, and feedback.).
However, Kosecoff does not expressly disclose identifying a motion difference by comparing a live user beauty motion with an exemplary motion, nor does Kosecoff expressly disclose that the motion difference includes a velocity difference. Amini supplies comparison of measured user motion to an exemplary/reference motion and feedback based on deviations (Amini teaches at (para. [0010]) “a computer program capable executing on the computer to analyze the sensor data under ongoing dynamic training conditions, wherein a user may select from a plurality of positions and motion types, and to produce an feedback signal to indicate deviations of the measured motion from a reference motion, where the reference motion profile is automatically selected by the computer program based on data from the measured motion.” This teaches comparing a measured user motion to a reference/exemplary motion and producing feedback based on deviations. Amini further teaches at (para. [0010]) “reference profiles may be captured, stored, and used during Measured Play mode to detect deviations from the correct movements and orientations and provide feedback to improve one or a plurality of movements and orientations.” This reinforces identifying motion differences and generating corrective feedback.). Chang supplies or reinforces the velocity-difference aspect and position/displacement-based coaching (Chang teaches at (para. [0029]) “The biomechanical quality of an exercise can be characterized by the smoothness of the displacement path or the wobbliness of the path, the velocity, velocity consistency and length of the path, orientation angle of device . . . frequency or intensity as the device shakes while performing the activity.” This expressly teaches evaluating user motion using displacement/path and velocity metrics. Chang further teaches at (para. [0030]) “The biomechanical quality of the motion can be quantified algorithmically using a logic-and-heuristics approach with error correction that calculates the vertical, forward and lateral displacements and velocities in physical space from the accelerations and angular velocities.” This expressly teaches calculating displacements and velocities in physical space, thereby supplying the claimed position-related and velocity-related motion differences. Chang further teaches at (para. [0061]) “A feedback interface 130 preferably enables activation of one or more feedback outlets such as a display, an audio system, haptic feedback, and the like.” This teaches generating user feedback corresponding to the analyzed motion metrics.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Amani and Chang into the teachings of Kosecoff to combine Kosecoff’s cosmetic video/AR tutorial system, stored cosmetic motion traces, applicator-position tracking, and face-part-specific guidance with Amini’s measured-motion-versus-reference-motion deviation feedback and Chang’s displacement/velocity-based motion-quality analysis because all three references address computer-assisted guidance for improving a user’s performance of a manual physical routine, and Kosecoff expressly identifies that cosmetic application by hand can be difficult and that tutorial guidance helps reproduce cosmetic routines. The benefit would be a cosmetic navigation apparatus that not only displays an exemplary beauty routine, but also compares the user’s actual hand motion against the exemplary motion and generates corrective navigation information for each beauty target part based on position and velocity deviations, thereby improving accuracy and repeatability of the user’s beauty application.
As per claim 2, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. Kosecoff also discloses “wherein the processor presents the navigation information to the user,” (Kosecoff teaches at (para. [0019]) “The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone.” This teaches that the cosmetic system presents guidance/navigation information to the user/viewer for performing the cosmetic routine. Kosecoff further teaches at (para. [0031]) “the cosmetic design 110 can be presented dynamically, in the form of a tutorial visualization, whereby both layers and traces are animated to demonstrate layers, motions, colors, and other cosmetic aspects of the cosmetic design 110.” This expressly teaches presenting navigation information visually to the user as an animated tutorial. Kosecoff further teaches at (para. [0082]) “the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.) to guide a user.” This directly maps to the processor presenting the navigation information to the user because the system outputs visual, audio, and/or haptic guidance through the user interface.)
As per claim 3, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 2 as discussed above. Kosecoff also discloses “wherein the processor generates a navigation image as the navigation information and displays the navigation image superimposed on the user video,” Kosecoff teaches generating visual cosmetic guidance in the form of an augmented-reality tutorial image/filter and presenting that guidance on the user’s captured facial video/image data (Kosecoff teaches at (para. [0021]) “the cosmetic design 110 can be deployed to a user device 190 as an animated augmented reality filter reproducing, step-by-step, the layer-wise application of the cosmetic design 110.” This teaches generating a navigation image, i.e., an animated AR filter, as the navigation information. The AR filter is a visual guidance image used to instruct the user through the beauty routine. Kosecoff further teaches at (para. [0031]) “the cosmetic design 110 can be presented dynamically, in the form of a tutorial visualization, whereby both layers and traces are animated to demonstrate layers, motions, colors, and other cosmetic aspects of the cosmetic design 110.” This further teaches that the navigation information is image-based and visually displayed to the user as a tutorial visualization. Kosecoff further teaches at (para. [0019]) “the smartphone captures images and a surface mapping of the user’s face,” and at (para. [0019]) “The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone.” These disclosures teach that the generated navigation image/filter is mapped onto the user’s captured facial imagery/video, which corresponds to displaying the navigation image superimposed on the user video. Kosecoff further teaches at (para. [0082]) “the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.) to guide a user.” This reinforces that the processor presents visual navigation guidance to the user, and in the context of Kosecoff’s AR filter and mapped cosmetic design, that visual guidance is overlaid on the user video.)
As per claim 6, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. Amini discloses “wherein the processor converts predetermined sound information depending on the motion difference to generate a navigation sound as the navigation information,” (Amini teaches at (para. [0033]) “a computer capable of receiving said sensor data and user input on training system configuration, executing a computer program to process said sensor data, and transmitting a indicator signal to the user as feedback; and a computer program capable executing on the computer to analyze the sensor data under ongoing dynamic training conditions, wherein a user may select from a plurality of positions and motion types, and to produce a feedback signal to indicate deviations of the measured motion from a reference motion, where the reference motion profile is automatically selected by the computer program based on data from the measured motion. ” This teaches that the processor generates feedback depending on the motion difference between measured motion and reference motion. Amini further teaches at (para. [0056]) “The feedback is displayed in the Measured Play status menu (811) and may also be communicated to the user via an audio signal. ” This expressly teaches generating the feedback as an audio signal, which corresponds to the claimed navigation sound. Amini further teaches at (para. [0034]) “The computer (109) includes a processor (112), which executes the program detailed in FIG. 4 to process the IMU readings, store reference profiles in the computer storage (113), and generate feedback, which can be shown on the visual display (110) and communicated by the audio speaker (111).” This teaches processor-generated feedback communicated by an audio speaker. Accordingly, Amini teaches converting predetermined feedback/sound information into a navigation sound depending on the detected motion difference, because the system stores reference profiles, detects deviations from those profiles, and outputs corresponding feedback as an audio signal to guide correction of the user’s motion.) The rationale from claim 1 is incorporated herein.
As per claim 7, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. Chang discloses “wherein the motion difference includes an acceleration difference related to an acceleration of the user’s hand,” (Chang teaches at (para. [0030]) “The biomechanical quality of the motion can be quantified algorithmically using a logic-and-heuristics approach with error correction that calculates the vertical, forward and lateral displacements and velocities in physical space from the accelerations and angular velocities.” This expressly teaches using accelerations to quantify the user’s motion. Because Chang calculates motion characteristics from accelerations, it teaches or at least renders obvious determining a motion difference related to acceleration when comparing a measured user motion to a desired or reference motion. Chang further teaches at (para. [0029]) “The biomechanical quality of an exercise can be characterized by the smoothness of the displacement path or the wobbliness of the path, the velocity, velocity consistency and length of the path, orientation angle of device . . . frequency or intensity as the device shakes while performing the activity.” This teaches evaluating the quality of the user’s motion based on dynamic movement characteristics, including motion instability and intensity, which are related to acceleration changes during the user’s hand/body movement. Chang further teaches at (para. [0061]) “A feedback interface 130 preferably enables activation of one or more feedback outlets such as a display, an audio system, haptic feedback, and the like.” This teaches outputting feedback corresponding to the analyzed motion. Accordingly, Chang supplies the claimed acceleration-related motion-difference feature because it analyzes measured user motion using acceleration data and provides feedback based on the resulting motion-quality determination.) The rationale of claim 1 is incorporated herein.
As per claim 9, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. Chang also discloses “wherein the motion difference includes a tempo difference related to a tempo of the user’s hand movement,” Chang teaches analyzing user motion based on timing/rhythm-related movement characteristics, including frequency and intensity of the motion (Chang teaches at (para. [0029]) “The biomechanical quality of an exercise can be characterized by the smoothness of the displacement path or the wobbliness of the path, the velocity, velocity consistency and length of the path, orientation angle of device . . . frequency or intensity as the device shakes while performing the activity.” This teaches evaluating motion based on frequency, which corresponds to tempo of the user’s hand movement. Chang further teaches at (para. [0030]) “The biomechanical quality of the motion can be quantified algorithmically using a logic-and-heuristics approach with error correction that calculates the vertical, forward and lateral displacements and velocities in physical space from the accelerations and angular velocities.” This teaches quantifying motion characteristics from sensed movement data, which supports determining a tempo-related difference when comparing the user’s movement to a reference or desired motion.) The rationale of claim 1 is incorporated herein.
Regarding claim 16, claim 16 recites similar claim language to claim 1. However, claim 16 recites “An information processing method using a computer, comprising steps executed by a computer of;” Kosecoff also discloses this at para.[0075] “In the example shown in FIG. 7 , the client computing device 130 of FIG. 1 includes a computer system 710, multiple components 720 for interacting with the biological surface 180, a computer-readable medium 730, and a client application 740, that can be stored as computer-executable instructions on the computer-readable medium 730, and, when executed by the computer system 710, can implement the operations described in reference to the system 100 of FIG. 1 , and the operations of the example techniques of FIGS. 2-3 .” The rest of claim 16 is rejected under the same rationale as claim 1.
Regarding claim 17, claim 17 recites similar claim language to claim 1. However, claim 17 recites “A non-transitory computer-readable medium storing instructions to operate a computer as a module configured to;” Kosecoff also discloses this at para.[0075] “In the example shown in FIG. 7 , the client computing device 130 of FIG. 1 includes a computer system 710, multiple components 720 for interacting with the biological surface 180, a computer-readable medium 730, and a client application 740, that can be stored as computer-executable instructions on the computer-readable medium 730, and, when executed by the computer system 710, can implement the operations described in reference to the system 100 of FIG. 1 , and the operations of the example techniques of FIGS. 2-3 .” The rest of claim 17 is rejected under the same rationale as claim 1.
Claim 18,20 similar in scope to claim 2, thus rejected under the same rationale.
Claim 19, similar in scope to claim 3, thus rejected under the same rationale.
Claim 5, is rejected under 35 U.S.C 103 as being unpatentable over Kosecoff as modified by Amini and Chang as applied to claim 1 above, and further in view of Coleman(US-20160314623-A1)
As per claim 5, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. However, the combination does not disclose “wherein the processor generates an image of a hand that changes depending on a position of the beauty motion as the navigation image.”
Coleman does disclose this limitation (Coleman at para.[0081] “Also visible in FIG. 13, the augmented reality overlay device 26 senses the position of the wearer's hands 74 and provides projected highlights 76 onto the wearer's hands 74 and possibly also onto the arms. Here, the wearer's hands 74 and arms are provided with a highlight 76 outlining each of the wearer's hands 74 in a highlight color. The highlight color 76 in the example is bright green, which distinguishes the hands highlight 76 from the blade highlight 54 in red. The hands highlight 76 in this example are provided as an outline only instead of a solid highlight over the hands. The outline changes in shape and position as the wearer moves his or her hands 74 in the view 50 of the augmented reality overlay device 26. Other colors and/or indicators may be provided as well.” This teaches the outline changing shape and position as the wearer moves their hand, such as when doing a beauty motion according to the claim language.) It would have been obvious to combine Colemans AR overlay with the combination of Kosecoff, Amini, Chang’s beauty-motion navigation system because all references address sensor-based guidance for improving a user’s manual personal-care/body-motion routine. This combination would allow for a more complete beauty-navigation apparatus that has more information on hand position and shape.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff as modified by Amini and Chang as applied to claim 1 above, and further in view of Yeates(US-20220167735-A1).
As per claim 8, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. However the combination does not disclose “wherein the motion difference includes a pressure difference related to a user pressure being a pressure applied to a face of the user.”(Yeates teaches at (para. [0040]) “therapy can be targeted to local areas of a user's face or body where treatment would be most beneficial.” This teaches that the detected force/pressure can be associated with treatment of a user’s face. Yeates further teaches at (para. [0040]) “a sound, a visual alert, or a vibration or haptic feedback can be communicated to the user to adjust the applied force.” This teaches determining whether the applied force differs from a desired force and generating feedback to correct that pressure difference. Yeates further teaches at (para. [0041]) “the force transducer 198 is configured to detect the force applied to the device 100 by the user and relay the force information to the CPU to determine a dampening of the oscillation of the brushhead 120 due to the applied force.” This teaches detecting the user-applied force/pressure and relaying that pressure information to the processor. Yeates further teaches at (para. [0042]) “the processor may not detect that the user is applying above average force to the device 100,” and further teaches “the user can apply a below average force and the predetermined reduction in oscillation to the brushhead 120 can be less than the standard amount based on the below average applied force by the user that is detected by the force transducer 198.” These disclosures teach comparing user-applied force to an expected or average force, which corresponds to a pressure difference. Yeates further teaches at (para. [0044]) “An example of the score 534 can be based on multiplying the oscillation speed, pressure, and duration with each other,” and “The regimen can have a protocol name, a type of brushhead, a duration, an applied force and a series of steps including a particular skin region to apply the protocol according to an example.” This expressly teaches that the skincare regimen may define an applied force/pressure for a particular skin region. Accordingly, Yeates teaches or at least renders obvious the claimed pressure difference because it detects a user-applied force/pressure during facial skincare treatment, associates applied force with particular skin regions, and provides feedback to adjust the applied force when the user-applied pressure differs from the desired regimen force.) It would have been obvious to combine Yeates’s pressure/force-sensing skincare feedback with the combination of Kosecoff, Amini, Chang’s beauty-motion navigation system because all references address sensor-based guidance for improving a user’s manual personal-care/body-motion routine. This combination would allow for a more complete beauty-navigation apparatus that guides not only position and velocity, but also user-applied facial pressure, improving consistency, comfort, safety, and treatment effectiveness.
Claims 10, 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff as modified by Amini and Chang as applied to claim 1 above, and further in view of Ye(US-20140016823-A1).
As per claim 10, the combination of Kosecoff, Amini and Chang disclose all the elements of claim 1 as discussed above. However the combination does not disclose “wherein the processor displays an avatar image of the user superimposed on an image of the user’s face,” Ye does disclose this limitation teaching displaying virtual makeup/makeover effects superimposed on the user’s captured facial image and further teaches applying the corresponding virtual makeup effects to a 3D avatar face image of the user (Ye teaches at (para. [0055]) “A plurality of live facial images of a user is captured from a camera in real-time. Step S55 : Facial tracking of the live facial images of the user are performed in real-time to find a plurality of tracking points on the captured live facial images.” This teaches capturing the user’s face image and tracking facial points. Ye further teaches at (para. [0055]) “Virtual makeover effects are produced in real-time (superimposed or overlapped) on the live facial images according to the tracking points on the captured live facial images in real-time.” This teaches superimposing visual makeup/avatar-related effects on the image of the user’s face. Ye further teaches at (para. [0071]) “virtual makeup can also be applied on a face image of a 3D avatar (aka avatar mode) or a 3D avatar with a face mask (aka mask mode) (in real-time) corresponding to the makeover effects as shown on the raw image of the user during facial tracking in real time.” This teaches an avatar face image corresponding to the user’s raw facial image and the same virtual makeover effects. Ye further teaches at (para. [0071]) “the user can switch back and forth between the two display modes, i.e. one display mode is a 3D avatar face image mode, and another display mode is the raw image mode of the user containing virtual makeover effects.” This teaches displaying an avatar image mode corresponding to the user’s face image.); “and changes a pixel of the avatar image at a position to which the beauty motion is applied,” Ye teaches changing color/visual effect in makeup-allowed regions based on the position of a finger, cursor, or tracked object used as a virtual makeup input tool (Ye teaches at (para. [0062]) “For example, referring to FIG. 18 in yet another embodiment, in Step S800, a user's finger acting as the virtual makeup input tool can slide over the screen of the touch panel. Then in Step S805, the application program checks to see which segments or layers the user's finger has slid over in the makeup-allowed area. Then in Step S810, virtual makeup is applied in real-time. ” This teaches applying virtual makeup at the position to which the user’s finger/beauty motion is applied. Ye further teaches at (para. [0063]) “In addition, the object tracking can be performed to track a hand as the virtual makeup tool. Furthermore, as shown in FIG. 22 b, in alternative embodiment, a finger can also be used as a virtual makeup tool, with the tip of the finger of the user functioning as the pointing input” This teaches using hand/finger motion as the beauty-motion input. Ye further teaches at (para. [0064]) “Paint or draw a selected cosmetic item to apply a cosmetic color effect on the selected segment in the makeup-allowed area according to the makeup effect mode.” This teaches changing the color/visual pixels of the makeup image at the selected position. Ye further teaches at (para. [0071]) “Virtual makeup customization effect can be further produced by making changes to the color within one or more makeup-allowed areas to provide for color overlapping as shown in FIG. 30.” This teaches changing the color of image pixels within the makeup-allowed area. Ye further teaches at (para. [0071]) “the virtual makeup or makeover effects can be directly applied on the 3D avatar in real time without time lag.” This ties the color/visual makeup change to the avatar image. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Ye into the combination of teachings of Kosecoff, Amini and Chang and in order to a allow for a more intuitive interface.
As per claim 12, the combination of Kosecoff, Chang, Amini and Ye disclose all the elements of claim 10 as discussed above. Ye also discloses “wherein the processor applies makeup to the avatar image by changing a color of a pixel of the avatar image at the position to which the beauty motion is applied” (Ye at para. [0049] “FIG. 31 shows virtual makeup applied on a face image of a 3D avatar (aka avatar mode) or a 3D avatar with a face mask (aka mask mode) (in real-time) corresponding to the makeover effects as shown on the raw image of the user during facial tracking in real time.” Here Ye teaches the application of makeup unto a 3D avatar. The figure also shows off the color change applied to the 3D avatar compared to the original. Ye further states at para. [0063] “ In addition, the object tracking can be performed to track a hand as the virtual makeup tool.” This means that beauty motions can be tracked as well to determine where makeup should be applied.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Ye into the combination of teachings of Kosecoff, Amini and Chang and in order to a allow for a more complete representation of the virtual avatar after makeup application.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRIS ALEJANDRO PUNTIER whose telephone number is (703)756-1893. The examiner can normally be reached Mc-F 7:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRIS ALEJANDRO PUNTIER/ Examiner, Art Unit 2616
/DAVID H CHU/ Primary Examiner, Art Unit 2616
Prosecution Insights