DETAILED ACTION
This is the first Office Action on the merits and is responsive to the papers filed on 10/02/2024.
Claims 1-10 are currently pending and are examined below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim(s) 1,6,7,8 are objected to because of the following informalities:
Claim 1 recites “configured to be worn by a user,” then “associated with eyes of a user,” then “a face of a user,” and thereafter “the user”, the second and third instances should refer to the first (associated with eyes of [the] user; a face of [the] user).
Claim 6 recites “applying one or more cosmetic products” which appears to be missing “the”, as the products were introduced in claim 1 (applying [the] one or more cosmetic products).
Claim 7 recites “applying one or more cosmetic products” which appears to be missing “the” (applying [the] one or more cosmetic products).
Claim 8 recites “wherein providing guidance associated with” which appears to be missing “the” (wherein providing [the] guidance associated with).
Appropriate correction is required.
Specification
The disclosure is objected to because of the following informalities:
Para 0034 recites “may integrated into” which appears to be missing “be” (may [be] integrated into).
Para 0038 recites “the memories 222”, the memories are designated 122 elsewhere and “222” should be corrected (the memories [122]).
Para 0064 recites “the user interface 230”, the user interface is designated 130 elsewhere and “230” should be corrected (the user interface [130]).
Para 0048 recites “AR interrace 112”, should be “interface” (AR [interface] 112).
Para 0093 recites “and automatically guidance, based on” which appears to be missing a verb (and automatically [adjusting] guidance, based on).
Para 0096 recites “away one or both eyes of the user” which appears to be missing “from” (away [from] one or both eyes of the user).
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1 – “Statutory Category Identification”
Claim 1 is directed to “an augmented reality (AR) eyewear device” (i.e. a machine), claim 19 is directed to “computer-implemented method” (i.e. a process), and claim 20 is directed to “a non-transitory computer-readable medium” (i.e. a machine), hence the claims are directed to one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter). In other words, Step 1 of the subject-matter eligibility analysis is “Yes.”
Step 2A, Prong 1 “Abstract Idea Identification”
However, the claims are drawn to the abstract idea of “guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user,” in the form of “certain methods of organizing human activity,” in terms of managing personal behavior or relationships or interactions between people (including social activities, teaching and following rules or instructions), or reasonably in the form of “mental processes,” in terms of processes that can be performed in the human mind (including an observation, evaluation, judgement or opinion). Regardless, the claims are reasonably understood as either “certain methods of organizing human activity” or “mental processes,” which require the following limitations:
Per claim 1:
“receive an indication of a makeup look selected by the user;
analyze the real-time data associated with the face of the user in order to generate a three-dimensional map associated with the face of the user; identify one or more facial features of the face of the user on the three-dimensional map associated with the face of the user; and
provide…guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user.”
Per claim 19:
“receiving…an indication of a makeup look selected by a user of the AR eyewear device;
analyzing…real-time data associated with a face of the user captured …in order to generate a three-dimensional map associated with the face of the user;
identifying…one or more facial features of the face of the user on the three-dimensional map associated with the face of the user; and
causing… to superimpose, upon a field of view (FOV)… associated with one or more eyes of the user, guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user.”
Per claim 20:
“receiving an indication of a makeup look selected by a user…;
analyzing real-time data associated with a face of the user captured …in order to generate a three-dimensional map associated with the face of the user;
identifying one or more facial features of the face of the user on the three-dimensional map associated with the face of the user; and
causing …to superimpose, upon a field of view (FOV) …associated with one or more eyes of the user, guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user.”
These limitations simply describe a process of data gathering and manipulation, which is partially analogous to “collecting information, analyzing it, and displaying certain results of the collection analysis” (i.e. Electric Power Group, LLC, v. Alstom, 830 F.3d 1350, 119 U.S.P.Q.2d 1739 (Fed. Cir. 2016)). Hence, these limitations are akin to an abstract idea which has been identified among non-limiting examples to be an abstract idea. In other words, Step 2A, Prong 1 of the subject-matter eligibility analysis is “Yes.”
Step 2A, Prong 2 – “Practical Application”
Furthermore, the claims do not include additional elements that either alone or in combination are sufficient to claim a practical application because to the extent that, e.g.,
“an augmented reality (AR) eyewear device,” “one or more processors,” “one or more sensors,” “one or more actuators,” and “one or more AR interfaces,” are claimed, as these are merely claimed to generally link the use of a judicial exception to a particular technological environment or field of use. In other words, the claimed “guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user,” is not providing a practical application. To the extent the applicant asserts that the recited eyewear constitutes more than a field of use, the specification describes the AR eyewear device 102 as including “smart glasses, smart sunglasses, smart goggles, etc.” (Para 0024 and the AR interface 112 as providing “interactive displays” in Para 0028, without describing any improvement to the eyewear itself, the claimed improvement is to the information presented to the user, not to the device presenting it. Thus, Step 2A, Prong 2 of the subject-matter eligibility analysis is “No.”
Step 2B – “Significantly More”
Likewise, the claims do not include additional elements that either alone or in combination are sufficient to amount to significantly more than the judicial exception because to the extent that, e.g. “an augmented reality (AR) eyewear device,” “one or more processors,” “one or more sensors,” “one or more actuators,” and “one or more AR interfaces,” are claimed, these are generic, well-known, and conventional elements. As evidence that these are generic, well-known, and conventional elements (or an equivalent term), as a commercially available product, or in a manner that indicates that the additional elements are sufficiently well-known, the Applicant’s specification discloses these in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. § 112(a), per MPEP § 2106.07(a) III (a). As such, this satisfies the Examiner’s evidentiary burden requirement per the Berkheimer memo.
Moreover, the element of “an augmented reality (AR) eyewear device” is described in para. [0024] as follows:
“[0024] The exemplary computer system 100 may include an AR eyewear device 102 (which may include, e.g., smart glasses, smart sunglasses, smart goggles, etc.) as well as, in some cases, one or more user computing devices 104 (which may include, e.g., smart phones, smart watches or fitness tracker devices, tablets, laptops, virtual reality headsets, wearables, etc.), and/or one or more server(s) 106.”
Due to the list of examples, this element is reasonably interpreted as a generic, well-known, and as a commercially available product which also provides no details of anything beyond ubiquitous standard off-the-shelf equipment.
Likewise, the element of “one or more processors,” is described in para. [0026] as follows:
“[0026] The AR eyewear device 102 may include one or more integrated sensors 110, an integrated AR interface 112, one or more integrated components 114, and/or an integrated light source 116. Additionally, the AR eyewear device 102 may include a controller 118, including one or more processor(s) 120, as well as one or more computer memories 122.”
This element is reasonably interpreted as part of a generic computer having generic computer components which provides no details of anything beyond ubiquitous standard off-the-shelf equipment.
Further, the element of “one or more sensors,” is described in para. [0027] as follows:
“[0027] Generally speaking, the sensors 110 may be operable to capture real-time data associated with the face of a user and/or the environment of the user before, during, and/or after a user applies a cosmetic product using the AR eyewear device 102. The sensors 110 may be oriented towards the user's face or away from the user's face in various examples. In some examples, the sensors 110 may include, for instance, a camera and/or a depth sensor operable to capture data associated with the user's face (e.g., directly and/or as reflected in a mirror), data associated with various cosmetic products to be applied to the user's face and/or their packaging, distances from various features of the user's face to various cosmetic products and/or applicators for the various cosmetic products, etc.”
This element is also reasonably interpreted as part of a generic computer as a generic computer component which provides no details of anything beyond ubiquitous standard off-the-shelf equipment.
Finally, the element of “one or more AR interfaces,” is described in para. [0028] as follows:
“[0028] The AR interface 112 may include, for instance, a user interface component operable to receive inputs and selections from the user of the AR eyewear device 102, and/or to provide audible or visual feedback to the user of the AR eyewear device 102. For instance, the AR interface 112 may provide interactive displays via which users may select a desired makeup look to be applied to the user's face.”
This element is also reasonably interpreted as part of a generic computer as a generic computer component which provides no details of anything beyond ubiquitous standard off-the-shelf equipment. Therefore, the Applicant’s own specification discloses ubiquitous standard equipment that is (1) generic, routine, conventional, and/or commercially available; and (2) does not provide anything significantly more. Thus, Step 2B, of the subject-matter eligibility analysis is “No.”
In addition, dependent claims 2-18 do not provide a practical application and are insufficient to amount to significantly more than the judicial exception. The additional elements recited in the dependent claims “a camera or a depth sensor” of claim 2, “an audio component” of claim 8, “communication interface” of claims 9-11, “light source” of claim 14-15, and “one or more actuators” of claim 5 are each recited solely by the functions they perform, without any particular structure, and serve only to place the user in a better position to perceive or act upon the information from the abstract idea rather than to improve the functioning of the device or any other technology (See MPEP § 2106.05(a)). As such, dependent claims 2-18 are also rejected under 35 U.S.C. § 101, based on their respective dependencies to claim 1. Therefore, claims 1-20 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites “The AR eyewear device of claim 1, wherein providing the guidance adjusting contrast of one or more portions of the face of the user, within the FOV of the one or more transparent components.” The wherein clause contains no main verb and does not state a complete limitation. It cannot be determined by one of ordinary skill in the art whether the claim is intended to recite that providing the guidance includes adjusting contract (aligning with claim 3), or that the guidance is adjusting contract, or that contract is adjusting while the guidance is provided, or some other relationship between the guidance and the contrast adjustment. Thus, the metes and bounds of the claim cannot be determined by someone of ordinary skill in the art, the claim is indefinite. The specification does not resolve said ambiguity: Specification Para 0086 describes “enhancing a level of contract” as one thing that “providing the AR guidance may include,” which is narrower than the “adjusting contrast” language of the claim.
For purposes of compact prosecution, claim 4 will be interpreted with structure aligning with claim 3 as “wherein providing the guidance includes adjusting contrast of one or more portions of the face [..]”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 7-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Troutman (US 20220202168) in view of Evangelista (US 11069094) in further view of Robaina (US 20200183171).
Regarding claim 1, Troutman discloses [a]n augmented reality (AR) eyewear, comprising: [(See Troutman Para 0043: The software application or mobile application 111 may be implemented as an augmented reality system that includes a makeup objective unit operably coupled to a graphical user interface, a makeup palette unit coupled to the makeup objective unit, and a makeup objective visualization unit) (Para 0011: as augmented reality system for makeup)]. The broadest reasonable interpretation of an AR device is a computer-implemented system that overlays digitally generated content on a view of the user’s face as seen above, the eyewear factor is addressed below. Troutman further discloses one or more sensors configured to capture real-time data associated with a face of a user; [(Para 44: the functions and processes of the mobile device 101 may be implemented by one or more respective processing/computation circuits 226. The same or similar processing/computation circuits 226 may be included in a tablet computer or a laptop computer. A desktop computer may be similarly configured, but in some cases, may not include a built-in touch screen 221, microphone 241 or camera 231) (Para 0002: Smartphones with front facing cameras offer a capability of taking pictures and videos of the person) (Para 0004: These try-on applications work by taking a self portrait photo with the smartphone camera)] one or more processors; and one or more non-transitory computer-readable memories storing instructions that, when executed by the one or more processors, cause the one or more processors to [(Para 0045: the processing/computation circuit 226 includes a Mobile Processing Unit (MPU) 200 which performs the processes described herein. The process data and instructions may be stored in memory 202.) (Para 0046: the instructions may be stored in FLASH memory, Secure Digital Random Access Memory (SDRAM), Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), solid-state hard disk)] receive an indication of a makeup look selected by the user [(Para 0011: a makeup objective unit including computation circuitry operably coupled to a graphical user interface configured to generate one or more instances of user selectable makeup objectives and to receive user-selected makeup objective information) (Para 0059: The App 111, via the makeup objective unit, may ask the user what type of look they wish to create. To assist the user in answering this question, the App 111 may generate a list of predefined makeup looks, and the user may select a predefined makeup look) (Para 0065: in S501, the user inputs, via the makeup objective unit, a desired look)] and provide, via the one or more AR interfaces, guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user. [(Para 0088: In S315, the mobile application 111 may display a message asking the user if they want a recommendation on how to apply the virtual makeup) (Para 0089: The user interface 1200 may display a button 1203 for selecting a recommendation on how to apply the virtual makeup 1205) (Para 0091: when the user selects to obtain a recommendation (YES in S315), in S319, the mobile application 111 indicates a location on the face image where the virtual makeup is to be applied) (Para 0092: The recommender system 1400 includes a recommendation engine 1407 that retrieves and ranks recommendations. In the case of application of a particular virtual makeup, a recommendation may be for the look that the user has input in step S301) (Para 0101: The border of the original feature may have been determined during the face analysis step, S309. A sequence of video frames may be generated as an animation to demonstrate how to apply virtual makeup to a particular face part) (Para 0102: the user may mimic the demonstrated application of the virtual makeup to apply the makeup)].
However, Troutman is silent when it comes to analyze the real-time data associated with the face of the user in order to generate a three-dimensional map associated with the face of the user; identify one or more facial features of the face of the user on the three-dimensional map associated with the face of the user. Evangelista is analogous art because it is directed to rendering makeup products on a user’s face within an augmented reality environment in real-time (See Evangelista Abstract). This is analogous art because it is within the same field of endeavor as the instant application.
Evangelista teaches analyze the real-time data associated with the face of the user in order to generate a three-dimensional map associated with the face of the user [(See Evangelista Para 27: the AR makeup system 102 can track a face within a live digital video stream and generate an augmented reality layer including digitally rendered makeup that tracks with the identified face within the digital video stream. The AR makeup system 102 can apply digitally rendered makeup by first identifying a face using face-tracking software and generating a three-dimensional mesh representing the identified face.) (Para 47: the client device 300 can include a mobile phone with a digital camera 304 for capturing images or video of the user) (Para 51: In response to the user selecting a product to render in the video stream, the AR makeup system 102 can generate a three-dimensional mesh corresponding to an identified face in the video stream […] the AR makeup system 102 can employ a facial landmark tracking algorithm […] the AR makeup system 102 can identify facial landmarks […] Having identified facial landmarks, the AR makeup system 102 can fit a generic or template mesh of a face to the identified facial landmarks to generate the three-dimensional mesh 308) (Para 52: The AR makeup system 102 can then generate the three-dimensional mesh 308 by creating a plurality of polygons in a wireframe mesh that maps to the shape of the user's face. The three-dimensional mesh 308 also tracks with the user's face as the user moves in the video stream.)]. identify one or more facial features of the face of the user on the three-dimensional map associated with the face of the user. [(Para 53: the AR makeup system 102 identifies different regions of the face within the three-dimensional mesh 308 […] the AR makeup system 102 can identify facial features such as the user's eyes, mouth, nose, cheeks, etc., corresponding to the different regions where the AR makeup system 102 can apply makeup. In one or more embodiments, the AR makeup system 102 labels each separate region by labeling the regions based on groupings of polygons and/or vertices of the three-dimensional mesh 308.)].
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman’s AR system for makeup that already captures the user’s face with a camera, detects facial landmarks with a machine learning model, and indicates a location on the face image where makeup is to be applied by anaylz[ing] the real-time data associated with the face of the user in order to generate a three-dimensional map associated with the face of the user; identify[ing] one or more facial features of the face of the user on the three=dimensional map associated with the face of the user; as taught by Evangelista, because Evangelista discloses the known technique of fitting a three-dimensional face mesh to detected facial landmarks and labeling facial regions on that mesh, and applying this technique to Troutman’s existing landmark-based face analysis is a direct extension of a step Troutman already performs. Thus, it would yield the predictable result of a three-dimensional map of the user’s face with the facial features identified on it. One of ordinary skill would have been inclined to make this combination for the advantage stated by Evangelista of a mesh that maps to the shape of the user’s face and tracks with the user’s face as the user moves in the video stream (See Evangelista Para 52), thereby keeping Troutman’s location indications and application demonstrations registered to the correct facial feature in a live view rather than a still photograph. This is consistent with Troutman’s stated goal of showing the user how to apply makeup to a particular face part (See Troutman Para 0101).
However, Troutman in view of Evangelista is silent when it comes to an eyewear frame configured to be worn by a user, the eyewear frame having one or more transparent components associated with eyes of a user; one or more AR interfaces configured to superimpose one or more of text or images upon a field of view (FOV) of the one or more transparent components [and providing the guidance] via the one or more AR interfaces. Troutman’s application runs on a mobile device, tablet, laptop, or desktop (See Para 0059). Robaina is analogous art because it is directed to a head-mounted display configured to project light to an eye of a user to display augmented reality image content and to alter user perception of a real or virtual object within the vision field of the user (See Robaina abstract). This is analogous art because it is reasonably pertinent to the problem of presenting augmented reality guidance in the user’s field of view while the user’s hands are occupied with a task. Thus is the same problem faced by the inventor of the instant application.
Robaina teaches an eyewear frame configured to be worn by a user, the eyewear frame having one or more transparent components associated with eyes of a user [(See Robaina Para 1464: The display 62 may be coupled to a frame 64, which is wearable by a display system user or viewer 60 and which is configured to position the display 62 in front of the eyes of the user 60. The display 62 may be considered eyewear) (Para 1451: A person wearing a head-mounted augmented reality display system or device such as described herein may be provided with a view of the external world or environment through transparent optics, e.g., windows, lens, waveguides, etc., that permit light from objects in the external environment to pass through the transparent optics to the eyes of the user) (Para 1450: this content is displayed on a head-mounted display, e.g., as part of eyewear, that projects image information to the user's eyes. In addition, the display may also transmit light from the surrounding environment to the user's eyes)] one or more AR interfaces configured to superimpose one or more of text or images upon a field of view (FOV) of the one or more transparent components [and providing the guidance] via the one or more AR interfaces. [(Para 1451: The AR system, may additionally present images to the users wearing the head mounted display, for example, by projecting light from a modulated light source into the eye of the user. These images may be referred to as “virtual” images as these “virtual” images are generated by a display and not from light from objects in the external world that passes through the transparent elements) (Para 1461: The AR system can add virtual elements to the existing physical objects viewed through the transparent optics of the head mounted display) (Para 1544: additional or “augmented” content can also include further text, arrows, user commands, duplicate images, etc.) (The local processing and data module 70 may comprise a hardware processor, as well as digital memory, such as non-volatile memory (e.g., flash memory or hard disk drives), both of which may be utilized to assist in the processing, caching, and storage of data. The data include data a) captured from sensors (which may be, e.g., operatively coupled to the frame 64).) (Para 1514: environment sensing cameras, cameras may be located, for example, facing outward (e.g., forward-looking) so as to capture images similar to at least a portion of an ordinary field of view of a user.)].
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the AR system for makeup of Troutman in view of Evangelista, which already captures the user’s face with a camera and applies guidance on the user’s face image, on an eyewear frame configured to be worn by a user, the eyewear frame having one or more transparent components associated with eyes of a user; one or more AR interfaces configured to superimpose one or more of text or images upon a field of view (FOV) of the one or more transparent components [and providing the guidance] via the one or more AR interfaces as taught by Robaina, because Robaina discloses the known technique of presenting augmented reality guidance registered to real objects in the wearer’s field of view through the transparent optics of head worn eyewear. Troutman treats the host platform as interchangeable (See Troutman Para 0059). This is the simple substitution of one known AR display platform for another. Thus, would yield the predictable result of Troutman’s makeup application guidance being superimposed on the user’s view of his or her own face through the transparent lenses of eyewear. One of ordinary skill would have been inclined to make this combination for the advantage stated by Robaina of providing visual information to a user whose hands are occupied with a task (See Robaina Para 1449). Since applying makeup typically requires both hands and cannot be performed while holding Troutman’s mobile device, thereby providing hands-free guidance, consistent with Troutman’s stated goal of demonstrating to the user how to apply makeup to a particular face part (See Troutman Para 0101).
Regarding claim 2, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the one or more sensors include one or more of a camera or a depth sensor. (See Robaina Para 1465: image capture devices (such as cameras) [..] depth sensors, pairs of binocular world cameras).
Regarding claim 3, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein providing the guidance includes magnifying one or more portions of the face of the user within the FOV of the one or more transparent components. [(See Robaina Para 1453: the display system may be configured to identify an object in the real world and display a magnified image of the object. In various embodiments, the display system may be configured to magnify a portion of the image in comparison to other portions of the image or other objects in the field of view of the user viewing objects in the environment ahead through the transparent optical elements (windows, lenses, waveguides) of the head mounted display.) (Para 1531: the image content may be magnified in comparison to the content when viewed without using the display system) (the display system may present a magnified (or enhanced) image of the object in relation to how the object would appear without the magnification)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device with Robaina’s magnification of a portion of the field of view, since Robaina teaches magnifying a real object of interest relative to the rest of the view through the transparent optics to give the user better visibility of that object (See Robaina Para 1453, 1571). Thus, the modification would provide the predictable benefit of enlarging the small facial feature being worked on so that the user can apply the cosmetic product to it with greater precision.
Regarding claim 4, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein providing the guidance adjusting contrast of one or more portions of the face of the user, within the FOV of the one or more transparent components. (NOTE: Claim 4 is interpreted as seen above in Claim Rejections - 35 USC § 112) [(See Robaina Para 1533: the presented image content [...] may be enhanced by increasing contrast in relation to how the content would appear without increasing contrast) (Para 1534: the presented image content may include an augmented image(s) that in combination with the actual real world content provide the adjusted contrast ratio compared to how the content would appear without adjustment and/or compared to image content presented to other portions of the user's field of view. In some examples, the brightness and/or darkness of adjacent colors may be adjusted by adjusting the brightness and/or darkness of at least one color […]. In some instances, the brightness and/or darkness of a color may be adjusted by providing additional color or colors […] to an object or background such that when viewed with the actual real world content, the combination of the real and additional colors provides the adjusted contrast ratio) (Para 1578: the portion of the image of the environment corresponding to the image of the object […] may be modified in intensity/brightness, contrast, color saturation, sharpness, edge enhancement)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device with Robaina’s magnification of a portion of the field of view, since Robaina teaches that images with more with more contrast may be more easily seen and that the contrast of an object of interest may be increased relative to the rest of the environment viewed through the transparent display (See Robaina Para 1536, 1578), this modification would provide the predictable benefit of making the boundaries of the facial feature being worked on more visible to the user during application of the cosmetic product.
Regarding claim 7, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the guidance includes tracing lines or arrows associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user. [(See Robaina Para 1544: “augmented” content can also include further text, arrows, user commands, duplicate images, etc.) (Para 1584: the display device may present an enhanced image of the defibrillator (e.g., a magnified image of it) along with additional augmented images such as more precise directions (e.g., arrows, texts, etc.)) (See Troutman Para 0091: the mobile application 111 indicates a location on the face image where the virtual makeup is to be applied.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by presenting Troutman’s indication of the location where the makeup is to be applied as arrows applied on the user’s face, as taught by Robaina, since Robaina teaches arrows as known form of augmented directional content overlaid on the user’s view to direct the user to a real object (See Robaina Para 1584), this modification would provide the predictable benefit of unambiguously indicating to the user where on the facial feature the cosmetic product is to be applied.
Regarding claim 8, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the AR eyewear device includes an audio component, and wherein providing guidance associated with applying the one or more cosmetic products to the one or more facial features of the user includes providing audio guidance via the audio component. [(See Troutman Para 0130: the feedback may be provided in the form of voice interaction with the mobile application 111 as the user applies a makeup product to a face image. The voice feedback may be provided using a microphone 103a, 241 and the feedback may be provided in response to questions and statements output through an audio circuit 242) (See Robaina Para 1464: a speaker 66 is coupled to the frame 64 and positioned adjacent the ear canal of the user 60) (Para 1541: some embodiments may provide a visual (e.g., a pop-up alert or a blinking light) or an audio alert (e.g., a bell or a voice) to the user.)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by outputting Troutman’s statements to the user through Robaina’s frame-mounted speaker during application of the makeup product. Since Troutman teaches voice interaction with the user as the user applies a makeup product (See Troutman Para 0130) and Robaina provides a speaker on the eyewear frame for audio output (See Robaina Para 1464), this modification would provide the predictable benefit of delivering the application guidance audibly as well as visually to a user whose vision is directed to the task.
Regarding claim 9, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) further comprising a communication interface configured to communicate with a mobile device, external to the AR eyewear device. [(See Robaina Para 1465: the display 62 is operatively coupled by communications link 68, such as by a wired lead or wireless connectivity, to a local data processing module 70 which may be mounted in a variety of configurations, such as […] removably attached to the user 60 (e.g., in a backpack-style configuration, in a belt-coupling style configuration) […] The local processing and data module 70 may be operatively coupled by communication links 76, 78, such as via a wired or wireless communication links, to the remote processing module 72 and remote data repository 74) (See Troutman Para 0059: the user may bring up an App 111 on a mobile device, tablet, laptop, or desktop computer.)].
Accordingly, It would have been obvious to one having ordinary skill in the art before the effective filing of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device such that the eyewear communicates with Troutman’s external mobile device over Robaina’s communication link, since Robaina teaches distributing processing between the head-worn display and a separate processing module carried by the user (See Robaina Para 1465), this modification would provide the predictable benefit of reducing the weight and power consumption of the head-worn unit by performing Troutman’s face analysis and recommendation processing on the mobile device.
Regarding claim 10, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 9 (See Citations Above in Claim 9) wherein the communication interface is a wired communication interface. [(See Robaina Para 1465: communications link 68, such as by a wired lead)].
Regarding claim 11, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 9 (See Citations Above in Claim 9) wherein the communication interface is a wireless communication interface. [(See Robaina Para 1465: communications link 68, such as by […] wireless connectivity)].
Regarding claim 12, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the instructions, when executed by the one or more processors, (Robaina 1465: The local processing and data module 70 may comprise a hardware processor, as well as digital memory, such as non-volatile memory […] both of which may be utilized to assist in the processing, caching, and storage of data.) further cause the one or more processors to generate a three-dimensional preview of the makeup look selected by the user as applied to the three-dimensional map associated with the face of the user, and wherein the one or more AR interfaces are further configured to generate and display an AR version of the three-dimensional preview of the makeup look selected by the user as applied to the three-dimensional map associated with the face of the user, superimposed on the face of the user within the FOV of the one or more transparent components. [(See Evangelista Para 27: generate an augmented reality layer including digitally rendered makeup that tracks with the identified face within the digital video stream […] The AR makeup system 102 can then generate a texture representing the selected makeup product and apply the texture to a portion of the mesh corresponding to the area of the user's face) (Para 54: in response to the user selecting a lipstick makeup product, the AR makeup system 102 can generate a mask for a corresponding portion of the three-dimensional mesh 308) (Para 56: the AR makeup system 102 can generate a texture using the base makeup color 312 and then apply the texture to the makeup selection mask 310 of the three-dimensional mesh 308.) (See Troutman Para 0065: The desired look, also referred to herein as a virtual try-on)].
Therefore, It would have been obvious to one having ordinary skill in the art before the effective filing of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device with Evangelista’s rendering of the selected makeup onto the three-dimensional mesh, since Evangelista teaches that rendering the makeup on the mesh keeps the rendered makeup registered to the face as it moves (see Evangelista Para 52), this modification would provide the predictable benefit of showing the user, on his or her own face, how the selected look will appear before it is applied.
Regarding claim 13, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 12 (See Citations Above in Claim 12) wherein the three-dimensional preview of the makeup look selected by the user includes a three-dimensional preview of an application process of the makeup look selected by the user. [(See Troutman Para 0101: A sequence of video frames may be generated as an animation to demonstrate how to apply virtual makeup to a particular face part) (Para 0102: the user may mimic the demonstrated application of the virtual makeup)].
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by rendering application-process animation on the same three-dimensional mesh used for the finished-look preview, since both are face-registered content demonstrating the selected look, this modification would provide the predictable benefit of a single tracked face model for all previews.
Regarding claim 14, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) further comprising a light source configured to provide light to the face of the user. (See Robaina Para 1507: The augmented reality display system 2010 may correspond to the system 80 of FIG. 2 and may include a viewer imaging system 22. The system 22 may include cameras 24 (e.g., infrared, UV, and/or visible light cameras) paired with light sources 26 (e.g., infrared light sources) directed at and configured to monitor the user (e.g., the eyes 2001, 2002 and/or surrounding tissues of the user).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by the light source configured by Robaina for the predictable benefit of providing light to the user’s face.
Regarding claim 15, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 14 (See Citations Above in Claim 14) when executed by the one or more processors, further cause the one or more processors to control the light source to provide particular lighting conditions while the one or more cosmetic products are applied to the one or more facial features of the user. [(See Robaina Para 1477: A controller 210 controls the operation of one or more of the stacked waveguide assembly 178, including operation of the image injection devices […] the light source 2040, and the light modulator 2030. In some embodiments, the controller 210 is part of the local data processing module 70. The controller 210 includes programming (e.g., instructions in a non-transitory medium)) (See Robaina Para 1507: The augmented reality display system 2010 may correspond to the system 80 of FIG. 2 and may include a viewer imaging system 22. The system 22 may include cameras 24 (e.g., infrared, UV, and/or visible light cameras) paired with light sources 26 (e.g., infrared light sources) directed at and configured to monitor the user (e.g., the eyes 2001, 2002 and/or surrounding tissues of the user).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by applying Robaina’s lighting condition-based control of the light directed toward the user during Troutman’s makeup application session. This modification would provide the predictable benefit of the user seeing the superimposed guidance and the facial feature being worked on under consistent lighting while the cosmetic products are applied.
Regarding claim 16, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the instructions, when executed by the one or more processors, further cause the one or more processors to analyze the real-time data in order to identify one or more cosmetic products within the FOV of the one or more transparent components. [(See Robaina Para 0116: recognize one or more objects in the user's environment by applying object recognition to images received from the one or more outwardly facing cameras) (Para 1695: The processing electronics may be configured to […] recognize one or more objects in the user's environment by applying object recognition to images received from the one or more outwardly facing cameras) (Para 1637: the display device can analyze the images acquired by the outward-facing imaging system to perform […] object recognition) (Para 1653: the device may access an internal or external system that includes pictures of defibrillators. Therefore, when the device detects the defibrillator using a computer vision algorithm and/or using data received from one or more environmental sensors, the device can then signal the presence of the defibrillator) (Para 1577: a magnified image of a label on a bottle in the user's peripheral field of view) (See Troutman Para 0092: a recommendation may be for the look that the user has input in step S301 and the virtual makeup.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by applying Robaina’s object recognition to the images from the frame-mounted camera to identify the cosmetic products in the user’s field of view, since Robaina teaches recognizing a user-defined object of interest in the environment and signaling its presence so the user can locate it (See Robaina Para 1584, 1653), this modification would provide the predictable benefit of enabling a user with limited vision to locate the particular product that Troutman’s guidance calls for at the current step.
Regarding claim 17, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 16 (See Citations Above in Claim 16) wherein the guidance further includes indications of the one or more identified cosmetic products within the FOV of the one or more transparent components. [(See Robaina Para 1577: a magnified image of a label on a bottle in the user's peripheral field of view may be presented laterally displaced above the actual labeled bottle.) (Para 1583: the alert may include an icon (e.g., a copy of the image)) (Para 1584: After the outward-facing cameras locate the defibrillator (e.g., within the user's field of regard), the display device may present an enhanced image of the defibrillator (e.g., a magnified image of it) along with additional augmented images such as more precise directions)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by presenting Robaina’s AR eyewear device by presenting Robaina’s indication of the recognized object as part of the guidance, since Robaina teaches presenting an enhanced image or icon of the recognized object to draw the user’s attention to it (See Robaina Para 1583, 1584), this modification would provide the predictable benefit of showing the user which product is the best one to use.
Regarding claim 18, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 17 (See Citations Above in Claim 17) wherein the indications of the one or more identified cosmetic products are superimposed upon the respective one or more identified cosmetic products within the FOV of the one or more transparent components. [(See Robaina Para 0163: highlighting the object comprising superimposing a partially transparent color over the object.) (Para 1680: Another example is to superimpose yellow highlighting in front of an real object that can be seen through the transmissive display [..] the combination of the yellow virtual content (highlighting) and the really object, speed limit sign, may draw the user's attention to the speed limit sign.) (Para 1700: Enhancing the object in the user's vision field may include highlighting the object by superimposing a partially transparent color over the object) (Para 1461: The AR system can add virtual elements to the existing physical objects viewed through the transparent optics)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina’s AR eyewear device by presenting Robaina’s AR eyewear device by superimposing highlighting in front of a real object seen through the transmissive display draws the user’s attention to that object (See Robaina Para 1680), this modification would provide the predictable benefit of directing the user to the correct product without requiring the user to read its label.
Regarding claim 19, Troutman in view of Evangelista and Robaina teaches [a] computer-implemented method for operating an augmented reality (AR) eyewear device, the computer-implemented method comprising: receiving, by one or more processors, an indication of a makeup look selected by a user of the AR eyewear device; (See Troutman Para 0011, 0045, 0059, 0065 as cited above) analyzing, by the one or more processors, real-time data associated with a face of the user captured by one or more sensors of the AR eyewear device in order to generate a three-dimensional map associated with the face of the user; (See Troutman Para 0044, Evangelista Para 27, 47, 51, 52 as cited above) identifying, by the one or more processors, one or more facial features of the face of the user on the three-dimensional map associated with the face of the user; (See Troutman Para 0077, Evangelista Para 53, 54, 55 as cited above) and causing, by the one or more processors, one or more AR interfaces of the AR eyewear device to superimpose, upon a field of view (FOV) of one or more transparent components of the AR eyewear device associated with one or more eyes of the user, guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user. (See Troutman Para 0088-0092, 0101, 0102 and Robaina Para 1450, 1451, 1461, 1464 as cited above in claim 1). The motivation to combine is set forth as seen above.
Regarding claim 20, Troutman in view of Evangelista and Robaina teaches [a] non-transitory computer-readable medium storing instructions for operating an augmented reality (AR) eyewear device that, when executed by one or more processors, cause the one or more processors to perform a method comprising [(See Troutman Para 0046: instructions may be stored in FLASH memory, Secure Digital Random Access Memory (SDRAM), Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), solid-state hard disk) (See Robaina Para 1709: Code modules or any type of data may be stored on any type of non-transitory computer-readable medium)] receiving an indication of a makeup look selected by a user of the AR eyewear device; analyzing real-time data associated with a face of the user captured by one or more sensors of the AR eyewear device in order to generate a three-dimensional map associated with the face of the user; identifying one or more facial features of the face of the user on the three-dimensional map associated with the face of the user; and causing one or more AR interfaces of the AR eyewear device to superimpose, upon a field of view (FOV) of one or more transparent components of the AR eyewear device associated with one or more eyes of the user, guidance associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user. (See Troutman, Evangelista, Robaina Paras as mentioned above in claims 1 and 19). The motivation to combine is set forth as seen above in.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Troutman
(US 20220202168) in view of Evangelista (US 11069094) in further view of Robaina (US 20200183171) in further view of Kolkmann (US 4405213).
Regarding claim 5, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the guidance includes guidance for applying the one or more cosmetic products to an eye of the user [(See Troutman Para 0011: eyelid shape) (Para 101: an animation to demonstrate how to apply virtual makeup to a particular face part.) (See Evangelista Para 54: the AR makeup system 102 can generate and label an eyelid region)].
However, Troutman in view of Evangelista and Robaina is silent when it comes to wherein the AR eyewear device further comprises one or more actuators configured to adjust the one or more transparent components away from the eye of the user while the guidance for applying the one or more cosmetic products to the eye of the user is provided. Kolkmann is analogous art because it is reasonably pertinent to the problem of applying cosmetic products to the eye while wearing eyeglasses, which is the problem faced by the inventor of the instant application.
Troutman in view of Kolkmann teaches wherein the AR eyewear device further comprises one or more actuators configured to adjust the one or more transparent components away from the eye of the user while the guidance for applying the one or more cosmetic products to the eye of the user is provided. (See Kolkmann Abstract: To enable the application of eye makeup the nose bridge has track-like shanks 5 extending downwardly which are slidingly fitted into mating grooves 6 in the frames so that either frame together with its lens 4 can be moved below the eye. Stop locks in the form of dimples 7 and nibs 8 are provided between the shanks and the grooves to hold the frames in the pushed-up and pushed-down position.) (Claim 1: either the left or the right frame along with a lens (4) mounted therein can be pushed perpendicularly away and downwardly from the transverse axis of the nose bridge) (See Troutman Para 0088: In S315, the mobile application 111 may display a message asking the user if they want a recommendation on how to apply the virtual makeup) (Para 0089: The user interface 1200 may display a button 1203 for selecting a recommendation on how to apply the virtual makeup 1205) (Para 0091: when the user selects to obtain a recommendation (YES in S315), in S319, the mobile application 111 indicates a location on the face image where the virtual makeup is to be applied) (Para 0092: The recommender system 1400 includes a recommendation engine 1407 that retrieves and ranks recommendations. In the case of application of a particular virtual makeup, a recommendation may be for the look that the user has input in step S301) (Para 0101: The border of the original feature may have been determined during the face analysis step, S309. A sequence of video frames may be generated as an animation to demonstrate how to apply virtual makeup to a particular face part) (Para 0102: the user may mimic the demonstrated application of the virtual makeup to apply the makeup)].
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina, which already provides guidance for applying makeup to the eyelid through a transparent component positioned in front of the eye, such that the one or more transparent components away from the eye of the user while the guidance for applying the one or more cosmetic products to the eye of the user is provided as taught by Kolkmann, because Kolkmann discloses the known technique of moving the lens in front of one eye away from that eye so that eye makeup can be applied while the other lens remains in position, and applying this technique to the eyewear of Troutman in view of Evangelista and Robaina would allow the user to apply the product to the eye while continuing to see the superimposed guidance through the remaining transparent component. Thus, it would yield the predictable result of a transparent component that clears the eye during eye-makeup step. One of ordinary skill would have been inclined to make this combination for the advantage stated by Kolkmann of allowing guidance of a user with limited vision and glasses through application of the product.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Troutman
(US 20220202168) in view of Evangelista (US 11069094) in further view of Robaina (US 20200183171) in further view of Yan (US 11393153).
Regarding claim 6, Troutman in view of Evangelita and Robaina teaches [t]he AR eyewear device of claim 1 (See Citations Above in Claim 1) wherein the guidance includes a plurality of steps associated with applying one or more cosmetic products to the one or more facial features of the user in order to achieve the makeup look selected by the user, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: provide, via the one or more AR interfaces, guidance associated with a first step of the plurality of steps; analyze the real-time data associated with the face of the user [(See Troutman Para 0101: A sequence of video frames may be generated as an animation to demonstrate how to apply virtual makeup to a particular face part) (Para 0102: the user may mimic the demonstrated application of the virtual makeup to apply the makeup by making one or more swipes at the facial location) (Para 0104: gestures may be analyzed […] the App 111 determines whether the gesture has been applied in error, i.e., as a mistake. If it is determined that the gesture is made outside of a tolerance amount (YES in S1707), in 51709, a notification message may be displayed).
However, Troutman in view of Evangelita and Robaina is silent when it comes to analyzing the real-time data in order to determine that the first step of the plurality of steps has been completed by the user; and based on determining that the first step of the plurality of steps has been completed by the user, provide, via the one or more AR interfaces, guidance associated with a second step of the plurality of steps. Yan is analogous art because it is directed at augmented reality-based step-by-step instructions for an assembly in which a virtual item depicts a next step in the step-by-step instructions (See Yan Abstract & Claim 1). This is analogous art because it is reasonably pertinent to the problem of advancing augmented reality step-by-step guidance for a manual task, which is the problem faced by the inventor of the instant application.
Yan teaches analyzing the real-time data in order to determine that the first step of the plurality of steps has been completed by the user; and based on determining that the first step of the plurality of steps has been completed by the user, provide, via the one or more AR interfaces, guidance associated with a second step of the plurality of steps. [(See Yan Claim 1: detecting completion of the next step in the step-by-step instructions for the physical assembly; and in response to detecting the completion, rendering the first virtual item) (Claim 6: detecting completion of the next step in the step-by-step instructions for the physical assembly comprises detecting completion of the next step in the step-by-step instructions for the physical assembly using computer-vision in the image.) (Para 56: The computing device 120 can determine that the assembly step has been completed based on the location and orientation marker 230, image capture module 110, image rendering module 140, and location and orientation module 130. The construction progress indicator 210 in the output device 175 can be increased upon the computing device 120 detecting that the assembly step has been completed) (Para 52: The output device 175 can show the user the construction progress 210, as well as both the location where the next item in the construction should be placed 1220 and a depiction 260 of the next item to be placed.) (Para 81: The algorithm to detect the completion of a step can be based on the comparison of […] the scene images […] inside the current virtual brick's bounding box, there is a high probability that the current step is completed, and the Construction Progress label could be updated to indicate the completion of the step) (Para 31: the output device 175 may comprise a pair of augmented reality goggles, or augmented reality glasses)
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Troutman in view of Evangelista and Robaina, which already presents application guidance per face part and analyz[ing] the user’s application gestures against the target region, by analyze the real-time data associated with the face of the user in order to determine that the first step of the plurality of steps has been completed by the user; and based on determining that the first step of the plurality of steps has been completed by the user, provid[ing], via the one or more AR interfaces, guidance associated with a second step of the plurality of steps as taught by Yan, because Yan discloses the known technique of detecting completion of an AR-guided step-by-step computer vision within the target region and advancing to the next step in response, and applying this technique to the existing step-by-step guidance of Troutman in view of Evangelista and Robaina uses image data the combined device already captures. Thus, would yield the predictable result of the guidance advancing from the first application step to the second upon detecting that the product has been applied to the target facial region. One of ordinary skill would have been inclined to make this combination for the advantage stated by Yan of automatic progression through the steps and detection of errors without requiring the user to advance the instructions by hand (See Yan Para 68, 80), thereby providing hands-free progression for a user whose hands are occupied with the applicator, consistent with the goal of Troutman in view of Evangelista and Robaina of guiding the user step-by-step through the selected look.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN Y MASTERS whose telephone number is (571)270-1769. The examiner can normally be reached M-F 7:30 am - 5 pm (First Friday of Biweek Off).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Vasat can be reached at (571) 270-7625. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEVEN YANG MASTERS/Examiner, Art Unit 3715
/PETER S VASAT/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights