DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 04/29/2026 in response to the Non-Final Office Action mailed 01/30/2026 has been entered.
Claims 1-10, 13-18 and 43-45 are currently pending in U.S. Patent Application No. 18/444,343 and an Office action on the merits follows.
Response to Double Patenting Rejections
Applicant’s 04/29/2026 remarks at page 7 request holding in abeyance provisional Double Patenting rejections until agreement on the scope of allowable claims enables evaluation of the same. Examiner notes that the File Wrapper for 18/779,833 comprises issue fee payment documentation (PTO-85B 04/22/2026), and corresponding rejections are likely to be non-provisional upon the issuance of any patent in due course. Examiner agrees to Applicant’s request and maintains the pertinent grounds of rejection so as to avoid piecemeal Examination. See e.g. MPEP 2173.06 and 707.07(g). See also MPEP 1205.02 regarding how any request to hold rejections in abeyance (in the context of Appeal Brief Content) may not meet the requirements of 37 CFR 41.37(c)(1)(iv). Examiner maintains that obvious modification to claims of reference warrants at least those concise Double Patenting rejection(s) reproduced below.
Response to 35 USC § 112 Rejections
Claim 16 as amended sufficiently resolves ambiguity regarding whether or not the recited language “cleaning component configured to” invokes the provisions of 112(f), in view of the manner in which the language in question is struck/eliminated entirely, however the claim remains rejected under 112(b) for the reasons identified below. Remarks identify that support may be found in [0094], and this portion of the Specification is similarly identified by the Examiner in the rationale provided for the 112(b) rejection (NF at page 7). Examiner notes for clarity purposes that the ambiguity regarding 112(f) invocation was only one of the grounds of rejection under 112(b) previously presented, and that it was also asserted that the term “cleaning component” does not have an ‘ordinary and customary’ meaning in the art and that Applicant’s Specification fails to provide a clear/unambiguous meaning for the term (see MPEP 2173.01, and more specifically that flow chart of 2111.01) when considered in view of the requirement that the cleaning component actively cleans (e.g. in response to a control signal), as opposed to passively (e.g. a cleaning solution/product that is not automatically controlled, but instead utilized by a user manually). This issue remains unresolved and is pertinent to the new grounds because the claim as amended requires “one or more processors to control one or more of: a cleaning or a disinfecting”, it remains unclear what component embodiments can be used to accomplish said cleaning and/or disinfecting, and as such it is not clear what such a control[ling] would entail. Stated differently, because it is not clear what is being controlled, it is not clear how it is necessarily controlled, even and especially in view of an implied control signal.
Response to Arguments/Remarks
Applicant's arguments filed 04/29/2026 have been fully considered but they are not persuasive. More specifically Applicant’s remarks at page 9 assert that Besen et al. (US 2019/0208887 A1) fails to fairly disclose limitations similar to now cancelled claims 11-12, because ‘Besen’s discussion of AR is limited to only previewing makeup and style looks’. Remarks at page 9:
PNG
media_image1.png
424
934
media_image1.png
Greyscale
It appears admitted by Applicant that Besen discloses augmented reality in the context of generating a preview, which is also what the recited claim language requires. Examiner does not agree with any implied assertion that Besen discloses only previewing and no AR in the context of guidance during application, however even if this were the case (which the Examiner does not concede), the claim requires generating a preview, and the amended limitations all precede that broad “provid[ing]… guidance”. Given this overlap and clear/explicit disclosure of an AR preview (Besen [0059]), further attempts to distill Applicant’s central argument suggest that Applicant finds Besen to lack a/an “AR version of the three-dimensional map associated with the face of the user”, because the ‘projected templated shapes’ (e.g. indicated by white outlined objects) themselves appear 2-D in nature, and because as illustrated in optional embodiment Fig. 6B, they may be reflected in a 2-D image and accordingly not “applied to the displayed three-dimensional map”. Examiner notes that the foregoing amendment has adopted the language of claim 11 (and arguments at page 10 with reference to Besen [0070] assert non-equivalence for that language), but alters the language of now cancelled claim 12 (now potentially requiring (see 112(b)) the three-dimensional map to itself be ‘displayed’). Applicant appears to argue that Besen’s AR preview content is not 3-D, but instead 2-D, and that perhaps the 3-D digitization of the user’s face at Besen’s Step S531, is used in some other capacity and not for the projection of those templated shapes. Examiner understands this to be unlikely, to involve at most a failure of the provided illustrations in Besen (similar to Applicant’s Fig. 3C), and not indicative of any limitations to or non-equivalence for Besen’s AR content. Stated differently, Examiner understands Besen’s AR preview content to be equivalent to/read on that recited in the claims, particularly in view of that “projection” disclosure as it relates to the templated shapes, and also S531 in particular of Besen. It should be noted that nowhere does Besen disclose the projected templated shapes as being limited to 2-D objects/shapes post-projection/customization, contrary to Applicant’s characterization of the projected templated shapes as being limited to 2-D. Besen’s only explicit disclosure of 2-D is in the context of acquired imagery at [0043] (distinct from the various depth data acquisitions). It is also clear that Besen discloses at [0056] "client device-based sensors are used to perform a three-dimensional digitization S531 of the facial features, color, and tone of the user” and at [0059] “Utilizing client device-based sensors, the client device utilizes prior and active depth mapping, including light filtering, to provide a realistic rendering of what a style may look like on a user”. See also [0043] “The combination of the abovementioned front facing sensors 323 allows for capture and recreation of realistic three-dimensional models of a user's facial features, skin color, and tone. Such depth-dependent digitization of the face is understood in the art, as evidenced by U.S. Pat. No. 9,582,889 B2, which is incorporated herein by reference”. The disclosed projected templated shapes are “customized to the facial features of the user” ([0070]). As understood by the Examiner, Besen’s projection accordingly includes applying templated shapes, even if themselves originally (prior to projection) 2-D in nature, so as to vary them in a manner accounting for 3-D features/topography of the user’s face. Examiner understands Besen’s projection of templated shapes to at least be equivalent to that as illustrated in e.g. Fig. 6 of DeAlmedia of record (reproduced below). What does Besen use the 3-D digitization of the user’s face for, if not for the projection of the templated shapes as suggested with reference to Fig. 6B, so as to “customize” them to the facial features of the user? Applicant’s remarks do not point to any illustrative examples/Figures for the instant Application highlighting any difference. Even Applicant’s Fig. 3C suggests the same/equivalent as e.g. Besen’s Fig. 6B and Fig. 6 of DeAlmedia. For illustrative purposes, from the 01/30/2026 PTO-892, Citation No. V, DeAlmedia et al. “Interactive Makeup Tutorial Using Face Tracking and Augmented Reality on Mobile Devices”, Fig. 6:
PNG
media_image2.png
626
640
media_image2.png
Greyscale
Applicant’s Fig. 3C (right side above ‘night look’ presumably illustrating a selected night look “as applied to the displayed three-dimensional map associated with the face of the user” (new/amended language) and with reference to Applicant’s [0070] “or illustrating examples of a three-dimensional rendering of the look as applied to a three-dimensional model of a face”):
PNG
media_image3.png
502
352
media_image3.png
Greyscale
Lastly, to Applicant’s assertion that prior art of record fails to remedy purported/ unrecognized (particularly if Applicant would agree that Fig 6 of DeAlmedia reads on the recited claim language, and that there is a displayed three-dimensional map even if the map conforms to the user’s face and is not displayed in lieu of the user’s face (arguing otherwise might exclude embodiments in [0070])) deficiencies of Besen, Examiner respectfully disagrees. Examiner has previously identified (1/30/2026 PTO-892) one or more Non-Patent Literature that suggests the non-obvious nature of displayed Augmented-Reality content embodiments, e.g. Treepong et al. “The development of an augmented virtuality for interactive face makeup system” (NPL Citation No. U in the 1/30/2026 PTO-892).
PNG
media_image4.png
306
536
media_image4.png
Greyscale
PNG
media_image5.png
318
806
media_image5.png
Greyscale
Under a reading such that the displayed AR content (Besen e.g. [0059]) reflects a three-dimensional map ([0043]) associated with the face of the user, conforming to, and not displayed in lieu of the user’s face, Examiner would assert Besen remains anticipatory – particularly in view of that disclosure regarding the recreation of “realistic three-dimensional models of a user’s facial features” ([0043]) and that projection/“customization” of the templated shapes for those facial features of the user ([0070], Fig. 6B).
Alternatively, under a reading that may be intended in view of the newly presented language (differing from claim 12 in view of the language identified in the 112(b) rejection below – line 22 at that second ‘generat[ing]’ step, line 23 “the displayed three-dimensional map”) wherein the AR content generated is displayed in a manner more similar to that of Treepong, Examiner would assert modification to Besen so as to display alternative AR content would at most require a substitution of known AR content elements for another in a manner characterized by both predictable results and a reasonable expectation of success.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
At least instant claims 1/43/44 are provisionally (upon issuance, non-provisionally) rejected on the ground of nonstatutory double patenting as being unpatentable over one or more claims of copending Application No. 18/779,833 (PGPUB US 2026/0024445 A1) in further view of references of record (and corresponding rationales) as applied in the prior art rejections that follow and where necessary. Examiner notes the claims e.g. 1/19/20 of reference recite “analyzing real-time data associated with a face of a user captured by one or more sensors in order to generate a three-dimensional map associated with the face of the user; identifying one or more facial features of the face of the user on the three- dimensional map associated with the face of the user;” and a final providing guidance limitation that is of a breadth falling under/within (even if narrower) the broader/genus recited in the instant claims. Examiner also notes Fig. 3, steps 302 and 304 of the reference Application closely correspond to the instant Application’s Fig. 4, steps 404 and 406 respectively. One or more dependent claims also appear to have a direct correspondence (e.g. instant claim 2 to reference claim 2, etc.,). This is a provisional (see note above) nonstatutory double patenting rejection.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10, 13-18 and 43-45 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1/43/44 recite(s) the limitation, following line 22 (of claim 1) at that second ‘generat[ing]’ step, in line 23 “the displayed three-dimensional map”. There is insufficient antecedent basis for this limitation in the claim. The first ‘generating’ step establishes basis for a generated (but not yet displayed) “AR version of the three-dimensional map associated with the face of the user;”. The second generating, that is the generation of that 3D preview, displays said AR version, but this limitation/displaying appears after the language in question and no prior limitations serve to establish basis for any displayed three-dimensional map distinct from the AR version. Requirements for clarity and precision as described in MPEP 2173.02 are also unmet because the language in question may serve to suggest a distinction between the AR version displayed and a displayed three-dimensional map (if ‘the’ is read as ‘a’ establishing minimally required basis therein), and it is not clear as to what this distinction would be. As best understood by the Examiner, and consistent with claim 12 as previously presented, Examiner does not understand the three-dimensional map to actually be displayed – but to instead be used/considered in the generation and display of AR content.
Claim 16 recites the limitation in part “to control one or more of a cleaning or a disinfecting”, for which it is unclear what such a control necessarily constitutes, particularly in view of those concerns previously raised regarding what structure accomplishes any cleaning or disinfecting. See the ‘Response to 35 USC § 112 Rejections’ section provided above.
Dependent claim(s) 2-10,13-18 and 45, inherit and fail to cure that/those deficiencies identified above for the case of independent claim(s) 1/43/44, and are rejected accordingly.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 43-45, 1-3, 5-13 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system”.
As to claim 43, Besen discloses a computer-implemented method for operating an intelligent cosmetic compact device (Fig. 2A, 210 in conjunction with 315, [0071] “In an embodiment, during the instructional unit, client device-based sensors actively monitor the performance of the user and compare output metrics to established standards for corresponding techniques. If it is determined, for example, that the user is applying the incorrect cosmetic, the client device may provide guidance and encouragement to the user via audible alerts, visual alerts, tactile alerts, or a combination thereof. The user may, in turn, respond to the alert and continue makeup application with the confidence of knowing the technique is being performed appropriately”, etc.,), the method comprising:
receiving, by one or more processors, an indication of a makeup look selected by the user via a user interface ([0044] “FIG. 4A is a flowchart of sensing and evaluation of sensor inputs during a coaching module, according to an exemplary embodiment. Following user selection of the desired look …”, [0057] “First, a user selects a makeup coaching module S534. For example, the user may select a specific style or look of interest. Alternatively, client device based software may intelligently offer occasion-based inspirations that complement the user”, [0058] “In an embodiment, the client device-based software may offer a variety of styles including, but not limited to, everyday office, first date, movie night, and fine dining. For each occasion, a style or look is offered to the user. To this end, client device software may also incorporate social media content associated with the user's user profile to better inform and predict styles of interest to the user based upon preferences indicated within the user's social media presence”, [0060] “The client device-based software will adjust recommended occasion-based looks accordingly and present them to the user for look selection”, [0076], etc.,);
analyzing, by the one or more processors, real-time data, captured by one or more sensors, associated with the face of the user, in order to generate a three-dimensional map associated with the face of the user ([0043] “The front-facing sensors 323 of the client device 315 may include, but are not limited to, an infrared camera 317, an infrared flood illuminator 318, a proximity sensor 322, a dot projector 319, a visible light camera 320, and a visible light flood illuminator 321. The combination of the abovementioned front-facing sensors 323 allows for capture and recreation of realistic three-dimensional models of a user's facial features, skin color, and tone. Such depth-dependent digitization of the face is understood in the art, as evidenced by U.S. Pat. No. 9,582,889 B2, which is incorporated herein by reference. In an embodiment, recognition of the facial features is performed via digital reconstruction of two-dimensional images acquired from a visible light camera”, [0056] “client device-based sensors are used to perform a three-dimensional digitization S531 of the facial features, color, and tone of the user. Client device-based software then adjusts to calibrate the image and color projection according to ambient light”, etc.,);
identifying, by the one or more processors, one or more facial features of the face of the user on the three-dimensional map associated with the face of the user ([0059] “In another embodiment, augmented reality capabilities enable the user to realize a look prior to look selection. Utilizing client device-based sensors, the client device utilizes prior and active depth mapping, including light filtering, to provide a realistic rendering of what a style may look like on a user”; [0069], [0046] “As the user applies makeup, the client device actively monitors user position, orientation, movement and facial features S442. Client device-based sensors, controlled by client device-based software, generate data including, but not limited to, user facial features, user head orientation … Stored data is accessed during display projection of future images to improve spatial projection of templated shapes relative to prior instances S443. To this end, stored data, including that which is related to relative user position and orientation, are used to predict future positions and orientations of the user so that adjustments to the display projection are more intuitive, allowing templated shapes to follow the contours and movements of the user S444. For example, initially, a user is applying makeup to the right side of the face and the client device display is projecting a templated shape onto the semi-transparent display, accordingly. As the user moves the head to more easily view a section of the face, client device-based software recognizes the movement and adjusts the display projection accordingly. There exists, however, delay in the rendering as the client device-based software generates the display projection. With subsequent use, the client device-based software will generate a library of prior user motions that can be called upon during future instances of similar motions. In this way, as the number of instances of a certain motion and stored data increase, the client device-based software will be able to better predict the velocity and direction with which a movement is occurring, thus eliminating lag time in display projection generation”, [0070] “These templated shapes, indicated by white outlined objects in FIG. 6B, are customized to the facial features of the user and are presented similarly to a "paint-by-numbers" approach, wherein each shape corresponds to a specific makeup cosmetic”, etc., Examiner understands this ‘identifying’ with reference to Applicant’s block 406 and [0080], to facilitate the AR rendering that is used to convey e.g. that ‘preview’ of [0081] of Applicant’s PGPUB, among other displayed content (not only a preview but also during application) – and Besen at least suggests the same as those templated shapes are “customized to the facial features of the user” and projected so as best align with the positions and orientations of user features in real-time, as the user/operator moves/performs portions of a makeup/cosmetic application process); and
generating, by the one or more processors, an augmented reality (AR) version of the three-dimensional map associated with the face of the user, and displaying, via an AR component of the intelligent cosmetic compact device, the AR version of the three-dimensional map associated with the face of the user (Fig. 6B 637, AR disclosure of [0059], [0069], [0046], [0070] “These templated shapes, indicated by white outlined objects in FIG. 6B, are customized to the facial features of the user and are presented similarly to a "paint-by-numbers" approach, wherein each shape corresponds to a specific makeup cosmetic”, etc., in view of that “realistic three-dimensional model” disclosure of e.g. [0043], [0059], etc., in further view of the remarks/considerations presented above);
generating, by the one or more processors, a three-dimensional preview of the makeup look selected by the user as applied to the displayed (see remarks above regarding possible interpretations for displayed AR content more like Besen/DeAlmedia, or that of Treepong, in addition to the 112(b) raised above) three-dimensional map associated with the face of the user, and displaying, via the AR component, an AR version of the three-dimensional preview of the makeup look; ([0059] “In another embodiment, augmented reality capabilities enable the user to realize a look prior to look selection. Utilizing client device-based sensors, the client device utilizes prior and active depth mapping, including light filtering, to provide a realistic rendering of what a style may look like on a user”, [0069] “To aid the user in style and look selection, augmented reality capabilities enable the user to realize a style prior to look selection. Utilizing client device-based sensors, the client device utilizes prior and active depth mapping, including light filtering, to provide a realistic rendering of what a style may look like on a user 635”); and
providing, by the one or more processors, via the user interface, guidance associated with applying one or more cosmetic products to the facial features of the user in order to achieve the makeup look selected by the user ([0061] “An instructional unit can include, but is not limited to, video tutorials, projection of templated shapes, or a combination thereof. Next, user controlled step-by-step makeup application steps are projected from the client device display and are visible through the semi-transparent display S537”, [0070], [0071] “In an embodiment, during the instructional unit, client device-based sensors actively monitor the performance of the user and compare output metrics to established standards for corresponding techniques. If it is determined, for example, that the user is applying the incorrect cosmetic, the client device may provide guidance and encouragement to the user via audible alerts, visual alerts, tactile alerts, or a combination thereof. The user may, in turn, respond to the alert and continue makeup application with the confidence of knowing the technique is being performed appropriately”, etc.,).
Under an assertion that permissible interpretation of the AR version of the three-dimensional preview requires content that should appear more like that of Treepong and less like that of Besen/DeAlmedia Fig. 6 (see remarks above), Treepong evidences the obvious nature of alternative AR content featuring an explicitly displayed 3D map associated with the face of the user (Treepong Figs 2, 3, 9, etc., – see also remarks above).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Besen such that the AR content displayed further comprises a displayed 3D map associated with the face of the user as taught/suggested by Treepong, since such a display/rendering of content constitutes a simple substitution of known AR content elements for and/or in addition to others/those of Besen, in a manner with predictable results (as evidenced by Treepong) and further characterized by a reasonable expectation of success.
As to claim 44, this claim is the non-transitory CRM claim corresponding to the method of claim 43 and is rejected accordingly. Besen further discloses memory associated with client device 715 in addition to that of e.g. remote system structure 750 (see Fig. 7).
As to claim 1, this claim is the system/device claim corresponding to the method of claim 43 and corresponding limitations are rejected accordingly – e.g. those same steps as performed by those one or more processors. Claim 1 requires additional structural limitations which Besen further discloses, i.e. a system/device comprising
a portable cosmetic compact housing (Figures 1 and 2A, portable makeup/cosmetic compact 210 comprising housing sub-component 216 for housing client device/smartphone 315, housing for color palette 211, upper lid 213, mirror 201, etc.,) configured to be opened and closed (Figs. 2A, 2C and 2D, [0037], [0037] “From the user's perspective, both a mirror reflection of the user and client device-generated objects will be visible simultaneously on through the semitransparent display 205”, [0038] opens/closes via flexible hinge 209, etc.,), the portable housing comprising:
a display that is accessible when the portable cosmetic compact housing is opened (Figs. 1, 2A, [0037], display of 315 is viewable/accessible through 205 when compact 210 is opened);
one or more compartments for storing cosmetic products that are accessible when the portable cosmetic compact housing is opened (211);
a user interface (UI of 315, [0037] “To implement the interactive user interface of the client device, the client device, with activated coaching module, is positioned within the client device housing 216”, [0073] “Further, the user interface or the client device can display tutorials on fundamentals of makeup application. The user interface can create and download protocols for a regimen or routine. The user interface can train, track usage and compare the tracked usage to the protocol, the regimen, and the routine. The user interface can calculate a score based on the tracked usage. The user interface can store the scores and the tracked usage of the coaching software in the memory of the client device. Moreover, the user interface can be used to make a purchase of any products related to the makeup products registered within the client device-based software as well as recommendations of color tones, product lines, and other products related to the current style, look, or future experimental techniques”; Examiner understands the ‘UI’ to implicitly be structural in nature, as it is a distinct structural limitation of the device of claim 1, like that/those display, compartment(s), sensor, processor, and memory – Applicant’s interface 210 is disclosed as housing the ‘AR component’, and ‘component’ is arguably a nonce term as it concerns 112(f) invocation, MPEP 2181, and the UI may accordingly serve as a structural modifier for disclosed ‘software component’ embodiments);
one or more sensors configured to capture real-time data associated with a face of a user (Fig. 3B, [0043] “The front-facing sensors 323 of the client device 315 may include, but are not limited to, an infrared camera 317, an infrared flood illuminator 318, a proximity sensor 322, a dot projector 319, a visible light camera 320, and a visible light flood illuminator 321”);
one or more processors (inherent as required for 315, and also as part of remote portions of system as a whole/server(s) 750); and
one or more memories ([0046] “store the acquired data to local storage, cloud-based storage, or a combination thereof”, [0053], [0073], Fig. 7, etc.,).
As to claim 2, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further discloses the device wherein the one or more sensors include one or more of a camera or a depth sensor ([0043] camera(s) 320/317, dot projector 319 for depth mapping, [0059] “Utilizing client device-based sensors, the client device utilizes prior and active depth mapping”, etc.,).
As to claim 3, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further discloses the device wherein the user interface includes a haptic feedback component, and wherein providing guidance associated with applying the one or more cosmetic products to the facial features of the user includes providing haptic guidance via the haptic feedback component ([0049] “According to an embodiment, data generated from client device-based sensors and stored to local storage, cloud-based storage, or a combination thereof, may be utilized to provide real-time feedback to the user regarding user performance in the form of visual commands, audible commands, tactile commands, or a combination thereof”, [0071] “If it is determined, for example, that the user is applying the incorrect cosmetic, the client device may provide guidance and encouragement to the user via audible alerts, visual alerts, tactile alerts, or a combination thereof”, Examiner understands haptic/tactile to be synonymous in the context of feedback/commands for directing the user – see also Hong et al. (US 2024/0108119 A1) as applied below for the case of claim 4).
As to claim 5, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further discloses the device wherein the user interface includes an audio component, and wherein providing guidance associated with applying the one or more cosmetic products to the facial features of the user includes providing audio guidance via the audio component (see [0049], [0071] audio command/alert embodiment(s), [0054-055] output of 315 comprises video and audio data).
As to claim 6, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further discloses the device wherein the providing guidance associated with applying the one or more cosmetic products to the facial features of the user includes providing visual guidance via the display (see [0049], [0071] video command/alert embodiment(s), in further view of that video tutorial disclosure of e.g. [0061] and the display of those projected template shapes [0046]).
As to claim 7, Besen in view of Treepong teaches/suggests the device of claim 6.
Besen further discloses the device further comprising an accelerometer, and wherein providing the visual guidance via the display includes adjusting an orientation of the visual guidance provided via the display based on data captured by the accelerometer (inherent to 315, in view of the manner in which most smartphones, and Apple iPhone as disclosed [0043] comprise accelerometers as a standard feature, for detecting device orientation, changes in motion etc., and e.g. modifying display functions/orientation accordingly (in addition to crash/fall/drop detection among others) – Examiner has previously taken Official Notice (2144.03) to the manner in which smartphones typically/commonly adjust displayed content orientation in response to accelerometer data and detected changes in device orientation).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Besen in view of Treepong so as to rely on an accelerometer built in to 315 to optimize/adjust the orientation of displayed content, since such a modification would amount to no more than applying known techniques to yield a predictable/expected result further characterized by a reasonable expectation of success. See MPEP 2143 Example Rationale A, with reference to KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007) (at page 401 of the bound volume “a combination of familiar elements according to known methods is likely to be obvious when it does no more than yield predictable results”).
As to claim 8, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen discloses the device further comprising a communication interface configured to communicate with a mobile device, external to the intelligent cosmetic compact device (Besen 315 comprises a communication interface configured to communicate with other/additional mobile devices, external/remotely located from 210, for the instances that detachable component 315 of 210 is situated within housing 216 or otherwise).
As to claim 9, Besen in view of Treepong teaches/suggests the device of claim 8.
Besen further discloses the device wherein the communication interface is a wired communication interface (Fig. 8, connector 827 wired/direct electrical connection with micro/mini/USB port of 315, [0074] “Such electrical connection of a makeup compact to a client device is understood in the art, as evidenced by U.S. Pat. No. 9,692,864 B1, which is incorporated herein by reference”).
As to claim 10, Besen in view of Treepong teaches/suggests the device of claim 8.
Besen further discloses the device wherein the communication interface is a wireless communication interface (common to 315, [0052] “The proximity sensor housed within the client device detects the presence of the client device housing and sends a signal to the client device-based software to begin the coaching module S457. In an embodiment, the client device and client device housing further comprise wireless identification tags. These wireless identification tags, including, but not limited to, near field communication devices, provide a unique makeup compact identifier that would expedite the user onboarding experience, instantly identifying the available makeup palette”, etc., while not required/relied upon, see also Truong US2021/0195713 A1 Fig. 2, wireless communication 162 of compact 150 for communication with 104).
As to claim 13, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further discloses the device wherein the three-dimensional preview of the makeup look selected by the user includes a three-dimensional preview of an application process of the makeup look selected by the user ([0061] “An instructional unit can include, but is not limited to, video tutorials, projection of templated shapes, or a combination thereof. Next, user controlled step-by-step makeup application steps are projected from the client device display and are visible through the semi-transparent display S537. A video tutorial of a first step of the instructional unit is displayed. Following the video tutorial, the appropriate templated shapes are projected onto the semitransparent display. As the user completes each makeup application step according to the templated shapes projected from the client device display, the user indicates as much to the client device-based software via audible command, visual command, tactile command, or a combination thereof. If additional steps are required to complete the instructional unit ( e.g. if the instructional unit requires makeup application of more than one cosmetic) S538, the instructional unit begins the next step, including the next step of the video tutorial and the appropriate templated shapes. If no additional steps are required S538, makeup application of the current instructional unit has ended”).
As to claim 17, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen discloses the device further comprising a light source configured to provide light to the face of the user ([0043] visible light flood illuminator 321 of 315).
As to claim 45, Besen in view of Treepong teaches/suggests system/device of claim 1.
Besen further teaches/suggests the device wherein identification of the one or more facial features of the face of the user on the three-dimensional map associated with the face of the user includes (Besen [0043] “Such depth-dependent digitization of the face is understood in the art, as evidenced by U.S. Pat. No. 9,582,889 B2, which is incorporated herein by reference”)
Besen fails to explicitly disclose utilizing/applying any trained ML model to the digitalized/realistic 3-D model associated with the user’s face, and instead appears to suggest that the corresponding features of the 3D map are determined as part of the model generation.
Treepong however evidences the obvious nature of utilizing/applying a trained ML model to identify one or more facial features (Treepong Figs. 1-3, Face Tracking in view of 3D face mesh and rigs, and tracking points, Section 3.1 “The 33 feature points of the human face including eye, eyebrow, mouth, nose, cheek, jaw, and chin, are tracked by using Kinect high detail face points function [12]. Those points are mapped to the facial skeleton of the model. Then, the face model will be able to move corresponding to the user’s facial expression in real-time, as shown in Fig. 2. To overlay the face texture on the 3D model, UV mapping technique, which is the process for projecting a 2D image to a surface of a 3D model, is used in this step. First, the mesh of 3D model is unwrapped by using the unwrap function in 3ds max. An image of the subject is captured and mapped onto the unwrapped UV for generating the face texture. Finally, the generated texture is applied on the model to see the mapping result, as illustrated in Fig. 3”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Besen in view of Treepong such that one or more facial features are identified by means of a trained ML model as taught/suggested by Treepong, the motivation as similarly taught/suggested therein that such a trained model would be readily accessible via software development kits SDK/APIs (such as with Kinect v1, v2, Azure Kinect) and implemented with a reasonable expectation of success (as evidenced by identified/extracted feature points in Treepong).
2. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system” and Hong et al. (US 2024/0108119 A1).
As to claim 4, Besen in view of Treepong teaches/suggests the device of claim 3.
Besen suggests the device wherein the haptic guidance includes patterns of haptic feedback associated with applying respective cosmetic products to the respective facial features of the user ([0071] “If it is determined, for example, that the user is applying the incorrect cosmetic, the client device may provide guidance and encouragement to the user via audible alerts, visual alerts, tactile alerts, or a combination thereof”; Examiner notes also that the ‘to the respective features’ appears at the minimum suggested by Besen in view of e.g. Fig. 6B and the manner in which those projected overlays pertain to different features – e.g. white overlay for area underneath eyebrow, eye lid, and inner corner of the eye; It also stands to reason that for a user to be efficiently/effectively guided by the feedback of Besen, different and readily discernable feedback, for various/different situations encountered by the user during the process of achieving the desired look, would be required).
Hong further evidences the obvious nature of haptic guidance including patterns of haptic feedback associated with applying respective cosmetic products to the respective facial features of the user (see path presentation engine 118, path determination engine 114, and face detection engine 112, [0013] “From this recording, a relative applicator path is determined that specifies locations of a makeup applicator during the application. The relative applicator path is defined with respect to distances from various facial landmarks, such that the relative applicator path can be presented with respect to other faces by calculating the distances from the various facial landmarks of the other faces. Presentations including, among other things, ghost outlines of applicators, audio feedback, haptic feedback, or visual prompts may be generated in order to help a second subject guide an applicator along the relative applicator path and thereby improve the application of makeup by the second subject”, [0055] “if the second applicator 602 is determined to be outside of a predetermined range around the relative applicator path, an auditory or haptic indication may be provided to prompt movement of the second applicator 602 back to the relative applicator path. Also, in embodiments wherein the relative applicator path includes an amount of pressure applied by the applicator, the presentation may include a visual, auditory, or haptic indication when it is determined that an amount of pressure being applied with the second applicator 602 is different from the amount of pressure indicated by the relative applicator path”, etc.,).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Besen to such that the tactile/haptic feedback disclosed therein further comprises that as provided for an applicator component distinct from 315, and including patterns of haptic feedback associated with applying respective cosmetic products to the respective facial features of the user as taught/suggested by Hong, the motivation as similarly taught/suggested therein that such feedback may further assist the user in a more exact/precise manipulation of applicators frequently used in conjunction with the compact device, and further defined with respect to the user specific relative applicator path (Hong [0013]).
3. Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system” and Jung (US 2020/0405039 A1).
As to claim 14, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen fails to disclose the device as further comprising one or more temperature control components configured to control temperatures associated with the one or more compartments for storing the one or more cosmetic products within a particular range of temperatures.
Jung however evidences the obvious nature of one or more temperature control components configured to control temperatures associated with the one or more compartments for storing the one or more cosmetic products within a particular range of temperatures ([0063] “Additionally, the controlling means 90 senses the temperature around the cosmetics contained in the cartridge 30 inside the body 11 by a temperature sensor, and controls the operation of the cooling means 80 accordingly”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Besen in view of Treepong so as to comprise one or more temperature control components configured to control temperatures associated with the one or more compartments for storing cosmetic products within a particular range of temperatures as taught/suggested by Jung, the motivation as similarly taught/suggested therein that such a temperature control means ensures the associated cosmetic products maintain their integrity/ useability for the user.
4. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system” and Kreuzer et al. (US 2020/0387942 A1).
As to claim 15, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen further suggests the device wherein the non-transitory computer-readable instructions, when executed by the one or more processors, further cause the one or more processors to:
track the usage of the one or more cosmetic products from the one or more compartments (Besen [0073] suggests tracking product usage as it pertains to a tracked usage of the coaching software and those various protocols, routines, and/or regimens selected/performed by the user – wherein at least which products are used, in addition to use frequency/amount, is readily measured in association with which and how frequently desired looks are selected by the user);
provide a notification soliciting related makeup products, via the user interface ([0072-0073]).
Besen fails to explicitly disclose determining a low-stock status/requirement for refill of one or more cosmetic products and a corresponding prompt/notification distinct from one that is soliciting makeup products more broadly (i.e. soliciting more specifically one that is low/requiring refill/replenishment).
Kreuzer however evidences the obvious nature of determining a low-stock status/requirement for refill of one or more cosmetic products and a corresponding prompt/notification provided on the basis of intelligently tracked use (Fig. 7 706-708, store product use data in user profile and analyzing use event data to generate feedback information, in view of Fig. 500 recommendation e.g. “Don’t forget to replace your skin care product after 5 uses”, [0052] “Still further, the database 210 may store a set of rules regarding the appropriate frequency, duration, and manner of use for a particular activity. The set of rules may also include an estimated total number of times the activity may be performed and/or an estimated total duration over multiple instances of performing the activity before products related to the activity need to be replenished, such as the number of showers before the user needs to replace the soap and shampoo. In addition to sets of rules, the database 210 may store machine learning models for determining the appropriate frequency, duration, and manner of use for the particular activity that is specific to a particular user based on the user's previous patterns of use and/or the results experienced by the user”, [0056] “The user feedback information may include a recommendation to replenish the skin care product (e.g., after a threshold number of uses of the skin care product which in some instances may be determined via the activity data, or when the skin care product exceeds a threshold age according to the set of rules and/or the machine learning models)”, [0063], etc.,)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Besen in view of Treepong such that product solicitation of e.g. Besen [0072-0073], more specifically concerns at least one product characterized by a low-stock status/requirement for refill/replenishment based on tracked use as taught/ suggested by Kreuzer, the motivation as similarly taught/suggested therein and readily recognized by POSITA, that such a solicitation might be more effective given the users usage history and further serve to ensure the user does not run out of the product and/or has sufficient product for subsequent application(s) (for one or more desired/frequently desired looks).
5. Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system” and McCormick et al. (US 2014/0096801 A1).
As to claim 16, Besen in view of Treepong teaches/suggests the device of claim 1.
Besen fails to explicitly disclose the device further causing the one or more processors to control one or more of a cleaning or a disinfecting of one or more components of the intelligent cosmetic compact device.
McCormick further evidences the obvious nature of instructions for a control of one or more of a cleaning or a disinfecting ([0020] “According to an embodiment of the present invention, the base portion is further comprised of a sanitation control means, such sanitation control means comprising one or more components selected from the group of a timer, a display element (e.g., LED, LCD), an alert source (e.g., speaker capable of creating an audible tone, LED or other notification indicator) one or more human interface elements allowing for the selection or editing of settings related to a cleaning cycle, including, but not limited to, the length of a cleaning cycle, the type of cleaning cycle (e.g., ultrasonic, agitation, soak), the type of brushes to be cleaned or any combination thereof”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system of Besen in view of Treepong so as to further comprise instructions for the issuance of control signals related to cleaning and/or disinfecting (e.g. the activation of sterilizing UV lights, etc.,) as taught/suggested by McCormick and/or readily recognized by PHOSITA (see also Ezzina [0021] previously applied) that such a control/signal may serve to automate one or more cleaning/ disinfecting operations broadly, thereby reducing and/or even eliminating a need for user intervention.
6. Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Besen et al. (US 2019/0208887 A1) in view of Treepong et al. “The development of an augmented virtuality for interactive face makeup system” and Truong (US 2021/0195713 A1).
As to claim 18, Besen in view of Treepong teaches/suggests the device of claim 17.
Besen fails to explicitly disclose the device wherein e.g. 315 provides particular lighting conditions while the one or more cosmetic products are applied to the facial features of users (absent a broad interpretation wherein the ‘particular’ lighting condition is, e.g. ‘on’ as distinguished from otherwise).
Truong however evidences the obvious nature of compact 150 configured to provide particular lighting conditions facilitating cosmetic application (Fig. 2, [0001] “lighting system can be embodied in a carried accessory, such as cosmetic compact case, a make-up brush, or a mirror having a computer system with network connectivity. In one embodiment, the lighting system connects to a server computing system that stores a plurality of light conditions. Light usually affects the way cosmetics and make-up are perceived by the user and observers. Generally, a user does not apply cosmetics or make-up under lighting conditions that are similar to the lighting conditions in which a user will be present”, [0044] “Then, the computing device 164 can issue instructions for adjusting the output of the light emitting diodes 158 in order so that the combined ambient lighting conditions and the light produced by the light emitting diodes together resembles a simulated lighting condition which is produced proximate to the compact case 150 so that the user can see what the make-up will look like in a selected simulated lighting conditions. The combined simulated lighting conditions can be verified by the light sensors 156 or else, the light emitting diodes 158 output may be further adjusted. The compact case 150 may include controls, such as button 166 to scroll through certain pre-selected locations, the lighting conditions of which may be simulated. Then, when a location is selected, the light emitting diodes 158 will be controlled to output light that when combined with the ambient light will result in the lighting conditions of the selected location”, etc.,).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Besen in view of Treepong to further provide particular lighting conditions so as to facilitate a desired cosmetic application as taught/suggested by Truong, the motivation as similarly taught/suggested therein that such a configurable lighting enables the user/operator to better achieve the desired look as it is likely to be perceived under expected/future/destination specific lighting conditions (Truong [0001-0003]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN L LEMIEUX whose telephone number is (571)270-5796. The examiner can normally be reached Mon - Fri 9:00 - 6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN L LEMIEUX/Primary Examiner, Art Unit 2669
Prosecution Insights