DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1-20 remain pending, and are rejected.
Claims 1 and 20 have been amended.
Response to Arguments
Applicant’s arguments filed on 1/23/2026 with respect to the rejection under 35 U.S.C. 103 have been fully considered, but are moot in light of new grounds of rejection. Applicant’s amendments necessitated new grounds of rejection.
Applicant’s amendment of the dispensing system physically integrated to the mirror device necessitated changing the combination of references to include Mao (US 20180174125 A1). It would have been obvious to one of ordinary skill in the art to combine the system of Saban disclosing the device emulating a mirror using a camera video stream to generate a digital mirror to apply virtual products and order the products from the device with the device of Mao that uses a camera to superimpose products on the user, and allows the user to purchase the product, and dispense the product to the user (see Mao: [0025]; [0031]; Fig. 2, #110,103), such that the product is immediately dispensed to the user after previewing the item (Mao: [0003]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-6, 10-11, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable by Saban (US 20180278879 A1) in view of Mao (US 20180174125 A1).
Regarding Claim 1: Saban discloses a system comprising:
a mirror device that comprises a mirror surface; (Saban: [0070] – “FIG. 5 illustrates an example of the digital mirror according to an embodiment of the invention. This embodiment is configured for close-up imaging, such as for makeup, eyewear, etc. The digital mirror 500 includes a digital display 505, at least one camera 510, and illumination device 515. In this particular example the digital display 505 is divided into two sections: section 503 displays the image from the camera, after appropriate transformation and translation, and section 504 that is used as a user interface using the touchscreen capability of the display screen”; Saban: Fig. 5 showing the mirror device with a mirror surface).
present a reflection of a body portion of a user on the mirror surface, wherein the body portion comprises a facial portion; (Saban: [0072] – “the camera obtains the image from above the display screen, such that if the image from the camera would be displayed as is, it will be distorted and would not appear as a mirrored image. Moreover, depending on the height of the user, the image of the head would appear at different positions within the digital display 505. Accordingly, the image from the camera is first transformed according to any of the embodiments described above. Additionally, the image is translated so as to position the user's head within an area 518 designated as the center of the screen”).
determine a plurality of facial attributes of the user based on the reflection of the body portion of the user; (Saban: [0087] – “the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc. Using the determined features, at step 702 the system classifies the face features according to preprogrammed list of classifications. Optionally the system may use the facial features classifications to generate a unitary facial classification”).
determine a plurality of environmental parameters in nearby surroundings of the user; (Saban: [0080] – “a sensor detects the lighting environment about the face of the person receiving the demonstration. According to another embodiment, a processor analyzes the image from the camera to determine the lighting environment about the face of the person receiving the demonstration. Using the lighting information, and optionally the skin complexion of the person receiving the demonstration, the controller adjusts the light sources to provide lighting at a desired temperature”).
present a modified reflection of the body portion of the user on the mirror surface along with one or more lifestyle products based on the determined plurality of facial attributes and the determined plurality of environmental parameters; (Saban: [0087] – “the user may select a look from a list provided by the system. In step 708 the system virtually applied the makeup look onto the image of the user projected on the screen”).
Saban does not explicitly teach present a dispensing system physically integrated to the mirror device configured to dispense out, in real time at a location of the mirror device, at least one lifestyle products of the one or more lifestyle products based on a real-time analysis of the modified reflection on the mirror surface. Notably, however, Saban does disclose providing results with the objective of the customer purchasing the items (Saban: [0005]).
To that accord, Mao does teach present a dispensing system physically integrated to the mirror device configured to dispense out, in real time at a location of the mirror device, at least one lifestyle products of the one or more lifestyle products based on a real-time analysis of the modified reflection on the mirror surface. (Mao: [0038] – “If the product is in stock at the current vending machine that the user is at, then the product collection arm is moved to the correct height and the bar code is scanned at 707 to ensure that the product purchased is the product having the product ID as selected by the user”; Mao: [0031] – “FIG. 2 illustrates a perspective view of the interactive jewelry vending machine 100 that includes product collection door 103 at the bottom of the apparatus, and point of sale system 110 connected to control board… the picture of the user may be superimposed with the product wherein the image of the user may be improved or otherwise altered using any type of image processing system”; Mao: Fig. 2, #110,#103 displaying the vending device that includes images of the user and a product collection door to dispense the product). In summary, the device includes the product dispensing mechanism and the display of the user superimposed with the product together.
It would have been obvious tom one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Saban disclosing the system or determining items and displaying them on a user with a mirror device with the dispensing system physically integrated into the device as taught by Mao. One of ordinary skill in the art would have been motivated to do so in order to allow a user to test a product and ensure they receive the correct item (Mao: [0004-0005]).
Regarding Claim 5: Saban in view of Mao discloses the limitations of claim 1 above.
Saban does not explicitly disclose wherein the dispense out of the at least one lifestyle product is performed automatically in response to a user confirmation on the presented modified reflection uniquely attuned to the plurality of facial attributes and the plurality of environmental parameters. Notably, however, Saban does disclose presenting a mirror image of the user with a virtual application of the items based on their facial features and lighting (Saban: [0087]).
To that accord, Mao does teach wherein the dispense out of the at least one lifestyle product is performed automatically in response to a user confirmation on the presented modified reflection. (Mao: [0037] – “If the user purchases the product at 704, then the system determines if the product is locally in stock in the machine; Mao: [0038] – “if the product is in stock at the current vending machine that the user is at, then the product collection arm is moved to the correct height and the bar code is scanned at 707 to ensure that the product purchased is the product having the product ID as selected by the user”).
It would have been obvious tom one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Saban disclosing the system or determining items and displaying them on a user with a mirror device with the dispensing performed automatically in response to a user confirmation as taught by Mao. One of ordinary skill in the art would have been motivated to do so in order to allow a user to test a product and ensure they receive the correct item (Mao: [0004-0005]).
Regarding Claim 6: Saban in view of Mao discloses the limitations of claim 1 above.
Saban further discloses wherein the modified reflection comprises at least a change in a facial feature of the user when the one or more lifestyle products is to be used by the user. (Mao: [0058] – “an augmented reality module that can change the appearance of a body part, color, orientation and texture of a facial feature or an item or object in the foreground or background of the image, e.g., the module can add hair to the user, change the user's eyes, skin and hair color, can change the pose of the eyes and the like”).
Regarding Claim 10: Saban in view of Mao discloses the limitations of claim 1 above.
Saban further discloses a physical try-on unit configured to form a user-specific visualization indicative of a lifestyle product visually discernible to be worn by the user or applied to a body portion of the user depending on a type of lifestyle product on the physical try-on unit. (Saban: [0040] – “The augmented reality module 104 can then process the model/mask in real-time at a given frame rate, which can be at a lower or higher frame rate than the original and at the same or a different size than the original. For example, once extracted the appearance of the lips can be modified by proper coloration—creating the impression of a desired shaped lips”; Saban: [0033] – “The augmented reality module 104 can be adapted, for example, to provide virtual color and texture replacement, virtual dressing, object insertion and the like. In the specific embodiments disclosed herein the augmented reality module 104 is configured to alter the color and intensity of selected pixels so as to provide virtualized makeup. In other embodiments, the augmented reality module 104 is configured to superimpose an image over the user's face, e.g., to virtualize glasses over the face of the user”).
Regarding Claim 11: Saban in view of Johnson discloses the limitations of claim 1 above.
Saban further discloses an automatic video tutorial creator system configured to generate instructional videos from a series of images that depict a step-by-step process for using the one or more lifestyle products. (Saban: [0075] – “the stored video is then automatically edited by breaking it into sub-sessions. The sub-sessions may include, for example: Foundations, Powder, Bronzer, Concealer, Lipstick, Lip gloss, Mascara, Eye shadow, Lush, Brows, and Eyeliner. This can be done, e.g., by identifying the timing that the demonstrator clicked on the respective button 508. The start of each of these sub-sessions is marked in the stored session video, e.g., with metadata, and icons are generated, each icon having a link to a corresponding mark. This will allow to mark each sub-session and for the user to jump or skip to particular sessions of interest”; Saban: [0078] – “option to have voice recording during session, which is good for tutorial; the ability to fast forward the recordings; integration with virtual make up, also post processing; integrations with virtual glasses and other accessories; screen at any size includes tablets and phone, the classic memory mirror; video can be stored and edited with and without distortion correction; stream sessions to friends family or forums Facebook rooms etc.”; Saban: [0079] – “Both sales associate and customer are using a digital mirror. The sales associate can demonstrate applying make-up remotely to the customer and save the session. In this case, the image captured by the camera of the mirror positioned at the customer location is projected on both the customer's screen and the sales associate screen. The sales associate then uses a user interface to apply make-up to the image projected on the mirror at the sales associate's location. The input from the user interface of the sales associate are used to modify the image projected on both screens, i.e., modify the image on the sales associate's screen and transmitted to the customer's location to modify the image displayed on the customer's mirror”).
Regarding Claim 20: Claim 20 recites substantially similar limitations as claim 1. Therefore, claim 20 is rejected under the same rationale as claim 1 above.
Claims 3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Varady (US 11,366,343 B2).
Regarding Claim 3: The combination of Saban and Mao discloses the limitations of claim 1 above.
The combination does not explicitly teach wherein the mirror device is further configured to detect a nose type and a corresponding impact indicator of the detected nose type on selection of the one or more lifestyle products, and wherein the plurality of facial attributes further comprises the detected nose type and the corresponding impact indicator. Notably, however, Saban does disclose identifying a variety of facial attributes (Saban: [0087]).
To that accord, Varady does teach wherein the mirror device is further configured to detect a nose type and a corresponding impact indicator of the detected nose type on selection of the one or more lifestyle products, and wherein the plurality of facial attributes further comprises the detected nose type and the corresponding impact indicator. (Varady: col. 17, ln. 57- col. 18, ln. 9 – “precisely measure the size and shape of a customer's nose, and then custom fit eyewear (lenses and frames) to perfectly fit that anatomy. Optimum comfort of an eyewear's nose pads positioned on a customer's nose may be achieved if the two contact surfaces are aligned properly and mate such that there are no high pressure-points and if the eyewear is naturally supported in the proper position by the nose. Each customer may have a unique preference as to where on the nose they prefer to wear eyewear for maximum comfort, aesthetic, or utility. Understanding the quantitative anatomy of the nose may not only allow the frame of a customized piece of eyewear to sit precisely on the nose where desired with maximum comfort, aesthetic, and utility, but also allow a customer immediate clarity and comfort in viewing objects for different tasks, according to their habits. For instance, the distance between nose pads of the eyewear may be related to the location of the lenses relative to a customer's pupils”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the detecting a nose type and impact indicator as taught by Varady. One of ordinary skill in the art would have been motivated to do so in order to find the perfect item for their face anatomy (col. 1, ln. 23-25).
Regarding Claim 7: The combination of Saban and Mao discloses the limitations of claim 1 above.
The combination does not explicitly teach an eyewear recommendation system configured to recommend a three-dimensional (3D) eyewear frame to the user based on two or more of: an emotional index of the user, gaze metrics of the user, a facial jawline of the user, an eye shape, a type of nose of the user, a context communicated by the user, and an intent and preferences of the user, wherein the recommended 3D eyewear frame is provided to the user in physical form. Notably, however, Saban does disclose where eyewear is among the items that can be seen with the mirror device (Saban: [0070]), and determining eyes shape of the user (Saban: [0087]).
To that accord, Varady does teach another of an emotional index of the user, gaze metrics of the user, a facial jawline of the user, an eye shape, a type of nose of the user, a context communicated by the user, and an intent and preferences of the user, wherein the recommended 3D eyewear frame is provided to the user in physical form. (Varady: col. 40, ln. 6-12 – “Some examples of feature of the face used for frame fit filters or recommendations are the shape of nose, ear locations, and cheek detection; for fit filters that take into account Rx considerations, such as if eyes are far from center, may affect overall lens thickness and weight and thus trigger certain fit rules”; Varady: col. 9, ln. 19-25 – “the parametric model 220 may further be adapted so that the lens dimensions fit optical corrections and/or preferences of a user. In one scenario, in addition to the lenses of the parametric model 220 modeling bifocal or progressive multifocal lenses, the placement of the various lens powers of the lenses may vary based on the user's preferences and use of the customized eyewear”; Varady: col. 21, ln. 29-31 – “enabling delivery-to-the-home of a “pre-adjusted” frame that fits the customer perfectly out-of-the-box”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the nose type and preferences and providing a physical form as taught by Varady. One of ordinary skill in the art would have been motivated to do so in order to find the perfect item for their face anatomy (col. 1, ln. 23-25).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Choe (US 20230214904 A1).
Regarding Claim 4: The combination of Saban and Mao discloses the limitations of claim 1 above.
The combination does not explicitly disclose wherein the plurality of environmental parameters comprises one or more of: a weather in a geographical location of the mirror device, a pollution level, an Ultraviolet (UV) index, a time of the data, and a humidity level of the geographical location. Notably, however, Saban does disclose tracking a GPS location and orientation of the user device (Saban: [0042]).
To that accord, Choe does teach wherein the plurality of environmental parameters comprises one or more of: a weather in a geographical location of the mirror device, a pollution level, an Ultraviolet (UV) index, a time of the data, and a humidity level of the geographical location. Examiner notes that Applicant recites at least one of in the claim. (Cho: [0064] – “the cosmetic determination unit 105 may determine cosmetics for a user in consideration of the skin characteristics of the user, an environment similar to that of the user and other users belonging to the similar environment. The cosmetic determination unit 105 may determine the cosmetics based on climate information, such as a monthly average temperature, a monthly maximum temperature, a monthly minimum temperature, and a monthly average humidity, on the smooth distribution area obtained from the Meteorological Administration server”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the environmental parameters of weather and humidity as taught by Choe. One of ordinary skill in the art would have been motivated to do so in order to also take into consideration cosmetic use of the items (Choe: [0064]).
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Rose (US 20170287044 A1).
Regarding Claim 8: The combination of Mao and Johnson discloses the limitations of claim 1 above.
Saban further discloses a dynamic product formulation system configured to formulate in a real time or near real time a customized product. Saban discloses determining a look for a user based on their identified features (Saban: [0087]).
The combination does not explicitly disclose dispense out the formulated customized product for the user based on an analysis of a plurality of physiological parameters and an activity level of the user. Notably, however, Saban does disclose determining items for the user based on their facial features (Saban: [0087]) and Johnson does disclose dispensing items to the user (Johnson: [0101-0102]).
To that accord, Rose does teach based on an analysis of a plurality of physiological parameters and an activity level of the user. (Rose: [0086] – “data relating to the user's physical activity data, nutrition, and/or social networking may be obtained from e.g., one or more user data servers 102, user devices 110, customer database 158, and/or other entities configured to store such information (e.g., the previously referenced databases described in co-owned, co-pending U.S. patent application Ser. No. 15/002,036). The data relating to the user's physical activity, nutrition, and/or social networking is displayed to the curator (not shown). In this manner, the curator may review the user's most recent patterns of behavior in order to make determinations relating to the curated items. For example, the curator may identify that the user's activity of running generally takes place in the evenings, which are generally colder than the average temperatures for a geographic region which are used by the recommendation engine”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the activity level of the user as taught by Rose. One of ordinary skill in the art would have been motivated to do so in order to take into account patterns of behavior to select items suited to their activity (Rose: [0086]).
Regarding Claim 9: The combination of Saban and Mao in view of Rose discloses the limitations of claim 8 above.
Saban further discloses the dynamic product formulation system is equipped with a sensing device to measure the plurality of physiological parameters, wherein the plurality of physiological parameters comprises a skin tone, a skin color, and a skin temperature. (Saban: [0073] – “the lighting condition and the skin complexion can be analyzed by the controller, and the controller may then apply different activation signals to the various temperature LEDs 525 so as to provide appropriate illumination on the user's face”; Saban: [0036] – “recognition of a user in front of the mirror, face recognition, recognition of user gesture commands, item recognition, distance measurement, user body measurements/assessments (including, e.g., height, age, weight, ethnic group, sex and the like) and calculation of user theoretical point of view in theoretical mirror. In the embodiments relating to makeup sessions, the trigger event module can be configured to identify the user's skin color and complexion”).
The combination does not explicitly teach the activity of the user, wherein the plurality of physiological parameters comprises a heart rate, and an emotion index. Notably, however, Saban does disclose identifying various features of the user (Saban: [0087]).
To that accord, Rose does teach the activity of the user, wherein the plurality of physiological parameters comprises a heart rate, and an emotion index. (Rose: [0086] – “The data relating to the user's physical activity, nutrition, and/or social networking is displayed to the curator (not shown). In this manner, the curator may review the user's most recent patterns of behavior in order to make determinations relating to the curated items”; Rose: [0036] – “the user device 110 is in communication with a plurality of health monitoring devices (not shown) and/or the user device 110 may further comprise a health monitoring device. Heath monitoring devices comprise portable computing devices designed to measure, sense, monitor, or otherwise receive biometric, environmental, and/or activity parameters. In one variant, the health monitoring devices may comprise wearable health-related parameter measurement and computing devices, such as e.g., a smart watch, an activity tracker, a heart rate monitor, a sleep tracking device, a nutrition tracking device, a smart scale, and/or smart eyeglasses. In addition, user device 110 may comprise a smart phone having one or more of the foregoing capabilities and/or which enables user entry of the foregoing health parameters; Rose: [0204] – “the recommendation system may further incorporate coaching features, such as via the curator, computer program, or other trained professional. In one embodiment, coaching is provided as a recommended item at discounted rates and/or one or more free sessions may be included among recommended items. Such coaching may include emotional support, digital coaching, and/or nutrition coaching”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the activity level of the user, heart rate, and emotion as taught by Rose. One of ordinary skill in the art would have been motivated to do so in order to take into account patterns of behavior to select items suited to their activity (Rose: [0086]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Rose (US 20170287044 A1), and in further view of Gebre (US 20220207807 A1).
Regarding Claim 12: The combination of Saban and Mao discloses the limitations of claim 1 above.
Saban further discloses a hairstyle recommendation system configured to recommend a specific customer hairstyle to the user based on a skin tone and a type of face parameter of the user. (Saban: [0087] – “The look includes a complete makeup according to a specific style fetched from a library of styles programmed beforehand. In step 700 the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc. Using the determined features, at step 702 the system classifies the face features according to preprogrammed list of classifications. Optionally the system may use the facial features classifications to generate a unitary facial classification. In step 704 the system search for match of the classification of the user to pre-stored facial classifications having associated virtual makeup. The system then selects a best match look”; Saban: [0025] – “Embodiments of the invention involve both hardware and software designs that are particularly tailored for use as a close-proximity mirror, i.e., situation wherein the user observes its own face, such as for applying makeup, hair styling, or glasses fitment”).
The combination does not explicitly teach based on an emotional index of the user, a forehead length, a neck length, an eye color, a face size parameter. Notably, however, Saban does disclose identifying facial features of the user in determining items or a hair styling (Saban: [0087]; [0025]).
To that accord, Rose does teach based on an emotional index of the user and an eye color. (Rose: [0204] - “the recommendation system may further incorporate coaching features, such as via the curator, computer program, or other trained professional. In one embodiment, coaching is provided as a recommended item at discounted rates and/or one or more free sessions may be included among recommended items. Such coaching may include emotional support, digital coaching, and/or nutrition coaching”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the emotion and eye color as taught by Rose. One of ordinary skill in the art would have been motivated to do so in order to take into account patterns of behavior to select items suited to their activity (Rose: [0086]).
The combination in view of Rose does not explicitly teach based on a forehead length, a neck length, and a face size parameter of the user. Notably, however, Saban does disclose identifying facial features of the user in determining items or a hair styling (Saban: [0087]; [0025]).
To that accord, Gebre does teach based on a forehead length, a neck length, and a face size parameter of the user. (Gebre: [0078] – “The method may find a subject who has similar characteristics (e.g. face shape, chin shape or beard or hairstyle) to the subject and recommend a hairstyle to the subject based on those characteristics”).
It would have been obvious o one of ordinary skill in the art, before the effective filing date of the claimed claimed invention, to modify the invention of the combination of Saban and Mao in view of Rose disclosing the system for determining items and displaying them on a user with a mirror device with the various features of the head as taught by Gebre. One ordinary skill in the art would have been motivated to do so in order to view a realistic representation of their head with the modified hairstyle (Gebre: [0003]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Rose (US 20170287044 A1), and in further view of Levin (US 20230334552 A1).
Regarding Claim 13: The combination of Saban and Mao discloses the limitations of claim 1 above.
Saban further discloses a makeup recommendation system configured to recommend at least one makeup product to the user based on a combined analysis of the plurality of facial attributes of the user, a plurality of ambient factors around the user. (Saban: [0087] – “The look includes a complete makeup according to a specific style fetched from a library of styles programmed beforehand. In step 700 the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc. Using the determined features, at step 702 the system classifies the face features according to preprogrammed list of classifications. Optionally the system may use the facial features classifications to generate a unitary facial classification. In step 704 the system search for match of the classification of the user to pre-stored facial classifications having associated virtual makeup. The system then selects a best match look”).
The combination does not explicitly teach based on a context communicated by the user, an intent and preferences of the user, an activity level of the user, and an allergic consideration of the user. Notably, however, Saban does disclose identifying facial features of the user in determining items or a hair styling (Saban: [0087]; [0025]).
To that accord, Rose does teach based on a context communicated by the user, an intent and preferences of the user, an activity level of the user. (Rose: [0086] – “data relating to the user's physical activity data, nutrition, and/or social networking may be obtained from e.g., one or more user data servers 102, user devices 110, customer database 158, and/or other entities configured to store such information (e.g., the previously referenced databases described in co-owned, co-pending U.S. patent application Ser. No. 15/002,036). The data relating to the user's physical activity, nutrition, and/or social networking is displayed to the curator (not shown). In this manner, the curator may review the user's most recent patterns of behavior in order to make determinations relating to the curated items. For example, the curator may identify that the user's activity of running generally takes place in the evenings, which are generally colder than the average temperatures for a geographic region which are used by the recommendation engine (noted elsewhere herein)”). In summary, the intent of what the user activities, when and how the user likes to do the activities (preferences), how active the user is (activity level), and various information regarding the activity, such as the temperature (context) is used to determine the recommendations.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the context, intent, preferences, and activity level as taught by Rose. One of ordinary skill in the art would have been motivated to do so in order to take into account patterns of behavior to select items suited to their activity (Rose: [0086]).
The combination in view of Rose does not explicitly teach based on an allergic consideration of the user. Notably, however, Saban does disclose identifying facial features of the user in determining items or a hair styling (Saban: [0087]; [0025]).
To that accord, Levin does teach based on an allergic consideration of the user. (Levin: [0047] – “Avoid list ingredients represent ingredients that will increase that user's risk, are counterintuitive or counterproductive to user goals, allergies, or concerns, and/or may exacerbate a disease or signs/symptoms of the disease. These personalized ingredient lists 104 then provide input to a product filter computer application 106 which, in turn, provides personalized product recommendations 108 of specific topical products matching results of the personalized ingredient lists”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao in view of Rose disclosing the system for determining items and displaying them on a user with a mirror device with the allergic consideration as taught by Levin. One of ordinary skill in the art would have been motivated to do so in order to take into consideration medical conditions of the user (Levin: [0003]).
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Levin (US 20230334552 A1).
Regarding Claim 14: The combination of Saban and Mao discloses the limitations of claim 1 above.
Saban further discloses a skincare recommendation system configured to recommend at least one skincare product to the user based on:
an analysis of an image of the facial portion of the user captured at different wavelengths using a multisensory device; (Saban: [0087] – “The look includes a complete makeup according to a specific style fetched from a library of styles programmed beforehand. In step 700 the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc. Using the determined features, at step 702 the system classifies the face features according to preprogrammed list of classifications. Optionally the system may use the facial features classifications to generate a unitary facial classification. In step 704 the system search for match of the classification of the user to pre-stored facial classifications having associated virtual makeup. The system then selects a best match look; Saban: [0050] – “Multiple techniques can be used to optimize the object mask/boundaries decision… Mask from other sensors like IR, microwave, depth, ultrasound, ultra band and the like”).
a plurality of ambient factors around the user wherein, the plurality of ambient factors comprises two or more of: a light illumination level, a humidity level, a spatial light distribution, and a temperature in surroundings of the user. Examiner notes that Applicant recites at least two of in the claim. (Saban: [0071] – “the illumination device 515 comprises a light diffuser 520 and a plurality of LEDs 525 of at least two different temperatures. The LEDs 525 are coupled to a controller that controls the intensity of each LED according to the desired light temperature”).
The combination does not explicitly disclose based on:
one or more visible and invisible skin attributes of the user;
a plurality of interactive user inputs, wherein the plurality of interactive user inputs comprises two or more of: genetic information of the user, a medical history of the user, a water intake amount of the user, a diet of the user, a stress level of the user, and a sleep quality indicator of the user;
a presence of one or more active ingredients in one or more skincare products;
Notably, however, Notably, however, Saban does disclose identifying various facial/skin features of the user (Saban: [0087]).
To that accord, Levin does teach based on:
one or more visible and invisible skin attributes of the user; Levin teaches taking into account history, habits, risk factors, sux exposure, type hydration, and sensitivity of the skin (Levin: [0089]).
a plurality of interactive user inputs, wherein the plurality of interactive user inputs comprises two or more of: genetic information of the user, a medical history of the user, a water intake amount of the user, a diet of the user, a stress level of the user, and a sleep quality indicator of the user; Examiner notes that Applicant recites two or more in the claim. (Levin: [0041] – “The method may comprise (a) receiving information about a user, wherein the information comprises (i) genetic data of the user, (ii) user responses to a health and profile survey, and (iii) user inputs corresponding to one or more ingredients to avoid, wellness concerns, allergies, and/or lifestyles; (b) using a user analysis algorithm to generate one or more user attributes based on the information about the user; (c) correlating the one or more user attributes to one or more ingredient effects (some of which may be location dependent) associated with one or more reference ingredients; and (d) using the correlations between the one or more user attributes and the one or more ingredient effects to generate (i) a preliminary ingredient avoid list. In some embodiments, the health/profile survey may be used to gather information on a user's age, ethnicity, lifestyle, skin/hair/nail concerns, skin/hair/nail goals, skin/hair/nail type, complete health history, reproductive history and goals, allergies (e.g., environmental, food, drug, and/or skin allergies), genetic concerns, individual risk factors, and wellness and/or product ingredient concerns”).
a presence of one or more active ingredients in one or more skincare products; (Levin: [0060] – “the user attribute profile block 156 can be programmed to limit the recommended avoidance of products containing an unfavorable ingredient for glaucoma to those products that, when administered as directed, could possibly cause the purported unfavorable effect. That is, the user attribute block 156 can use the glaucoma attribute to recommend avoiding use of an eye cream or any product used around the eye containing the unfavorable ingredient but not necessarily avoiding use of a foot cream containing the unfavorable ingredient. Another example is that certain ingredients may only cause harm when formulated with other ingredients in the same product or when a user is using more than one product in the same area”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the skin attributes, medical history and genetic information, and ingredients as taught by Levin. One of ordinary skill in the art would have been motivated to do so in order to take into consideration medical conditions of the user (Levin: [0003]).
Regarding Claim 15: The combination of Saban and Mao in view of Levin discloses the limitations of claim 14 above.
Saban further discloses wherein the analysis of the image of the facial portion of the user captured at different wavelengths further comprises calibrating and correcting any distortion present in the captured image in real time and further analyse the one or more visible and invisible skin attributes of the user. (Saban: [0087] – “The look includes a complete makeup according to a specific style fetched from a library of styles programmed beforehand. In step 700 the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc. Using the determined features, at step 702 the system classifies the face features according to preprogrammed list of classifications. Optionally the system may use the facial features classifications to generate a unitary facial classification. In step 704 the system search for match of the classification of the user to pre-stored facial classifications having associated virtual makeup. The system then selects a best match look; Saban: [0050] – “Multiple techniques can be used to optimize the object mask/boundaries decision… Mask from other sensors like IR, microwave, depth, ultrasound, ultra band and the like”).
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Kornilov (US 20190164210 A1).
Regarding Claim 16: The combination of Saban and Mao discloses the limitations of claim 1 above.
Saban further discloses analyse the plurality of facial attributes of the user and the plurality of environmental parameters and according recommend the one or more lifestyle products of the user. (Saban: [0087] – “The look includes a complete makeup according to a specific style fetched from a library of styles programmed beforehand. In step 700 the system identifies facial features of the user's image projected on the screen. The features may include, e.g., head shape, skin complexion, lips shape, cheekbones shape and/or height or pronouncement, eyes shape (e.g., deep-set, monolid, upturned, downturned, hooded, protruding, round, close set, wide set, or almond shaped), etc.”; Saban: [0080] – “a sensor detects the lighting environment about the face of the person receiving the demonstration. According to another embodiment, a processor analyzes the image from the camera to determine the lighting environment about the face of the person receiving the demonstration. Using the lighting information, and optionally the skin complexion of the person receiving the demonstration, the controller adjusts the light sources to provide lighting at a desired temperature”).
The combination does not explicitly teach wherein the mirror device further comprises a pretrained neural network module. Notably, however, Saban does disclose the identifying of the facial features and environment in recommending items (Saban: [0087]; [0093]).
To that accord, Kornilov does teach wherein the mirror device further comprises a pretrained neural network module. (Kornilov: [0067] – “deep neural networks may be used to find correlations between a user's face and purchases. Data about user faces, purchase history, and product information (such as product characteristics) is stored. The deep neural networks may be trained to determine combinations (relationships, correlations) of faces and product information and purchase history. In other words, a data model (deep neural network) may be constructed to describe which faces will purchase which products. Using the data model, recommendations may be made for users without a purchase history or with a short purchase history”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the use of a pretrained neural network as taught by Kornilov. One of ordinary skill in the art would have been motivated to do so in order to determine correlations between data without a purchase history (Kornilov: [0067]).
Regarding Claim 17: The combination of Saban and Mao in view of Kornilov discloses the limitations of claim 16 above.
The combination does not explicitly teach wherein the pretrained neural network module is trained using a dataset in combination with one or more of: an image segmentation operation, a color analysis operation, a pattern recognition operation, a skin analysis operation, and a facial recognition operation. Notably, however, Saban does disclose facial recognition, color segmentation, and pattern recognition (Saban: [0036]; [0042]).
To that accord, Kornilov does teach wherein the pretrained neural network module is trained using a dataset. (Kornilov: [0067] – “deep neural networks may be used to find correlations between a user's face and purchases. Data about user faces, purchase history, and product information (such as product characteristics) is stored. The deep neural networks may be trained to determine combinations (relationships, correlations) of faces and product information and purchase history. In other words, a data model (deep neural network) may be constructed to describe which faces will purchase which products. Using the data model, recommendations may be made for users without a purchase history or with a short purchase history”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao in view of Kornilov disclosing the system for determining items and displaying them on a user with a mirror device with the training of a pretrained neural network as taught by Kornilov. One of ordinary skill in the art would have been motivated to do so in order to determine correlations between data without a purchase history (Kornilov: [0067]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Mao (US 20180174125 A1) in view of Barron (US 20210390311 A1).
Regarding Claim 18: The combination of Saban and Mao discloses the limitations of claim 1 above.
Saban further discloses a specific tutorial item depending on the selection of a lifestyle product of the one or more lifestyle products. (Saban: [0075] – “the stored video is then automatically edited by breaking it into sub-sessions. The sub-sessions may include, for example: Foundations, Powder, Bronzer, Concealer, Lipstick, Lip gloss, Mascara, Eye shadow, Lush, Brows, and Eyeliner. This can be done, e.g., by identifying the timing that the demonstrator clicked on the respective button 508. The start of each of these sub-sessions is marked in the stored session video, e.g., with metadata, and icons are generated, each icon having a link to a corresponding mark. This will allow to mark each sub-session and for the user to jump or skip to particular sessions of interest”; Saban: [0078] – “option to have voice recording during session, which is good for tutorial; the ability to fast forward the recordings; integration with virtual make up, also post processing; integrations with virtual glasses and other accessories; screen at any size includes tablets and phone, the classic memory mirror; video can be stored and edited with and without distortion correction; stream sessions to friends family or forums Facebook rooms etc.”; Saban: [0079] – “Both sales associate and customer are using a digital mirror. The sales associate can demonstrate applying make-up remotely to the customer and save the session. In this case, the image captured by the camera of the mirror positioned at the customer location is projected on both the customer's screen and the sales associate screen. The sales associate then uses a user interface to apply make-up to the image projected on the mirror at the sales associate's location. The input from the user interface of the sales associate are used to modify the image projected on both screens, i.e., modify the image on the sales associate's screen and transmitted to the customer's location to modify the image displayed on the customer's mirror”).
The combination does not disclose wherein the mirror device further comprises a Virtual Reality (VR) headset. Notably, however, Saban does disclose a digital mirror device (Saban: [0070]; see also: [0010]; [0030]; Fig. 5, #500) that supports augmented reality (Saban: [0027]).
To that accord, Barron does teach wherein the mirror device further comprises a Virtual Reality (VR) headset. (Barron: [0050] – “An application enables a user (e.g., presenter) to edit and prepare a beauty product augmented reality (AR) tutorial for a user of the AR tutorial. The presenter may create an AR tutorial with multiple steps where different beauty products are used in different steps. The application detects when a presenter applies a beauty product to a body part of the presenter and identifies the beauty product”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the use of a VR headset as taught by Barron. One of ordinary skill in the art would have been motivated to do so in order to detect products and their effect should be used on a use (Barron: [0051]).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable by the combination of Saban (US 20180278879 A1) and Johnson (US 20220148272 A1) in view of Kaufman (US 20210365915 A1).
Regarding Claim 19: The combination of Saban and Mao discloses the limitations of claim 1 above.
The combination does not explicitly disclose wherein the interactive system is connected to a cloud server for real-time data tracking of the user and replenishment tracking and facilitation of the one or more lifestyle products in the dispensing system. Notably, however, Saban does disclose the system being run through the cloud (Saban: [0055]), and Johnson does teach launching a product mobile dispensary delivery to deliver the products to the location of the user (Johnson: [0101-0102]).
To that accord, Kaufman does teach wherein the interactive system is connected to a cloud server for real-time data tracking of the user and replenishment tracking and facilitation of the one or more lifestyle products in the dispensing system. (Kaufman: [0040] – “Preferably computer system 70 and inventory-databank 92 work together to enable product inventory data at the site location of dispenser 20 to be decremented and thus tracked in realtime as products are sold and dispensed. Further, the contents displayed on menu 42 can be dynamically modified at any time by computer system 70. For example, if there is no more inventory for a specific product or product size, menu 42 will stop displaying information relating to such product until product inventory within dispenser 20 is replenished”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of the combination of Saban and Mao disclosing the system for determining items and displaying them on a user with a mirror device with the data tracking and replenishment tracking and facilitation of the products as taught by Kaufman. One of ordinary skill in the art would have been motivated to do so in order to manage inventory and display appropriate information regarding the product (Kaufman: [0008]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PTO-892 Reference U discloses a skin car device that can blend and dispense a personalized skin care product for a user with an attached mirror to the device.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY J KANG whose telephone number is (571)272-8069. The examiner can normally be reached Monday - Friday: 7:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Maria-Teresa Thein can be reached at 571-272-6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.J.K./ Examiner, Art Unit 3689
/VICTORIA E. FRUNZI/ Primary Examiner, Art Unit 3689 3/27/2026