Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-8, 16-18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff (WO 2023056333 A1), hereinafter Kosecoff, in view of Kitaguchi et al.: "Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis", Jama Network, Submitted 13 August 2021 [retrieved on 7-17-2026]. Retrieved from the internet <https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2782991>, hereinafter Kitaguchi.
Regarding claim 1, Kosecoff teaches A system comprising: one or more sensors configured to capture, in real-time, one or more images of a face of a user; (Abstract see "Systems, devices, and methods for generating, sharing, and presenting augmented reality cosmetic design filters." Para. 6 see "The biological surface can be a face." Para. 29 see "The depth sensor 143 can capture one or more images of the biological surface 180, including, but not limited to, images of a target body surface 181 of the biological surface 180. In the illustration provided in FIG. 1, the biological surface 180 is a human using the example system 100 to create the cosmetic design 110 and the target body surface 181 is the face of the human in the region around the eye and eyebrow." Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques"). one or more processors; and one or more memories storing non-transitory computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: (Para. 79 see "The one or more processors 715 are configured to execute computer-executable instructions stored on the computer-readable medium 730. In an embodiment, the processor(s) 715 are configured to receive and transmit signals to and/or from the components 720 via a communication bus or other circuitry, for example, as part of executing the client application 740."). capture, using the one or more sensors, the one or more images of the face of the user; (Para. 6 see "The biological surface can be a face." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected" Para. 29 see "The depth sensor 143 can capture one or more images of the biological surface 180, including, but not limited to, images of a target body surface 181 of the biological surface 180." Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140"). analyze the one or more images of the face of the user as the user applies one or more cosmetic products to at least one facial feature of the user to identify techniques used by the user; (Para. 38 see "At operation 203, the computer system identifies one or more applications 215 of cosmetic formulation(s) 220 to the biological surface 180. The cosmetic formulation(s) 220 can include, but are not limited to, makeup, mascara, lipstick, lipliner, eyeliner, glitter, aesthetic creams, ointments, or other materials that can be applied topically to the biological surface 180. The application(s) 215 of the cosmetic formulation(s) 220 can be detected by the camera 140 using radiation sensors, such as visible light sensors, invisible EM radiation sensors, or the like. In this way, the application(s) 215 can be detected as a shift in the biological surface relative to the baseline description, for example, in terms of coloration, surface reflectivity, or absorbance/reflectance of invisible EM radiation." Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces."). cosmetic applications, (Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces."). generate in real-time, based on (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "The filters can reproduce step-by-step cosmetic routines, including augmented reality renderings of specific tools, application traces, and finished effects in a manner that is sensitive to shape type and skin tone of the end user of the augmented reality filters. ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape" Para. 39 see "The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Examiner Note: The guidance or 'trace' is generated based on how the user applies the cosmetic formulation.). an overlay projection of cosmetic product application guidance associated with the one or more images of the face of the user; and using an augmented reality module, display the overlay projection of cosmetic product application guidance onto one of: the one or more images of the face of the user, a reflection of the face of the user as it appears in a mirror in real-time, or the face of the user in real-time. (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Para. 86 see "the user interface 745 includes interactive functionality including but not limited to graphical guides or prompts, presented via the display to assist a user in selecting cosmetic designs, tutorial videos, or animations. In some embodiments, the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.)").
While Kosecoff teaches evaluating the techniques used by the user, Kosecoff does not teach using a precision detection machine learning model wherein the precision detection machine learning model is trained using training data of previous cosmetic applications and wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; the determined level of precision determined by the precision detection machine learning model,.
However, Kitaguchi teaches evaluate the techniques used by the user using a precision detection machine learning model (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon.). wherein the precision detection machine learning model is trained using training data of previous steps and procedures (Pg. 1 Section Design, Setting, And Particpants, "This prognostic study used surgical videos acquired prior to 2017." Pg. 1, Section Key Points, Sub-Section Findings see "Findings In this prognostic study, 1480 video clips from 74 laparoscopic colorectal surgical videos with reliable skill assessment scores were extracted and assigned 4:1 to training and test sets. The proposed model automatically classified video clips into 3 different score groups with 75% accuracy."). and wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers.). the determined level of precision determined by the precision detection machine learning model, (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff to incorporate the teachings of Kitaguchi to train a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 2, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches wherein the non-transitory computer-readable instructions, when executed by the one or more processors, further cause the one or more processors to provide relevant content to the user based on user precision (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Para. 86 see "the user interface 745 includes interactive functionality including but not limited to graphical guides or prompts, presented via the display to assist a user in selecting cosmetic designs, tutorial videos, or animations. In some embodiments, the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.)" Examiner Note: Visuals guides are provided to the user. The guidance or 'trace' is generated based on how the user applies the cosmetic formulation.).
Kosecoff does not teach the determined level of precision.
However, Kitaguchi teaches the determined level of precision. (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Kitaguchi to use a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 4, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches cosmetic applications (Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces.").
While Kosecoff teaches user precision, Kosecoff does not teach wherein the training data of previous applications includes one or more images of users performing historically identified techniques and historical levels of precision associated with each historically identified technique.
However, Kitaguchi teaches wherein the training data of previous steps and procedures includes one or more images of users performing historically identified techniques and historical levels of precision associated with each historically identified technique. (Pg. 1 Section Design, Setting, And Particpants, "This prognostic study used surgical videos acquired prior to 2017." Pg. 1, Section Key Points, Sub-Section Findings see "Findings In this prognostic study, 1480 video clips from 74 laparoscopic colorectal surgical videos with reliable skill assessment scores were extracted and assigned 4:1 to training and test sets. The proposed model automatically classified video clips into 3 different score groups with 75% accuracy." Examiner Note: The model is trained to assess surgical skill and therefore identifies the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Kitaguchi to train a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 5, Kosecoff in view of Kitaguchi teaches The system of claim 1.
Kosecoff does not teach wherein the determined level of precision includes a range from a beginner level to an expert level.
However, Kitaguchi teaches wherein the determined level of precision includes a range from a beginner level to an expert level. (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Pg. 1, Section Key Points, Sub-Section Findings see "The proposed model automatically classified video clips into 3 different score groups with 75% accuracy." Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers; Less than average, average, above average (Beginner, Intermediate, Expert).).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Kitaguchi to determine a level of precision in a range from beginner to expert. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 6, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches at least one of a set of criteria including: symmetry and balance, wherein the precision detection machine learning model analyzes sensor data to determine an evenness of an application as the user applies the one or more cosmetic products, blending and color matching, wherein the precision detection machine learning model evaluates a smoothness of a transition between at least one color and at least one texture as the user applies the one or more cosmetic products, precision of lines and edges, wherein the precision detection machine learning model analyzes sensor data to determine an accuracy and a sharpness of lines and edges as the user applies the one or more cosmetic products, or overall aesthetic appeal, a harmony and an overall visual impact of a completed cosmetic application, wherein the training data includes indications of measurements and evaluations associated with historically identified techniques. (Para. 22 see "The filters can reproduce step-by-step cosmetic routines, including augmented reality renderings of specific tools, application traces, and finished effects in a manner that is sensitive to shape type and skin tone of the end user of the augmented reality filters." Para. 38 see "At operation 203, the computer system identifies one or more applications 215 of cosmetic formulation(s) 220 to the biological surface 180. The cosmetic formulation(s) 220 can include, but are not limited to, makeup, mascara, lipstick, lipliner, eyeliner, glitter, aesthetic creams, ointments, or other materials that can be applied topically to the biological surface 180. The application(s) 215 of the cosmetic formulation(s) 220 can be detected by the camera 140 using radiation sensors, such as visible light sensors, invisible EM radiation sensors, or the like. In this way, the application(s) 215 can be detected as a shift in the biological surface relative to the baseline description, for example, in terms of coloration, surface reflectivity, or absorbance/reflectance of invisible EM radiation." Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Examiner Note: The computer system detects how the user applies the cosmetic formulation and can reproduce finished effects in a manner that is sensitive to face shape type. Additionally, the detected motion includes information about the overall cosmetic design / aesthetic effect.).
Kosecoff does not teach wherein the level of precision determined by the precision detection machine learning model is based on wherein the precision detection machine learning model evaluates.
However, Kitaguchi teaches wherein the level of precision determined by the precision detection machine learning model is based on (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.). wherein the precision detection machine learning model evaluates (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Kitaguchi to use a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 7, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches wherein the non-transitory computer-readable instructions, when executed by the one or more processors, further cause the one or more processors to: after displaying the overlay projection of cosmetic product application guidance, capture, using the one or more sensors, one or more new images of the face of the user; (Para. 6 see "The biological surface can be a face." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone" Para. 24 see "the cosmetic design 110 can be deployed to a user device 190 as an animated augmented reality filter reproducing, step-by-step, the lay er-wise application of the cosmetic design 110." Para. 29 see "The depth sensor 143 can capture one or more images of the biological surface 180, including, but not limited to, images of a target body surface 181 of the biological surface 180." Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140" Para. 54 see "the computer system presents the filters 325 integrated with images 335 of the user surface 315. Presenting the filters 325 can include displaying the filters 325 through a screen 337. For example, the user device 190 can be a smartphone or a tablet including a touchscreen display. The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Examiner Note: Images are continuously taken, the filter and guidance are mapped onto the image and virtually displayed / projected in augmented reality.). analyze the one or more new images of the face of the user as the user applies the one or more cosmetic products to the at least one facial feature of the user to identify new techniques used by the user; (Para. 38 see "At operation 203, the computer system identifies one or more applications 215 of cosmetic formulation(s) 220 to the biological surface 180. The cosmetic formulation(s) 220 can include, but are not limited to, makeup, mascara, lipstick, lipliner, eyeliner, glitter, aesthetic creams, ointments, or other materials that can be applied topically to the biological surface 180. The application(s) 215 of the cosmetic formulation(s) 220 can be detected by the camera 140 using radiation sensors, such as visible light sensors, invisible EM radiation sensors, or the like. In this way, the application(s) 215 can be detected as a shift in the biological surface relative to the baseline description, for example, in terms of coloration, surface reflectivity, or absorbance/reflectance of invisible EM radiation." Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Examiner Note: Images are continuously taken, the computer system continuously detects and uses the motion of the applicator during operations to generate traces.). new techniques and new level of (precision) (Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Examiner Note: Images are continuously taken, the computer system continuously detects and uses the motion of the applicator during operations to generate traces and update the traces on the display. Kitaguchi teaches precision as cited.). generate in real-time, based on (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "The filters can reproduce step-by-step cosmetic routines, including augmented reality renderings of specific tools, application traces, and finished effects in a manner that is sensitive to shape type and skin tone of the end user of the augmented reality filters. ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape" Para. 39 see "The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Examiner Note: The guidance or 'trace' is generated based on how the user applies the cosmetic formulation.). a new overlay projection of cosmetic product application guidance associated with the one or more new images of the face of the user; and using the augmented reality module, display the new overlay projection of cosmetic product application guidance onto one of: the one or more new images of the face of the user, the reflection of the face of the user as it appears in the mirror in real-time, or the face of the user in real-time. (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Para. 86 see "the user interface 745 includes interactive functionality including but not limited to graphical guides or prompts, presented via the display to assist a user in selecting cosmetic designs, tutorial videos, or animations. In some embodiments, the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.)" Examiner Note: New overlay projections are continuously updated and displayed based on images continuously processed by the computer system.).
Kosecoff does not teach evaluate the techniques used by the user using the precision detection machine learning model, wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; the level of precision determined by the precision detection machine learning model,.
However, Kitaguchi teaches evaluate the techniques used by the user using the precision detection machine learning model, wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.). the level of precision determined by the precision detection machine learning model, (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Kitaguchi to continuously use a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 8, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches wherein the one or more sensors, one or more processors, and one or more memories are part of a smart device and wherein the smart device includes a display screen. (Para. 8 see "In some embodiments, the one or more non-transitory computer memory devices are electronically coupled with a smart phone." Para. 28 see "the camera 140 can be or include multiple sensors and/or sources including, but not limited to a visible light image sensor 141, a depth sensor 143 and/or a source of invisible EM radiation 145, including but not limited to infrared or nearinfrared radiation. ... the client computing device 130 can incorporate the camera 140 (e.g., as a smartphone or tablet)" Para. 35 see "the operations can be or include operations performed by one or more processors of a computer system, such as client computing device(s) 130 of FIG. 1, in response to execution of computer-readable instructions stored on non-transitory memory of the computer system." Para. 83 see "The components of the client computing device 130 can be adapted to the application or can be specific to the application (e.g., ASICs). For example, the components 720 can include one or more cameras 721, a display 723, one or more radiation sources 725, and/or one or more radiation sensors 727, as described in more detail in reference to FIG 1.").
Claim 16 is rejected under the same analysis as claim 1 above.
Claim 17 is rejected under the same analysis as claim 7 above.
Claim 18 is rejected under the same analysis as claim 2 above.
Claim 20 is rejected under the same analysis as claim 4 above.
Claims 3, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff (WO 2023056333 A1), hereinafter Kosecoff, in view of Kitaguchi et al.: "Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis", Jama Network, Submitted 13 August 2021 [retrieved on 7-17-2026]. Retrieved from the internet <https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2782991>, hereinafter Kitaguchi, and Breazeal (US 20160199977 A1), hereinafter Breazeal.
Regarding claim 3, Kosecoff in view of Kitaguchi teaches The system of claim 1.
In addition, Kosecoff teaches wherein the one or more sensors include one or more sensors of a camera or smart device, wherein the camera or smart device captures the one or more images of the face of the user (Para. 25 see "the client computing device 130 can incorporate the camera 140. Similarly, the example system 100 can include multiple client computing devices 130, where a first client computing device 130 is a mobile electronic device (e.g., a tablet, smartphone, or laptop) that is configured to host user interface elements and to connect to the server(s) 160 over the network(s) 170" Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140").
Kosecoff does not teach using high-resolution.
However, Breazeal teaches using high-resolution. (Para. 131 see "The stereo cameras are designed to stream video in continuous mode. In addition, the camera interface board may support a single RGB application camera for taking high resolution photos and video conference video quality.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Breazeal to use a high-resolution camera to detect images of the user. Doing so would predictably increase the accuracy of detecting actions performed by the user and evaluating their precision level by capturing higher detailed images than a low-resolution camera.
Claim 19 is rejected under the same analysis as claim 3 above.
Claims 9-10, 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff (WO 2023056333 A1), hereinafter Kosecoff, in view of Tamura et al. (US 20150366328 A1), hereinafter Tamura, and Kitaguchi et al.: "Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis", Jama Network, Submitted 13 August 2021 [retrieved on 7-17-2026]. Retrieved from the internet <https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2782991>, hereinafter Kitaguchi.
Regarding claim 9, Kosecoff teaches A system comprising: one or more sensors configured to capture, in real-time, one or more images of a face of a user; (Abstract see "Systems, devices, and methods for generating, sharing, and presenting augmented reality cosmetic design filters." Para. 6 see "The biological surface can be a face." Para. 29 see "The depth sensor 143 can capture one or more images of the biological surface 180, including, but not limited to, images of a target body surface 181 of the biological surface 180. In the illustration provided in FIG. 1, the biological surface 180 is a human using the example system 100 to create the cosmetic design 110 and the target body surface 181 is the face of the human in the region around the eye and eyebrow." Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques"). one or more processors; and one or more memories storing non-transitory computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: (Para. 79 see "The one or more processors 715 are configured to execute computer-executable instructions stored on the computer-readable medium 730. In an embodiment, the processor(s) 715 are configured to receive and transmit signals to and/or from the components 720 via a communication bus or other circuitry, for example, as part of executing the client application 740."). capture, using the one or more sensors, the one or more images of the face of the user; (Para. 6 see "The biological surface can be a face." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected" Para. 29 see "The depth sensor 143 can capture one or more images of the biological surface 180, including, but not limited to, images of a target body surface 181 of the biological surface 180." Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140"). cosmetic applications, (Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces."). generate in real-time, based on (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "The filters can reproduce step-by-step cosmetic routines, including augmented reality renderings of specific tools, application traces, and finished effects in a manner that is sensitive to shape type and skin tone of the end user of the augmented reality filters. ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape" Para. 39 see "The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Examiner Note: The guidance or 'trace' is generated based on how the user applies the cosmetic formulation.). an overlay projection of cosmetic product application guidance for achieving the user's desired makeup look; and using an augmented reality module, display the overlay projection of cosmetic product application guidance onto one of: the one or more images of the face of the user, a reflection of the face of the user as it appears in a mirror in real-time, or the face of the user in real-time. (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "the smartphone captures images and a surface mapping of the user’s face. Initial images are used to generate a baseline with which subsequent applications of cosmetics, such as foundation, highlight, filler, eyeshadow, blush, mascara, or the like, can be detected ... The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape, and to guide a viewer through a cosmetic routine in a way that is specific to the viewer’s face shape and skin tone" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Para. 86 see "the user interface 745 includes interactive functionality including but not limited to graphical guides or prompts, presented via the display to assist a user in selecting cosmetic designs, tutorial videos, or animations. In some embodiments, the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.)" Examiner note: the user selects a desired look and the system overlays a projection of the cosmetic routine to guide a user through the application.).
Kosecoff does not teach receive an image of a face depicting a user's desired makeup look; compare the image of the face depicting the user's desired makeup look to the one or more images of the face of the user, in order to identify techniques used by the user; evaluate the techniques used by the user using a precision detection machine learning model wherein the precision detection machine learning model is trained using training data of previous and wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; the determined level of precision determined by the precision detection machine learning model,.
However, Kitaguchi teaches evaluate the techniques used by the user using a precision detection machine learning model (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon.). wherein the precision detection machine learning model is trained using training data of previous steps and procedures (Pg. 1 Section Design, Setting, And Particpants, "This prognostic study used surgical videos acquired prior to 2017." Pg. 1, Section Key Points, Sub-Section Findings see "Findings In this prognostic study, 1480 video clips from 74 laparoscopic colorectal surgical videos with reliable skill assessment scores were extracted and assigned 4:1 to training and test sets. The proposed model automatically classified video clips into 3 different score groups with 75% accuracy."). and wherein the precision detection machine learning model determines a level of precision associated with the techniques used by the user; (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers.). the determined level of precision determined by the precision detection machine learning model, (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff to incorporate the teachings of Kitaguchi to train a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Furthermore, Tamura teaches receive an image of a face depicting a user's desired makeup look; (Para. 76 see "The target face input image unit 20 is used for inputting the target face image which the user desires to achieve prior to actually applying the make-up to the user."). compare the image of the face depicting the user's desired makeup look to the one or more images of the face of the user, in order to identify techniques used by the user; (Para. 73 see "The synthesized face image generation unit 12 applies various make-up techniques (or performs various simulations) on the face image of the user contained in the image captured by the camera unit 1, and thereby generates face images (synthesized face images). The similarity determination unit 22 determines the degree of similarity between each synthesized face image and the target face image, and extracts the particular make-up technique that was applied to produce the synthesized face image having the highest degree of similarity to the target face image. This make-up technique is displayed on the display unit 5." Para. 76 see "The target face input image unit 20 is used for inputting the target face image which the user desires to achieve prior to actually applying the make-up to the user." Para. 81 see "The make-up technique extraction unit 24 then selects the technique that achieves a synthesized face image which is closest to the target face image of all the make-up techniques according to the output of the Euclidian distance computation unit 29." Para. 88 see "The camera unit 10 captures an image containing the face of the user (step ST101).").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Kitaguchi to incorporate the teachings of Tamura to receive an image of a face depicting a user's desired makeup look and compare the images of the user's face to the image of the desired makeup look to identify techniques used by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user using an image as opposed to a verbal description.
Regarding claim 10, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
In addition, Kosecoff teaches wherein the non-transitory computer-readable instructions, when executed by the one or more processors, further cause the one or more processors to provide relevant content to the user based on user precision (Para. 5 see "The operations can include identifying an application of a cosmetic formulation to the biological surface using one or more of the radiation sensors. The operations can include generating a trace describing the application in reference to the baseline description. The operations can also include outputting the trace." Para. 22 see "The filter is available to different users for virtual projection through an augmented reality interface. The filter can be used to reproduce layer effects, to realistically map a cosmetic design onto a different face shape" Para. 54 see "The filters 325 can be mapped to the user surface 315 in real time or near real time using feature tracking or other augmented reality techniques" Para. 86 see "the user interface 745 includes interactive functionality including but not limited to graphical guides or prompts, presented via the display to assist a user in selecting cosmetic designs, tutorial videos, or animations. In some embodiments, the user interface 745 provides guidance (e.g., visual guides such as arrows or targets, progress indicators, audio/haptic feedback, synthesized speech, etc.)" Examiner Note: Visuals guides are provided to the user. The guidance or 'trace' is generated based on how the user applies the cosmetic formulation.).
Kosecoff does not teach the determined level of precision.
However, Kitaguchi teaches the determined level of precision. (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Tamura and Kitaguchi to incorporate the teachings of Kitaguchi to use a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 12, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
In addition, Kosecoff teaches cosmetic applications (Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces.").
Kosecoff does not teach wherein the training data of previous applications includes one or more images of users performing historically identified techniques and historical levels of precision associated with each historically identified technique.
However, Kitaguchi teaches wherein the training data of previous applications includes one or more images of users performing historically identified techniques and historical levels of precision associated with each historically identified technique. (Pg. 1 Section Design, Setting, And Particpants, "This prognostic study used surgical videos acquired prior to 2017." Pg. 1, Section Key Points, Sub-Section Findings see "Findings In this prognostic study, 1480 video clips from 74 laparoscopic colorectal surgical videos with reliable skill assessment scores were extracted and assigned 4:1 to training and test sets. The proposed model automatically classified video clips into 3 different score groups with 75% accuracy." Examiner Note: The model is trained to assess surgical skill and therefore identifies the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Tamura and Kitaguchi to incorporate the teachings of Kitaguchi to train a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 13, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
Kosecoff does not teach wherein the determined level of precision includes a range from a beginner level to an expert level.
However, Kitaguchi teaches wherein the determined level of precision includes a range from a beginner level to an expert level. (Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Pg. 1, Section Key Points, Sub-Section Findings see "The proposed model automatically classified video clips into 3 different score groups with 75% accuracy." Examiner Note: The level of precision demonstrated by the surgeon (user) is divided into three tiers; Less than average, average, above average (Beginner, Intermediate, Expert).).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Tamura and Kitaguchi to incorporate the teachings of Kitaguchi to determine a level of precision in a range from beginner to expert. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 14, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
In addition, Kosecoff teaches at least one of a set of criteria including: symmetry and balance, wherein the precision detection machine learning model analyzes sensor data to determine an evenness of an application as the user applies one or more cosmetic products, blending and color matching, wherein the precision detection machine learning model evaluates a smoothness of a transition between at least one color and at least one texture as the user applies the one or more cosmetic products, precision of lines and edges, wherein the precision detection machine learning model analyzes sensor data to determine an accuracy and a sharpness of lines and edges as the user applies the one or more cosmetic products, or overall aesthetic appeal, a harmony and an overall visual impact of a completed cosmetic application, wherein the training data includes indications of measurements and evaluations associated with historically identified techniques. (Para. 22 see "The filters can reproduce step-by-step cosmetic routines, including augmented reality renderings of specific tools, application traces, and finished effects in a manner that is sensitive to shape type and skin tone of the end user of the augmented reality filters." Para. 38 see "At operation 203, the computer system identifies one or more applications 215 of cosmetic formulation(s) 220 to the biological surface 180. The cosmetic formulation(s) 220 can include, but are not limited to, makeup, mascara, lipstick, lipliner, eyeliner, glitter, aesthetic creams, ointments, or other materials that can be applied topically to the biological surface 180. The application(s) 215 of the cosmetic formulation(s) 220 can be detected by the camera 140 using radiation sensors, such as visible light sensors, invisible EM radiation sensors, or the like. In this way, the application(s) 215 can be detected as a shift in the biological surface relative to the baseline description, for example, in terms of coloration, surface reflectivity, or absorbance/reflectance of invisible EM radiation." Para. 39 see "the computer system detects an applicator 225, such as a brush, sponge, pencil, finger, or other applicator 225 in proximity to the biological surface 180. ... The application 215 of cosmetic formulation 220 can include a motion 230 of the applicator 225 relative to the biological surface 180. The motion 230 can include meaningful information with respect to the aesthetic effect of the overall cosmetic design 110, such as in blending and smudging motions. As such, motions 230 of applicators 220 can be detected and used during operations for generating traces." Examiner Note: The computer system detects how the user applies the cosmetic formulation and can reproduce finished effects in a manner that is sensitive to face shape type. Additionally, the detected motion includes information about the overall cosmetic design / aesthetic effect.).
Kosecoff does not teach wherein the level of precision determined by the precision detection machine learning model is based on wherein the precision detection machine learning model evaluates.
However, Kitaguchi teaches wherein the level of precision determined by the precision detection machine learning model is based on (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.). wherein the precision detection machine learning model evaluates (Pg. 1, Section Objectives see "To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos." Pg. 6 Section Discussion see "The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%)" Examiner Note: The model assesses surgical skill and therefore evaluates the techniques used by the surgeon. The level of precision demonstrated by the surgeon (user) is divided into three tiers.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Tamura and Kitaguchi to incorporate the teachings of Kitaguchi to use a machine learning model to evaluate the techniques used by a user and determine a level of precision performed by the user. Doing so would predictably increase the accuracy of evaluating a user's needs thereby improving the user's experience by providing more helpful guidance specific to the user.
Regarding claim 15, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
In addition, Kosecoff teaches wherein the one or more sensors, one or more processors, and one or more memories are part of a smart device and wherein the smart device includes a display screen. (Para. 8 see "In some embodiments, the one or more non-transitory computer memory devices are electronically coupled with a smart phone." Para. 28 see "the camera 140 can be or include multiple sensors and/or sources including, but not limited to a visible light image sensor 141, a depth sensor 143 and/or a source of invisible EM radiation 145, including but not limited to infrared or nearinfrared radiation. ... the client computing device 130 can incorporate the camera 140 (e.g., as a smartphone or tablet)" Para. 35 see "the operations can be or include operations performed by one or more processors of a computer system, such as client computing device(s) 130 of FIG. 1, in response to execution of computer-readable instructions stored on non-transitory memory of the computer system." Para. 83 see "The components of the client computing device 130 can be adapted to the application or can be specific to the application (e.g., ASICs). For example, the components 720 can include one or more cameras 721, a display 723, one or more radiation sources 725, and/or one or more radiation sensors 727, as described in more detail in reference to FIG 1.").
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kosecoff (WO 2023056333 A1), hereinafter Kosecoff, in view of Tamura et al. (US 20150366328 A1), hereinafter Tamura, and Kitaguchi et al.: "Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis", Jama Network, Submitted 13 August 2021 [retrieved on 7-17-2026]. Retrieved from the internet <https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2782991>, hereinafter Kitaguchi, and Breazeal (US 20160199977 A1), hereinafter Breazeal.
Regarding claim 11, Kosecoff in view of Tamura and Kitaguchi teaches The system of claim 9.
In addition, Kosecoff teaches wherein the one or more sensors include one or more sensors of a camera or smart device, wherein the camera or smart device captures the one or more images of the face of the user (Para. 25 see "the client computing device 130 can incorporate the camera 140. Similarly, the example system 100 can include multiple client computing devices 130, where a first client computing device 130 is a mobile electronic device (e.g., a tablet, smartphone, or laptop) that is configured to host user interface elements and to connect to the server(s) 160 over the network(s) 170" Para. 37 see "the baseline description 210 includes generating one or more images of the biological surface 180, using the camera 140 or other radiation sensors that are configured to detect visible light or invisible EM radiation. Invisible EM radiation can be projected onto the biological surface 180 using a radiation source of the camera 140").
Kosecoff does not teach using high-resolution.
However, Breazeal teaches using high-resolution. (Para. 131 see "The stereo cameras are designed to stream video in continuous mode. In addition, the camera interface board may support a single RGB application camera for taking high resolution photos and video conference video quality.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kosecoff and Tamura and Kitaguchi to incorporate the teachings of Breazeal to use a high-resolution camera to detect images of the user. Doing so would predictably increase the accuracy of detecting actions performed by the user and evaluating their precision level by capturing higher detailed images than a low-resolution camera.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Troutman et al. (WO 2022146615 A1) discloses an augmented reality system that allows a user to select a makeup objective.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER VAUGHN whose telephone number is (571) 272-5253. The examiner can normally be reached M-F 11am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER MEHMOOD can be reached on (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER VAUGHN/Examiner, Art Unit 2675
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights