Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 27-26 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 27-26 of U.S. Patent No.12053301. Although the claims at issue are not identical, they are not patentably distinct from each other because they are obvious variant.
application
U.S. Patent No.12053301.
27. A method comprising:
forming a personalization image of a first user based on one or more images of the first user evincing one or more expressions;
capturing a first image of the first user using an eye tracking sensor implemented in a head mounted device worn by the first user; and
inferring a label of an expression evinced by the first user in the first image using a machine learnt algorithm that is trained to predict a label for an expression of the first user based on the personalization image of the first user and the first image.
28. The method of claim 27, wherein the machine learnt algorithm comprises a convolutional neural network algorithm.
29. The method of claim 27, wherein the machine learnt algorithm is trained using second images of one or more second users concurrently with the one or more second users evincing a plurality of expressions, wherein the plurality of expressions correspond to values of parameters that indicate facial deformations of the one or more second users evincing the plurality of expressions.
30. The method of claim 29, wherein inferring the label of the expression evinced by the first user comprises comparing values of the parameters derived from the first image with the values of the parameters that indicate the facial deformations corresponding to the expression evinced by the first user.
31. The method of claim 30, wherein inferring the label of the expression evinced by the first user comprises identifying an expression that corresponds to a best match between the values of the parameters derived from the first image and the values of the parameters that indicate the facial deformation corresponding to the identified expression.
32. The method of claim 30, wherein the values of the parameters comprise values of action units that indicate states of muscle contraction in independent muscle groups on respective faces of the one or more second users.
33. The method of claim 27, further comprising:
combining the personalization image with the first image to form a modified first image; and inferring the label of the expression evinced by the first user in the first image by applying the machine learnt algorithm to the modified first image.
34. The method of claim 33, wherein the machine learnt algorithm is trained to predict the labels based on a plurality of expressions.
35. The method of claim 33, wherein combining the personalization image with the first image comprises generating a mean neutral image of the first user using the personalization image and subtracting the mean neutral image from the first image.
36. The method of claim 27, further comprising:
modifying at least one of a representation of a three-dimensional model of a face of the first user or an avatar representative of the first user based on the inferred label of the expression.
37. The method of claim 27, further comprising:
utilizing the inferred label of the expression to perform at least one of evaluating effectiveness of content viewed by the first user wearing the head mounted device in eliciting a desired emotional response, adapting interactive content viewed by the first user wearing the head mounted device based on the inferred label of the expression, and generating a user behavior model to inform creation of content for viewing by the first user wearing the head mounted device.
38. A method comprising:
forming a personalization image of a first user by calculating a mean image of the first user based on one or more images of the first user evincing one or more expressions, and wherein the personalization image includes the mean image;
capturing a live input stream of images of the first user using an eye tracking sensor implemented in a head mounted device; and
inferring a label of an expression evinced by the first user in a first image of the live input stream of images based on the personalization image of the first user using a machine learnt algorithm that is trained to predict labels of a plurality of expressions.
39. The method of claim 38, further comprising:
generating a modified image by combining the first image with the personalization image, wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image.
40. The method of claim 39, wherein combining the first image with the first personalization image comprises:
subtracting the mean image from the first image.
41. The method of claim 38, wherein the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user.
42. A system comprising:
head mounted device comprising:
an eye tracking sensor configured to capture a first image of a first user; and
a processor configured to execute computer-readable instructions that, when executed, cause the processor to:
form a personalization image of the first user based on one or more images of the first user evincing one or more expressions; and
infer a label of an expression evinced by the first user in the first image using a machine learnt algorithm that is trained to predict a label for an expression of the first user based on the personalization image of the first user and the first image.
43. The system of claim 42, wherein the computer-readable instructions, when executed, further cause the processor to:
generate a modified image by combining the first image with the personalization image, wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image.
44. The system of claim 43, wherein combining the first image with the first personalization image comprises:
subtracting a mean image from the first image.
45. The system of claim 42, wherein the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user.
46. The system of claim 42, wherein the computer-readable instructions, when executed, further cause the processor to:
modify a computer-generated representation of the first user to represent an emotion corresponding to the label inferred using the machine learnt algorithm.
27. A method comprising:
forming at least one personalization image of a first user evincing a plurality of expressions by:
calculating a mean neutral image of the first user by averaging corresponding pixel values of images of the first user, wherein the images of the first user comprise the first user evincing one or more neutral expressions, and wherein the at least one personalization image includes the mean neutral image;
capturing a first image of the first user using an eye tracking sensor implemented in a head mounted device (HMD) worn by the first user; and
inferring a label of an expression evinced by the first user in the first image using a machine learnt algorithm trained to predict a label for an image of the first user based on a live image of the first user and the at least one personalization image of the first user.
28. The method of claim 27, wherein the machine learnt algorithm comprises a convolutional neural network algorithm.
29. The method of claim 27, wherein the machine learnt algorithm is trained using second images of a plurality of second users concurrently with the plurality of second users evincing the plurality of expressions, wherein the plurality of expressions correspond to values of parameters that indicate facial deformations of the plurality of second users evincing the plurality of expressions.
30. The method of claim 29, wherein inferring the label of the expression evinced by the first user comprises comparing values of the parameters derived from the first image with the values of the parameters that indicate the facial deformations corresponding to the plurality of expressions.
31. The method of claim 30, wherein inferring the label of the expression evinced by the first user comprises identifying one of the plurality of expressions that corresponds to a best match between the values of the parameters derived from the first image and the values of the parameters that indicate the facial deformation corresponding to the identified one of the plurality of expressions.
32. The method of claim 30, wherein the values of the parameters comprise values of action units that indicate states of muscle contraction in independent muscle groups on respective faces of the plurality of second users.
33. The method of claim 27, further comprising:
combining the at least one personalization image with the first image to form a modified first image; and inferring the label of the expression evinced by the first user in the first image by applying the machine learnt algorithm to the modified first image.
34. The method of claim 33, wherein the machine learnt algorithm is trained to predict the labels based on a subset of the plurality of expressions.
35. The method of claim 34, wherein combining the at least one personalization image with the first image comprises generating the mean neutral image of the first user using the at least one personalization image and subtracting the mean neutral image from the first image.
36. The method of claim 27, further comprising:
modifying at least one of a representation of a three-dimensional (3-D) model of a face of the first user or an avatar representative of the first user based on the inferred label of the expression.
37. The method of claim 27, further comprising:
utilizing the inferred label of the expression to perform at least one of evaluating effectiveness of content viewed by the first user wearing the HMD in eliciting a desired emotional response, adapting interactive content viewed by the first user wearing the HMD based on the inferred label of the expression, and generating a user behavior model to inform creation of content for viewing by the first user wearing the HMD.
38. A method comprising:
forming at least one personalization image of a first user evincing a plurality of expressions by:
calculating a mean neutral image of the first user by averaging corresponding pixel values of images of the first user, wherein the images of the first user comprise the first user evincing one or more neutral expressions, and wherein the at least one personalization image includes the mean neutral image;
capturing a first image of the first user using an eye tracking sensor implemented in a head mounted device (HMID); and
inferring a label of an expression evinced by the first user in the first image using a machine learnt algorithm trained to predict labels of a plurality of expressions using a plurality of images of a plurality of users evincing the plurality of expressions and a plurality of personalization images of the plurality of users, each personalization image of the plurality of personalization images being the mean neutral image of a corresponding user of the plurality of users.
39. The method of claim 38, further comprising:
generating a modified image by combining the first image with the first personalization image of the plurality of personalization images, the first personalization image corresponding to the mean neutral image of the first user, wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image.
40. The method of claim 39, wherein combining the first image with the first personalization image comprises:
subtracting the mean neutral image from the first image.
41. The method of claim 38, wherein the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user.
42. A system comprising:
head mounted device (HMD) comprising:
an eye tracking sensor configured to a first image of a first user; and
a processor configured to execute computer-readable instructions that, when executed, cause the processor to:
form at least one personalization image of the first user evincing a plurality of expressions by:
calculating a mean neutral image of the first user by averaging corresponding pixel values of images of the first user, wherein the images of the first user comprise the images of the first user, and wherein the at least one personalization image includes the mean neutral image; and
infer a label of an expression evinced by the first user in the first image using a machine learnt algorithm trained to predict labels of the plurality of expressions using a plurality of images of a plurality of users evincing the plurality of expressions and a plurality of personalization images of the plurality of users, each personalization image of the plurality of personalization images being a mean neutral image of a corresponding user of the plurality of users.
43. The system of claim 42, wherein the computer-readable instructions, when executed, further cause the processor to:
generate a modified image by combining the first image with a first personalization image of the plurality of personalization images, the first personalization image corresponding to the mean neutral image of the first user, wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image.
44. The system of claim 43, wherein combining the first image with the first personalization image comprises:
subtracting the mean neutral image from the first image.
45. The system of claim 42, wherein the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user.
46. The system of claim 42, wherein the computer-readable instructions, when executed, further cause the processor to:
modify a computer-generated representation of the first user to represent an emotion corresponding to the label inferred using the machine learnt algorithm.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 27, 29 and 42 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Tzvieli et al. (US 2016/0360970).
With respect to claim 27, Tzvieli et al. teach
forming a personalization image of a first user based on one or more images of the first user evincing one or more expressions (para a[0128], upper part of the face from vision-related features identified from images captured by the first and second cameras.);
capturing a first image of a first user (images captured by first and second camera) using an eye tracking sensor implemented in a head mounted device (HMD) (Head mounted system (HMS) worn by the first user (para [0126] and [0128]), [0158], the HMS further includes an eye tracker and a processor; the eye tracker is configured to track gaze of the user in order to identify an object the user is looking at; the processor is configured to decode a facial expression of the user based on data received from at least one of the first and second cameras); and
inferring (derive) a label of an expression (facial expression) evinced by the first user (expressed by the user) in the first image using a machine learnt algorithm trained to predict labels of a plurality of expressions of the first user based on the personalization image of the first user and the first image (para [0128]).
using second images of a plurality of second users (images of multiple users) concurrently with the plurality of second users evincing the plurality of expressions (para [0258], collecting a set of images of users taken while the users express various emotional responses; and [0294], training set may contain images of multiple users)), and
wherein the second images are captured using at least one eye tracking sensor implemented in at least one HMD worn by the plurality of second users (para [0158] ,the HMS further includes an eye tracker and a processor; the eye tracker is configured to track gaze of the user in order to identify an object the user is looking at; the processor is configured to decode a facial expression of the user based on data received from at least one of the first and second cameras).
With respect to claim 29, Tzvieli teaches that the machine learnt algorithm is trained using second images of one or more second users concurrently with the one or more second users evincing a plurality of expressions (para [0258], collecting a set of images of users taken while the users express various emotional responses, set includes images of one or more cameras that are not mounted to a frame of an HMS worn by users, the set of images is collected for training various predictors such as emotional response predictors (ERPs) discussed in this disclosure), wherein the plurality of expressions correspond to values of parameters that indicate facial deformations of the one or more second users evincing the plurality of expressions (para [0309], using action units, action units are defined as the fundamental actions of individual muscles or groups of muscles. Action units are given an action unit number using the Facial Action Coding System).
With respect to claim 42, claim 42 is rejected same reason as claim 27 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
A. Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Tzvieli in view of Ghosh et al. ("A multi-label convolutional neural network approach to cross-domain action unit detection." 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2015.)
With respect to claim 28, Tzvieli teaches that a neural network can be used (para. [0367]) but does not explicitly state: wherein training the machine learnt algorithm comprises training a convolutional neural network algorithm.
However in the same field of endeavor of image analysis and expression recognition, Ghosh teaches: wherein training the machine learnt algorithm comprises training a convolutional neural network algorithm (a convolutional neural network is used to train the AU (action units), pg. 610, Section IV )
Therefore, it would have obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify the combination of Tzvieli’s system with Ghosh explicit recitation of a convolutional neural network. The motivation for combining being that Tzvieli teach that multiple types of machine learning techniques can be used but does not disclose one explicit technique. Ghosh teaches the specific neural network that can be used for machine learning and training.
B. Claims 30-32 are rejected under 35 USC 103 as being unpatentable over Tzvieli et al. (US 2016/0360970) in view of Kapoor et al. ("Fully automatic upper facial action recognition", ANALYSIS AND MODELING OF FACES AND GESTURES, 2003. AMFG 2003. IEEE INTERNATIONAL WORKSHOP ON, 17 OCT. 2003).
With respect to claim 30, Tzvieli et al. teach all the limitations of claim 29 as applied above from which claim 30 respectively depend.
Tzvieli et al. do not teach expressly that inferring the label of the expression evinced by the first user comprises comparing values of the parameters derived from the first image with the values of the parameters that indicate the facial deformations corresponding to the plurality of expressions.
Kapoor et al. teach inferring the label of the expression evinced by the first user comprises comparing values of the parameters derived from the first image with the values of the parameters that indicate the facial deformations corresponding to the plurality of expressions.(3.2 Feature Extraction, 3rd para., To recover the shape parameters in a test image, say itest, a very naive approach will be to find an image, imatch, from the training set of pre-annotated images that most closely resembles itest. The shape parameters of itest then can be approximated by the shape parameters smatch
, which corresponds to imatch).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to compare values of the parameters derived from the first image with the values of the parameters that indicate the facial deformations corresponding to the plurality of expressions in the method of Tzvieli et al.
The suggestion/motivation for doing so would have been that to track the facial features robustly and efficiently .(3.2 Feature Extraction, 1st para)
Therefore, it would have been obvious to combine Kapoor et al. with Tzvieli et al. to obtain the invention as specified in claim 30.
With respect to claim 31, Kapoor et al. teach inferring the label of the expression evinced by the first user comprises identifying one of the plurality of expressions that corresponds to a best match (pre-annotated images that most closely resembles itest.) between the values of the parameters derived from the first image and the values of the parameters that indicate the facial deformation corresponding to the identified one of the plurality of expressions (3.2 Feature Extraction, 3rd para.,)
With respect to claim 32, Kapoor et al. teach that the values of the parameters comprise values of action units that indicate states of muscle contraction in independent muscle groups on respective faces of the plurality of second users (abstract).
C. Claims 33-41 and 43-46 are rejected under 35 USC 103 as being unpatentable over Tzvieli et al. (US 2016/0360970) in view of Cha et al. (WO 2015/167909).
With respect to claim 33, Tzvieli et al. teach all the limitations of claim 27 as applied above from which claim 33 respectively depend.
Tzvieli et al. do not teach expressly that
combining the at least one personalization image with the first image to form a modified first image; and
inferring the label of the expression evinced by the first user in the first image by applying the machine learnt algorithm to the modified first image.
Cha et al. teach combining (additive displacement) the at least one personalization image with the first image to form a modified first image(para [0033]); and
inferring the label of the expression (track facial expression) evinced by the first user in the first image by applying the machine learnt algorithm to the modified first image (para [0033] and [para 0048])
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to inferre the label of the expression evinced by the first user in the first image by applying the machine learnt algorithm to the modified first image in the method of Tzvieli et al.
The suggestion/motivation for doing so would have been that use known method to track the facial features robustly and efficiently
Therefore, it would have been obvious to combine Cha et al. with Tzvieli et al. to obtain the invention as specified in claim 33.
With respect to claim 34, Cha et al. teach that the machine learnt algorithm is trained to predict the labels based on the subset of the plurality of expressions (certain facial expressions) (para [0033] and [para 0048]).
With respect to claim 35, Cha et al. teach that combining the at least one personalization image with the first image comprises generating a mean neutral image of the first user (rest pose) using the at least one personalization image and subtracting (additive displacement) the mean neutral image from the first image (para [0033])
With respect to claim 36, Cha et al. teach that modifying at least one of a representation of a three-dimensional model of a face of the first user or an avatar representative of the first user based on the inferred label of the expression.(para 0015), avatar)
With respect to claim 37, Cha et al. teach that utilizing the inferred label of the expression to perform at least one of evaluating effectiveness of content viewed by the first user wearing the HMD in eliciting a desired emotional response, adapting interactive content viewed by the first user wearing the HMD based on the inferred label of the expression, and generating a user behavior model to inform creation of content for viewing by the first user wearing the head mounted device. (para [0015], the users can communicate with one another, and their respective avatars can reflect their eye movements, facial expressions, jaw movements, and/or mouth movements (such as caused by enunciation) to the other user.).
With respect to claim 38, Tzvieli et al. teach capturing a live input stream of images of the first user using an eye tracking sensor implemented in a head mounted device (Head mounted system (HMS) worn by the first user (para [0126] and [0128]), [0158], the HMS further includes an eye tracker and a processor; the eye tracker is configured to track gaze of the user in order to identify an object the user is looking at; the processor is configured to decode a facial expression of the user based on data received from at least one of the first and second cameras, para [0168], integrated operation of two or more HMDs with inward facing cameras, which can exchange posture and/or facial data in real time); and
inferring (derive) a label of an expression (facial expression) evinced by the first user (expressed by the user) in the first image the live input stream of images based on the personalization image of the first user using a machine learnt algorithm trained to predict labels of a plurality of expressions (para [0128]) using a plurality of images of a plurality of users (images of multiple users) evincing the plurality of expressions (para [0258], collecting a set of images of users taken while the users express various emotional responses; and [0294], training set may contain images of multiple users)).
Tzvieli et al. do not teach plurality of personalization images of the plurality of users, each personalization image of the plurality of personalization images being a mean neutral image of a corresponding user.
Cha et al. teach plurality of personalization images of the plurality of users, each personalization image of the plurality of personalization images being a mean neutral image of a corresponding user (para [0033], personalized mesh model, B0 , rest pose).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to use personalization images being a mean neutral image of a corresponding user in the method of Tzvieli et al.
The suggestion/motivation for doing so would have been that use known method to track the facial features robustly and efficiently
Therefore, it would have been obvious to combine Cha et al. with Tzvieli et al. to obtain the invention as specified in claim 38.
With respect to claim 39, Cha et al. teach generate a modified image by combining the first image with a first personalization image of the plurality of personalization images, the first personalization image corresponding to a mean neutral image of the first user (see para. [0033] and eq. (1), where the "blending weights" are obtained by subtracting the "rest pose" (i.e., neutral image) B0 from the acquired facial expression image), wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image (para [0048]).
With respect to claim 40, Cha et al. teach that combining the first image with the first personalization image comprises: subtracting the mean neutral image from the first image (para. [0033], additive displacements which represent the difference between the rest pose and certain facial expressions).
With respect to claim 41, Cha et al. teach that the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user (Fig. 7, fig. 8).
With respect to claim 43, Tzvieli et al. teach all the limitations of claim 42 as applied above from which claim 43 respectively depend.
Tzvieli et al. do not teach generate a modified image by combining the first image with a first personalization image of the plurality of personalization images, the first personalization image corresponding to a mean neutral image of the first user wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image
Cha et al. teach generate a modified image by combining the first image with a first personalization image of the plurality of personalization images, the first personalization image corresponding to a mean neutral image of the first user (see para. [0033] and eq. (1), where the "blending weights" are obtained by subtracting the "rest pose" (i.e., neutral image) B0 from the acquired facial expression image), wherein the label of the expression evinced by the first user in the first image is inferred using the machine learnt algorithm based on the modified image (para [0048]).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to use personalization images being a mean neutral image of a corresponding user in the method of Tzvieli et al.
The suggestion/motivation for doing so would have been that use known method to track the facial features robustly and efficiently
Therefore, it would have been obvious to combine Cha et al. with Tzvieli et al. to obtain the invention as specified in claim 38.
With respect to claim 44, Cha et al. teach that combining the first image with the first personalization image comprises: subtracting the mean neutral image from the first image (para. [0033], additive displacements which represent the difference between the rest pose and certain facial expressions).
With respect to claim 45, Cha et al. teach that the first image depicts only a portion of a face of the first user, the portion being proximal to one or both eyes of the first user (Fig. 7, fig. 8).
With respect to claim 46, Cha et al. teach modify a computer-generated representation of the first user to represent an emotion corresponding to the label inferred using the machine learnt algorithm. (para [0015], avatar)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Randolph Chu whose telephone number is 571-270-1145. The examiner can normally be reached on Monday to Thursday from 7:30 am - 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached on (571) 272-7778.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/RANDOLPH I CHU/
Primary Examiner, Art Unit 2663
Prosecution Insights