DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 48 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because it recites a “computer-readable storage medium” which can encompass non-statutory transitory forms of signal transmission. Applicant’s specification fails to exclude signals as a form of medium. As a result, the “computer-readable storage medium" can be interpreted as a signal, which is non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 26-29, 32, 34-40, 43, 45-48 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chen et al. (US 2023/0222721 A1, “Chen”).
As to claims 26, 37, 48, Chen discloses a computing device for generating a three-dimensional (3D) representation of a head of a participant in a video communication session, the computing device comprising processing circuitry configured to cause the computing device to be operative to:
acquire a captured 3D representation of the head (system 100 may receive multiple scans of a user’s face to generate a personalized 3D head mesh model representing the user, para. 0041),
identify positions of a set of facial landmarks in the captured 3D representation, the set of facial landmarks comprising facial landmarks indicative of a boundary of the human face (facial landmarks are defined, para. 0042),
determine a pose of the head (facial expressions include “Head turn left” and “Head turn right”, para. 0048),
determine a boundary between an inner part and an outer part of the captured 3D representation, based on the identified positions of the set of facial landmarks, the inner part of the captured 3D representation representing the face of the participant (face shape basis may be computed which correspond to facial landmark features, para. 0042), and
generate an avatar representation corresponding to the outer part of the captured 3D representation, using a Machine-Learning (ML) model trained for human heads, with the determined pose of the head as input (local machine learning network receives and processes video images of the video conference participant, determines facial expression values, including head turns, and applies the determined values to an avatar model to render the video conference participant video stream into a modified video stream in a 3D animation of an avatar model, para. 0052-0063).
As to claims 27, 38, Chen discloses: the processing circuitry configured to cause the computing device to be further operative to: extract the inner part of the captured 3D representation, and merge the extracted inner part of the captured 3D representation and the generated avatar representation into a merged 3D representation of the head (determined facial expression parameters are used to select blendshapes to morph or adjust a 3D mesh-based model, para. 0032, 0036, 0055, 0062).
As to claims 28, 39, Chen discloses: the processing circuitry configured to cause the computing device to be further operative to display the merged 3D representation of the head using a display device (modified video streams is displayed, para. 0063).
As to claims 29, 40, Chen discloses: wherein the display device is any one of: a computer display, a television, an Augmented-Reality (AR) device, a Virtual-Reality (VR) device, a Mixed-Reality (MR) device, an extended-Reality (XR) device, and a Head-Mounted Display (HMD) device (client device may be a computer desktop, laptop, etc., para. 0017, 0021, 0033).
As to claims 32, 43, Chen discloses: wherein the ML model is trained for the head of the participant (trained machine learning network to morph a 3D mesh-based model using user’s facial expression parameters, para. 0017, 0032, 0046-0051).
As to claims 34, 45, Chen discloses: the processing circuitry configured to cause the computing device to be further operative to train the ML model using at least the outer part of the captured 3D representation and the determined pose of the head (machine learning network may be trained on sets of images to determine facial expression parameter values, which include inner and outer raised brows, head turn left, head turn right, etc., para. 0047-0055).
As to claims 35, 46, Chen discloses: the processing circuitry configured to cause the computing device to be operative to train the ML model further based on the inner part of the captured 3D representation (machine learning network may be trained on sets of images to determine facial expression parameter values, which include inner and outer raised brows, head turn left, head turn right, etc., para. 0047-0055).
As to claims 36, 47, Chen discloses: wherein the captured 3D representation, the inner part of the captured 3D representation, the merged 3D representation, and the avatar representation are point clouds, meshes, or depth map images (3D mesh-based model, para. 0032, 0035-0045).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 30-31, 41-42 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Adkinson et al. (US 2022/0392167 A1, “Adkinson”).
Chen discloses: the processing circuitry configured to cause the computing device to be operative to acquire the captured 3D representation of the head by capturing the 3D representation of the head (receives multiple scans of user’s face to generate a 3D head mesh model, para. 0041), but differs from claims 30, 41 in that it does not explicitly disclose: using a 3D sensor.
Adkinson teaches the well known use of a 3D camera for generating a 3D mesh from a captured video stream (Abstract, para. 0026-0027, 0041). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chen with the above teaching of Adkinson in order to capture a 3D representation using a 3D camera device well known to be built in a smartphone, tablet, laptop, etc., as taught by Adkinson (para. 0026-0027).
As to claims 31, 42, Chen in view of Adkinson teaches: wherein the 3D sensor comprises one or more of: a 3D camera, a LIDAR, and an optical 3D sensor (Adkinson: 3D camera, LIDAR, or other depth measuring sensors, para. 0027, 0041).
Claim(s) 33, 44 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Fear et al. (US 2021/0178269 A1, “Fear”).
Chen differs from claims 33, 44 in that it does not disclose: the processing circuitry configured to cause the computing device to be further operative to acquire the ML model from a data storage associated with the participant.
Fear teaches associating a machine learning model trained according to a particular user and associated with the user’s account (para. 0020). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chen with the above teaching of Fear in order to generate virtual representations of user according the learned behavior pattern of the particular user, as taught by Fear (Abstract).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Eden et al. (US 2023/0135071 A1) teach a plurality of ML models associated with a plurality of users (para. 0069).
Chen et al. (US 2023/0260184 A1) teach rendering an avatar representing a video conference participant.
Kwatra et al. (US 2023/0343010 A1) teach generating photorealistic 3D talking faces.
Navarro et al. (US 2022/0270314 A1) teach generating a 3D animated avatar using a trained face detection model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stella L Woo whose telephone number is (571)272-7512. The examiner can normally be reached Monday - Friday, 8 a.m. to 5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
STELLA L. WOO
Primary Examiner
Art Unit 2693
/Stella L. Woo/ Primary Examiner, Art Unit 2693
Prosecution Insights