CTNF 18/470,032 CTNF 83215 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Continued Examination Under 37 CFR 1.114 07-42-04 AIA A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/13/2026 has been entered. Claim Objections 07-29-01 AIA Claim 8 is objected to because of the following informalities: claim 8 recites “the method of claim 6, further comprising determining a third loss value based on the first expression feature and the second expression feature ”. It is currently depending on claim 6 and lacks antecedent basis for the underlined limitation. It is suggested that claim 8 depends on claim 7 . Appropriate correction is required. Response to Arguments Applicant’s arguments with respect to claim(s) 1-3, 5-15, 17-27 and 29-33 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1-3, 5, 13-15, 17, 25-27 and 29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Berlin et al. (US Patent Number 11,308,657 B1) in view of Wiles et al. (NPL, “X2Face: A network for controlling face generation using image, audio, and pose code”, 2018) . (1) regarding claim 1: As shown in fig. 10, Berlin disclosed a method for generating a representation of a face ( note that a method of training an autoencoder configured to perform face generation is disclose, abs. ), the method comprising: obtaining one or more images of a face ( col. 1, lines 39-41, note that methods configured to train an autoencoder using images that include faces, wherein the autoencoder comprises an input layer, an encoder configured to output a latent image from a corresponding input image ); obtaining an audio signal obtained concurrently with obtaining the one or more images of the face ( col. 6, lines 1-5, note that one or more microphones 116 may be provided to record audio content (e.g., the speech of the person whose face is being recorded) in synchronization with the image/video content. For example, the audio content may be stored in the data store 110 as part of (a track of) a destination video ); wherein predetermined characteristics of the face remain constant relative to the encoded expression ( col. 1, lines 52-57, note that a source face may be mapped to the target face using the trained encoder and the fine-tuned decoder to generate a reconstructed face having the appearance of the target whose expression, pose, and/or lighting match those of the source face ); mapping the encoded expression to a corresponding expression of a facial model ( col. 2, lines 7-11, note that the trained second decoder is used to generate an output using a source image of a first face having a facial expression, where the facial expression of the first face from the source image is applied to the first specific target face ); and generating the representation of the facial model based on the encoded expression ( col. 2, lines 42-45, note that to generate an output using a source image of a first face having a facial expression, where the facial expression of the first face from the source image is applied to the first specific target face ). Berlin disclosed most of the subject matter as described as above except for specifically teaching generating an encoded expression representing an expression of the face based on the one or more images of the face and the audio signal. However, Wiles disclosed generating an encoded expression representing an expression of the face based on the one or more images of the face and the audio signal ( fig. 1, page 2, note that Fig.1: Overview of X2Face: a model for controlling a source face using a driving frame i.e. image, audio data, or specifying a pose vector. Also note that the generated frame resulting from X2Face has the identity, hairstyle, etc. of the source face but the properties of the driving vector (e.g. the given pose, if pose information is given; or the driving frame’s expression/pose, if a driving frame is given). ). At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach generating an encoded expression representing an expression of the face based on the one or more images of the face and the audio signal. The suggestion/motivation for doing so would have been in order to achieve a robust method that makes fewer assumptions about the input data generate face model (abs.). Therefore, it would have been obvious to combine Berlin with Wiles to obtain the invention as specified in claim 1. (2) regarding claim 2: Berlin further disclosed the method of claim 1, wherein the encoded expression is based on motion features determined based on images of the face ( col. 4, lines 6-10, note that the source face in the output preserves the expressions of the face in the original destination image/video (e.g., has lip motions, eye motions, eyelid motions, eyebrow motions, nostril flaring, etc.) ). (3) regarding claim 3: Berlin further disclosed the method of claim 1, wherein the predetermined characteristics of the face include at least one of a view angle, color style, or identity of the face (c ol. 7, lines 48-50, note that single camera or a multi-camera array (e.g., an oscillating camera array) may be utilized to capture images of an object or person from a variety of angles with varying lighting (e.g., optionally while varying lighting intensity and/or lighting angle using one or more motorized lights or lighting arrays) ). (4) regarding claim 5: Berlin further disclosed the method of claim 1, further comprising: receiving a frame, the frame including at least a portion of a face ( col. 10, lines 63-65, note that the training images may be option from videos, where the presence of a face may be detected in a given frame or series of frames, the face may be isolated, and a determination may be made as to whether the face satisfies one or more criteria configured to be used to determine if the face images is suitable to be used for training ); encoding motion features of the frame into the encoded expression ( col. 11, lines 6-8, note that at block 304A, one or more images are provided to the learning engine, including the encoder/decoder for generic face training ); and outputting the encoded expression for transmission ( col. 11, lines 13-15, note that At block 308, the learning engine's classification may be examined (e.g., by a human or another face classification system) and a determination is made as to whether the classification is correct ). The proposed rejection of claims 1-3, 5 renders inherent the steps of the apparatus (fig. 1) claims 13-15, 17 and the non-transitory computer readable medium (col. 33, lines 19-22) claims 25-27, 29 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claims 1-3, 5 are equally applicable to claims 13-15, 17, 25- 27 and 29 . 07-21-aia AIA Claim (s) 6, 8, 18, and 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Berlin et al. (US Patent Number 11,308,657 B1) and Wong et al. (US Publication Number 2018/0349682 A1), further in view of Wiles et al. (NPL, “X2Face: A newrork for controlling face generation using image, audio, and pose code”, 2018) . (1) regarding claim 6: As shown in figs. 3A-3C, Berlin disclosed a method for training an expression encoder ( col. 10, lines 41-44, note that FIGS. 3A-3B illustrate an example learning engine (e.g., CNN autoencoder) generic face training process (3A) and an example learning engine (e.g., CNN autoencoder) target face training process (3B) ), comprising: obtaining a first frame and a second frame, the first frame and second frame including at least a portion of a face ( col. 10, lines 58-665, note that at block 302A, generic face training images are accessed from one or more data sources. As described elsewhere herein, the training images may be option from videos, where the presence of a face may be detected in a given frame or series of frames, the face may be isolated ); generating a first expression feature for the first frame, the first expression feature representing a first expression of the face ( col. 11, lines 48-52, note that the different expressions may include some or all of the following: Anger (e.g., flared nostrils, eyebrows squeezed together to form a crease, eyelids tight and straight, slightly lowered head, eyes looking upwards through a lowered brow, tightening of facial muscles, tight lips) ); generating a second expression feature for the second frame, the second expression feature representing a second expression of the face (c ol. 11, lines 54-56, note that note that the different expressions may include some or all of the following boredom (e.g., half-open eyelids, raised eyebrows, frowning lips, relaxed muscles, vacant gaze, immobile face) ); determining a first loss value based on the combining ( col. 19, lines 38-41, note that the face-swapping may results in a certain amount of blurriness as a result of pixel loss, particularly with respect to certain features, such as teeth ); and adjusting a feature encoder based on the determined first loss value ( col. 19, lines 41-44, note that image processing tools may be provided to sharpen the image or selected portions thereof. Further, image processing tools may be provided to remove or reduce undesirable shadowing ). Berlin disclosed most of the subject matter as described as above except for specifically teaching combining an expression feature from one of the first frame or the second frame with a view angle feature from the other of the first frame or the second frame; generating a first view angle feature for the first frame, the first view angle feature representing a first angle from which the face is viewed from; and generating a second view angle feature for the second frame, the second view angle feature representing a second angle from which the face is viewed from. However, Wong disclosed generating a first view angle feature for the first frame, the first view angle feature representing a first angle from which the face is viewed from ( para. [0007], note that determining a first angle associated with a first area of the eye in the image frame ); and generating a second view angle feature for the second frame, the second view angle feature representing a second angle from which the face is viewed from ( para. [0007], note that determining a second angle associated with a second area of the eye in the image frame ). At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to configure generating a first view angle feature for the first frame, the first view angle feature representing a first angle from which the face is viewed from; and generating a second view angle feature for the second frame, the second view angle feature representing a second angle from which the face is viewed from. The suggestion/motivation for doing so would have been in order to improve techniques for face-based user authentication and to detect facial forgery (e.g., using still images) during image-based user authentication, based on more accurate liveness detection of the face of a user of a secure system (para. [0004]). Therefore, it would have been obvious to combine Berlin with Wong to obtain the invention as specified in claim 6. In addition to that, Wiles disclosed combining an expression feature from one of the first frame or the second frame with a view angle feature from the other of the first frame or the second frame ( page 10, 5.3 Controlling the image generation with pose, para. [0002], note that e then use fv→p to train fp→v (Section 4.1) and present generated frames for different, unseen test identities using the learnt mappings in Fig. 6. The source frame corresponds to psource in Section 4.1 while pdriving is used to vary one head pose angle while keeping the others fixed. See fig. 6, note that frames are combined ). At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach combining an expression feature from one of the first frame or the second frame with a view angle feature from the other of the first frame or the second frame. The suggestion/motivation for doing so would have been in order to achieve a robust method that makes fewer assumptions about the input data generate face model (abs.). Therefore, it would have been obvious to combine Berlin with Wong Wiles to obtain the invention as specified in claim 6. (2) regarding claim 8: Berlin further disclosed the method of claim 6, further comprising determining a third loss value based on the first expression feature and the second expression feature ( col. 19, lines 7-10, note that a perceptual loss function may be utilized to compare high level differences between images, such as content and style discrepancies. By way of yet further example, triple-consistency loss function may be utilized ). The proposed rejection of claim 6 renders obvious the steps of the apparatus (fig. 1) claim 18 and the non-transitory computer readable medium claim (col. 33, lines 19-22) 30 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claim 6 is equally applicable to claims 18 and 30 . 07-21-aia AIA Claim (s) 7, 19-20 and 31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Berlin, Wong and Wiles, further in view of Bouaziz et al. (US Publication Number 2019/0156549 A1) . (1) regarding claim 7: Berlin further disclosed the method of claim 6, determining the first loss value based on a comparison between the first reconstructed image and the first frame ( col. 14, lines 51-53, note that the alignment differences between a face image in a source dataset may be compared to corresponding face images in the destination dataset, and the mean square alignment differences may be calculated ); and determining a second loss value based on a comparison between the second reconstructed image and the second frame ( col. 17, lines 3-6, note that a perceptual loss function may be utilized to compare high level differences between images, such as content and style discrepancies ). Berlin disclosed most of the subject matter as described as above except for specifically teaching wherein the first expression matches the second expression, and further comprising: generating a first reconstructed image based on combining the first view angle feature with the second expression feature; and generating a second reconstructed image based on the combining the second view angle feature with the first expression feature. However, Bouaziz disclosed wherein the first expression matches the second expression ( para. [0012], note that the dynamic expression model and the plurality of blendshapes are better matched to the individual facial characteristics of the user ), and further comprising: generating a first reconstructed image based on combining the first view angle feature with the second expression feature ( para. [0063], note that the online avatar may further include a reconstruction of texture and other facial features such as hair in order to allow for a complete digital online avatar that can directly be integrated into online applications or communication applications and tools ); and generating a second reconstructed image based on the combining the second view angle feature with the first expression feature ( para. [0066], note that the alternating processing may be bootstrapped by initializing the dynamic expression model 316, 318 with a PCA reconstruction for the neutral expression and a deformation transfer from the template model of the adaptive dynamic expression model 318 to the user-specific dynamic expression model 316 ). At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the first expression matches the second expression, and further comprising: generating a first reconstructed image based on combining the first view angle feature with the second expression feature; and generating a second reconstructed image based on the combining the second view angle feature with the first expression feature. The suggestion/motivation for doing so would have been in order to providing a dynamic expression model and receiving tracking data corresponding to a facial expression of a user (para. [0008]). Therefore, it would have been obvious to combine Berlin, Wong, Wiles with Bouaziz to obtain the invention as specified in claim 7. (2) regarding claim 20: Berlin further disclosed the apparatus of claim 19, wherein the at least one processor is further configured to determine a third loss value based on the first expression feature and the second expression feature ( col. 19, lines 7-10, note that a perceptual loss function may be utilized to compare high level differences between images, such as content and style discrepancies. By way of yet further example, triple-consistency loss function may be utilized ). The proposed rejection of claims 7 renders obvious the steps of the apparatus (fig. 1) claim 19 and the non-transitory computer readable medium claim (col. 33, lines 19-22) 31 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claim 7 is equally applicable to claims 19 and 31 . Allowable Subject Matter Claims 9-12, 21-24 and 32-33 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior arts made of record do not teach “wherein the first angle matches the second angle, and further comprising: generating a first reconstructed image based on combining the first expression feature with the second view angle feature; generating a second reconstructed image based on combining the second expression feature with the first view angle feature; determining the first loss value based on a comparison between the first reconstructed image and the first frame; and determining a second loss value based on a comparison between the second reconstructed image and the second frame”, as recited in claims 9, 21 and 32. Dependent claims 10 and 22 depend on claims 9 and 21 respectively. In addition to that, the prior arts made of record do not teach “augmenting the first frame to generate an augmented frame; generating an augmented expression feature based on the augmented frame; generating an augmented view angle feature based on the augmented frame; obtaining a semantic labelled version of the first frame; generating a surrogate expression feature based on the semantically labelled version of the first frame; generating a surrogate view angle feature based on the semantically labelled version of the first frame; and generating a third loss value based on a comparison between the augmented expression feature and the surrogate expression feature and a comparison between augmented view angle feature and the surrogate view angle feature”, as recited in claims 11, 23 and 33. Dependent claims 12 and 24 depend on claims 11 and 23 respectively. Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zeng et al . (NPL, “Talking Face Generation with Expression-Tailored Generative Adversarial Network”, 2020) disclosed generating vivid talking faces is to synthesize identity-preserving natural facial expressions beyond audio-lip synchronization, which usually need to disentangle the informative features from multiple modals and then fuse them together. Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HILINA K DEMETER/Primary Examiner, Art Unit 2617 Application/Control Number: 18/470,032 Page 2 Art Unit: 2617 Application/Control Number: 18/470,032 Page 3 Art Unit: 2617 Application/Control Number: 18/470,032 Page 4 Art Unit: 2617 Application/Control Number: 18/470,032 Page 5 Art Unit: 2617 Application/Control Number: 18/470,032 Page 6 Art Unit: 2617 Application/Control Number: 18/470,032 Page 7 Art Unit: 2617 Application/Control Number: 18/470,032 Page 8 Art Unit: 2617 Application/Control Number: 18/470,032 Page 9 Art Unit: 2617 Application/Control Number: 18/470,032 Page 10 Art Unit: 2617 Application/Control Number: 18/470,032 Page 11 Art Unit: 2617 Application/Control Number: 18/470,032 Page 12 Art Unit: 2617 Application/Control Number: 18/470,032 Page 13 Art Unit: 2617 Application/Control Number: 18/470,032 Page 14 Art Unit: 2617 Application/Control Number: 18/470,032 Page 15 Art Unit: 2617 Application/Control Number: 18/470,032 Page 16 Art Unit: 2617
Prosecution Insights