DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/22/2026 has been entered.
Response to Amendment
The office action is in response to Applicant’s amendment filed 04/22/2026 which has been entered and made of record. Claims 1, 6, 7, 12, 13, 14 and 17 have been amended. Claims 4-5, 15 and 19 have been cancelled. Claims 21-24 have been newly added. Claims 1-3, 6-14, 16-18 and 20-24 are pending in the application.
Response to Arguments
Applicant’s arguments, filed 04/22/2026, with respect to the rejection(s) under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Liu, Rey and Kim as fully explained below.
Applicant argues Liu, Rey and Hussen taken individually or in combination, do not teach the newly amended limitation.
Examiner agrees Liu, Rey and Hussen do not teach the newly amended independent claims. However, a new ground of rejection is made in view of Liu, Rey and Kim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9, 13, 17 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL Liu et al. (“MusicFace: Music-driven Expressive Singing Face Synthesis”), hereinafter as Liu, in view of de la Rey et al. (US 20230130844 A1), hereinafter as Rey, further in view of Kim et al. (US 20240338874 A1), hereinafter as Kim.
Regarding claim 1, Liu teaches A method (Liu Page 1, Left column, Abstract, “we present a method for this task with natural motions of the lip, facial expression, head pose, and eye states.”) comprising: accessing an audio input comprising a mixture of vocal sounds and non-vocal sounds (Liu Page 1, Right Column, Figure 1, “Our goal is to synthesize a vivid dynamic singing face coherent with the input music audio, which is mixed with human voice and background music.”); separating, …… , the audio input into a first audio output representing the vocal sounds and a second audio output representing the non-vocal sounds (Liu Page 3, Right Column, last paragraph “firstly adopts an audio source separation model O to decompose music into human voice Av and background music Ab , then gets encoded lyric feature L and melody feature M respectively using an attention-assisted two-stream encoder E”); determining, by one or more trained avatar animation models and by separately encoding the first audio output representing the vocal sounds and the second audio output representing the non-vocal sounds, an avatar animation temporally corresponding to the audio input (Liu teaches two stream decoder for vocal and non-vocal sounds, further teaches expression generation model, pose generation model and eye state generation model. Page 3, Figure 2, “the Generator module generates facial driving parameters …… the Renderer module aims to synthesize a photo-realistic video. Specifically, eye state parameters are encoded into eye attention maps, and other parameters provide a 3D model guidance to render faces. Finally, an expressive and rhythmic singing face video is rendered by combining rendered faces with eye attention maps”, Page 4, Figure 3, “Our generator contains an Encoder and a Decoder. The Encoder consists of a Two-stream Audio Encoder (TSAE) and an Attention-based Modulator (ATM). The Decoder contains three downstream generators, including Expression Generation Network (EGN), Pose Generation Network (PGN), and Eye State Generation Network (ESGN).” And Page 4, Left column, Second paragraph, “The Encoder (Sec. 3.2) consists of a Two-stream Audio Encoder (TSAE) to encode lyric and melody separately”)); ……and rendering, in real time and temporally coincident with the audio input, the determined avatar animation (Liu teaches temporally match the input audio with the video sequence, and the avatar animation is synchronized displayed with audio input. Page 5, Left Column, first paragraph, “Furthermore, in order to incorporate temporal information and match the frequency of video frames (30 fps), the feature sequence are converted to overlapping windows of size 39 (corresponding to 390ms ) at 30 fps.”, Page 5, Figure 5, Right Column, “our generated head pose dynamics are smoother than others. And the turn of dominant varying angle (shown as green curve) occurs nearly at the same time with ground truth, meaning that our generated head dynamics have more similar rhythm to the ground truth recorded by a performer.” And Page 5, right column, last paragraph, “our two-stream design greatly reduces the complexity of the lip synchronization task, thus leading to a better synchronization result.”).
Liu does not explicitly teach by a trained audio source-separation model……receiving a user input comprising a user instruction for animating the avatar; determining, based on the user input, one or more of (1) an animation mode for the avatar or (2) a facial expression for the avatar by: encoding the user input using a text encoder; classifying, by a trained instruction classifier, at least a portion of the encoded user input as at least one of (1) emotion-related input or (2) movement-related input; selecting, based on the classification of the encoded user input, a trained emotion encoder or a trained dance encoder; and encoding, using the selected trained emotion encoder or the trained dance encoder, at least the portion of the user input corresponding to the classified portion of the encoded user input; …… Rey teaches by a trained audio source-separation model (Rey paragraph [0070] “In one method a crude, general model 410 is trained on a general training dataset. General training dataset may comprise labeled source audio data and labeled noise audio data. General model 410 may be referred to as a general source separation model 410 or trained audio source separation model 410.” And paragraph [0021] “train an audio source separation model using, at least in part, the received single-track audio input stream, wherein the audio separation model is trained to receive the single-track audio input stream and generate a plurality of audio stems corresponding to one or more audio sources of the plurality of sources, and separate audio sources, using the audio source separation model, from the audio input stream in accordance with one or more processing recipes to generate a plurality of source separated output stems.”).
Liu and Rey are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Rey teaches an improved audio source separation model to improve the accuracy and quality of source separation (Rey paragraph [0074] “This process 420 can be repeated iteratively, each improving the model's separation quality (e.g., fine tuning to improve the accuracy and/or quality of the source separation).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the audio source separation model of Liu with the teaching of Rey to achieve better accuracy and quality with audio source separation, and eventually achieve better avatar animation result.
Liu in view of Rey are not relied on for the below claim language …… receiving a user input comprising a user instruction for animating the avatar; determining, based on the user input, one or more of (1) an animation mode for the avatar or (2) a facial expression for the avatar by: encoding the user input using a text encoder; classifying, by a trained instruction classifier, at least a portion of the encoded user input as at least one of (1) emotion-related input or (2) movement-related input; selecting, based on the classification of the encoded user input, a trained emotion encoder or a trained dance encoder; and encoding, using the selected trained emotion encoder or the trained dance encoder, at least the portion of the user input corresponding to the classified portion of the encoded user input; ……
Kim teaches …… receiving a user input comprising a user instruction for animating the avatar (Kim paragraph [0019] “FIG. 1 illustrates an example of gesture generation from text, according to some embodiments. As described herein, gesture generation may include a number of steps. First, text 105 is received.”, paragraph [0024] “Framework 200 begins with an action token 204 , subject token 206 , emotion token 208 , and text tokens 210 . The action token 204 , subject token 206 , and emotion token 208 may be initialized at a default value/token which will be transformed by the first encoder 202 . Text tokens 210 may include one or more tokens associated with words in a sentence or phrase, i.e., “Sentence 1” as depicted in FIG. 2 . For example, the sentence may be “Get some rest.” as depicted in text 105 in FIG. 1”); determining, based on the user input, one or more of (1) an animation mode for the avatar or (2) a facial expression for the avatar by (Kim Figure 1 and paragraph [0019] “First, text 105 is received. Then, text classification 110 is used to extract relevant features from the text 105 . Finally, an autoregressive model 115 is used to generate a 3D gesture 120”): encoding the user input using a text encoder (Kim paragraph [0025] “In some embodiments, first encoder 202 may be a pre-trained language model (PLM), e.g., BERT, ALBERT, GPT-3, GPT-3.5, and GPT-4. Other PLMs beyond those listed may be used for first encoder 202”); classifying, by a trained instruction classifier, at least a portion of the encoded user input as at least one of (1) emotion-related input or (2) movement-related input (Kim paragraph [0104] “At step 902, a system (e.g., computing device 400, user device 610, model server 640, device 1000, or device 1015) generates, via a pre-trained language model (e.g., first encoder 202), a first subject latent representation (e.g., subject latent representation 226), a first action latent representation (e.g., action latent representation 224), and a first emotion latent representation (e.g., emotion latent representation 228).”, paragraph [0021] “Text classification 110 makes use of natural language processing models to extract relevant features from the text. Many models for natural language processing have been pre-trained to understand the relationships between words in a large vocabulary; for example, the Bidirectional Encoder Representations from Transformers (BERT)”); selecting, based on the classification of the encoded user input, a trained emotion encoder or a trained dance encoder (Kim teaches multiple encoders based on output of first encoder Figure 9 and paragraph [0025-0026] “first encoder 202 encodes action embedding 214 into an action latent representation 224, subject embedding 216 into subject latent representation 226, emotion embedding into 208 into emotion latent representation 228, and text embeddings 220 into text latent representations 230. The action latent representation 224, subject latent representation 226, and emotion latent representation 228 are received at second encoder 234, third encoder 236, and fourth encoder 238 …… The second encoder 234 , third encoder 236 , and fourth encoder 238 may be classification models.”); and encoding, using the selected trained emotion encoder or the trained dance encoder, at least the portion of the user input corresponding to the classified portion of the encoded user input (Kim Figure 9 and paragraph [0105-0108] “At step 904, the system generates, via an action classifier (e.g., second encoder 234), a predicted action label based on the first action latent representation. At step 906, the system generates, via a subject classifier (e.g., third encoder 236), a predicted subject label based on the first subject latent representation. At step 908, the system generates, via an emotion classifier (e.g., fourth encoder 238), a predicted emotion label based on the first emotion latent representation.”); ……
Liu, Rey and Kim are in the same field of endeavor, namely computer graphics, especially in the field of audio and text driven avatar animation. Kim teaches a method for animating avatar based on user input to achieve better flexibility and accuracy (Kim paragraph [0027] “correct classification of subject and emotion improve the accuracy of the action/gesture classification.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Kim with the method of Liu in view of Rey to achieve better flexibility and accuracy.
Regarding claim 9, Liu in view of Rey and Kim teach The method of Claim 1, and further teach wherein the one or more trained avatar animation models comprise a trained facial expression model (Liu Page 5, right column, fourth paragraph, “3.3.1 Expression Generation Network We employ a simple MLP consisting of two fully connected layers and one ReLU activation layer to regress facial expression (including lip motion) parameters from the encoded lyric and melody features.”), the method further comprising: generating, by a trained vocal encoder of the trained facial expression model, a set of encoded vocal features (Liu teaches a vocal encoder
A
E
v
, Page 5, Left column, second paragraph, “3.2.2 Two-stream Audio Encoder (TSAE) Given the separated human voice feature
A
v
and background music feature
A
b
, we adopt a Two-stream Audio Encoder (TSAE) that consists of two networks
A
E
v
and
A
E
b
to encode the MFCC features of human voice
a
t
v
and background music
a
t
b
, separately:
f
t
v
=
A
E
v
a
t
v
”); generating, by a trained non-vocal encoder of the trained facial expression model, a set of encoded non-vocal features (Liu teaches a non-vocal encoder
A
E
b
,
the background music feature as the non-vocal feature, and a separate network to encode the background music feature,
f
t
b
=
A
E
b
a
t
b
)
; and generating, by a decoder of the trained facial expression model, a facial-expression animation for the avatar based on the set of encoded vocal features and the set of encoded non-vocal features (Liu, Page 4, Figure 3, “The Encoder consists of a Two-stream Audio Encoder (TSAE) and an Attention-based Modulator (ATM). The Decoder contains three downstream generators, including Expression Generation Network (EGN), Pose Generation Network (PGN), and Eye State Generation Network (ESGN)” and Page 3, Figure 2, “the Generator module generates facial driving parameters (expressions, head poses and eye states)…… the Renderer module aims to synthesize a photo-realistic video.).
Regarding claim 13, it recites similar limitations of claim 1 but in a non-transitory computer readable storage media form. The rationale of claim 1 rejection is applied to reject claim 13. In addition, Rey teaches One or more non-transitory computer readable storage media storing instructions and coupled to one or more processors that are operable to execute the instructions to (Rey paragraph [0021] “a system includes a memory component storing machine-readable instructions, and a logic device and/or processor configured to execute the machine-executable instructions” and paragraph [0180] “Software in accordance with the present disclosure, such as non-transitory instructions, program code, and/or data, can be stored on one or more non-transitory machine-readable mediums.”):
Liu, Rey and Kim are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Rey teaches an improved audio source separation model to improve the accuracy and quality of source separation (Rey paragraph [0074] “This process 420 can be repeated iteratively, each improving the model's separation quality (e.g., fine tuning to improve the accuracy and/or quality of the source separation).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the audio source separation model of Liu in view of Kim with the teaching of Rey to achieve better accuracy and quality with audio source separation, and eventually achieve better avatar animation result.
Regarding claim 17, it recites similar limitations of claim 1 but in an apparatus form. The rationale of claim 1 rejection is applied to reject claim 17. In addition, Rey teaches An apparatus comprising: one or more non-transitory computer readable storage media storing instructions; and one or more processors coupled to the non-transitory computer readable storage media, the one or more processors operable to execute the instructions to (Rey paragraph [0048] “[0048] The systems and methods disclosed herein may be implemented on at least one computer-readable medium carrying instructions that, when executed by at least one processor causes the at least one processor to perform any of the method steps disclosed herein. Some implementations relate to a computer system including at least one processor and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform any method steps disclosed herein. In various implementations, models described herein may be implemented as stored data and software modules and/or code acting on the stored data.”):
Liu, Rey and Kim are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Rey teaches an improved audio source separation model to improve the accuracy and quality of source separation (Rey paragraph [0074] “This process 420 can be repeated iteratively, each improving the model's separation quality (e.g., fine tuning to improve the accuracy and/or quality of the source separation).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the audio source separation model of Liu in view of Kim with the teaching of Rey to achieve better accuracy and quality with audio source separation, and eventually achieve better avatar animation result.
Regarding claim 23, claim 23 has similar limitations as claim 9, therefore it is rejected under the same rationale as claim 9.
Claim(s) 2, 14 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL Liu et al. (“MusicFace: Music-driven Expressive Singing Face Synthesis”), hereinafter as Liu, in view of de la Rey et al. (US 20230130844 A1), hereinafter as Rey, further in view of Kim et al. (US 20240338874 A1), hereinafter as Kim, and Xiong et al. (CN 111091800 A), hereinafter as Xiong. The original and a machine translation of Xiong are provided by the examiner.
Regarding claim 2, Liu in view of Rey and Kim teach The method of Claim 1, and further teach wherein the trained audio source-separation model is defined by a self-supervised training process (Rey paragraph [0123] “the neural network 1500 may be trained using supervised learning where combinations of training data that include a combination of input data and a ground truth (e.g., expected) output data.”) comprising: providing, to a source-separation model, a plurality of training audio inputs (Rey paragraph [0070] “The training dataset may comprise a plurality of datasets, each of the plurality of datasets comprising labeled audio samples configured to train the system to address a source separation problem. The plurality of datasets may comprise a speech training dataset comprising a plurality of labeled speech samples, and/or a non-speech training dataset comprising a plurality of labeled music and/or noise data samples.”); for each of the training audio inputs (Rey paragraph [0117] “A neural network 1500 is implemented as a recurrent neural network, deep neural network, convolutional neural network or other suitable neural network that receives a labeled training dataset 1510 to produce audio output 1512 (e.g., one or more audio stems) for each input audio sample.”): separating, by the source-separation model, each of the plurality of training audio inputs into a first training audio output representing vocal sounds and a second training audio output representing non-vocal sounds (Rey paragraph [0095] “The process 1000 begins with a general model 1002, which may be implemented as a pretrained model as previously discussed. An input mixture 1004, such as a single-track audio signal, or a plurality of single-track audio signals, with an unseen source mixture, is processed through the general model 1002 in step 1006 to generate separated audio signals from the mixture, including machine learning separated source signals 1008 and machine learning separated noise signals 1010.”); …… classifying, by a sound classifier, (1) the encoded first training audio output as vocal or non-vocal sounds and (2) the encoded second training audio output as vocal or non-vocal sounds (Rey Figure 10 and 13D, paragraph [0095] “including machine learning separated source signals 1008 and machine learning separated noise signals 1010. In step 1012, the results are evaluated to confirm the separated audio sources have a sufficient quality (e.g., comparing an estimated MOS and threshold as previously described and/or other quality measurement). If the results are determined to be good, then the separated sources are output in step 1014.” And paragraph [0108] “output fidelity may be measured use an MOS algorithm that is able to measure divergence of the stem's output from a particular labeled dataset, such as a collection of speech samples from an individual. In some implementations, such an algorithm may be implemented as a neural network that has been pre-trained to either classify a source or to measure divergence of the source from a given dataset.”); and updating the source separation model based on (1) a similarity between the composite audio output and the respective training audio input and (2) the classifications made by the sound classifier (Rey teaches updating network parameters based on misclassification and difference of the output audio and input audio mix, paragraph [0063] “If the network mislabels the input audio sample, then a backward pass through the network may be used to adjust parameters of the network to correct for the misclassification.” And paragraph [0118] “The training process to generate a trained neural network model includes a forward pass through the neural network 1500 to produce an audio stem or other desired audio output 1512. Each data sample is labeled with the desired output of the neural network 1500, which is compared to the audio output 1512. In some implementations, a cost function is applied to quantify an error in the audio output 1512 and a backward pass through the neural network 1500 may then be used to adjust the neural network coefficients to minimize the output error.”).
Liu, Rey and Kim are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Rey teaches an improved audio source separation model to improve the accuracy and quality of source separation (Rey paragraph [0074] “This process 420 can be repeated iteratively, each improving the model's separation quality (e.g., fine tuning to improve the accuracy and/or quality of the source separation).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the audio source separation model of Liu in view of Kim with the teaching of Rey to achieve better accuracy and quality with audio source separation, and eventually achieve better avatar animation result.
Liu in view of Rey and Kim are not relied on for the below claim language encoding, by a vocal encoder, the first training audio output; encoding, by a music encoder, the second training audio output; constructing, by an audio decoder and based on the encoded first training audio output and the encoded second training audio output, a composite audio output; Xiong teaches encoding, by a vocal encoder, the first training audio output (Xiong paragraph [0013] “an encoding unit, configured to use a speaker voiceprint encoder in a trained singing optimization model to encode the user singing signal”); encoding, by a music encoder, the second training audio output (Xiong paragraph [0013] “and use a music encoder in the trained singing optimization model to encode the reference singing signal and the accompaniment signal”); constructing, by an audio decoder and based on the encoded first training audio output and the encoded second training audio output, a composite audio output (Xiong paragraph [0013] “a decoding unit, configured to use a spectrum decoder in a trained singing optimization model to decode based on the encoding of the user singing signal, the encoding of the reference singing signal, and the encoding of the accompaniment signal, to obtain a spectrum signal of an optimized song; a conversion unit, configured to convert the spectrum signal of the optimized song into the audio of the optimized song.”);
Liu, Rey, Kim and Xiong are in the same field of endeavor, namely computer graphics, especially audio related processing. Xiong teaches a neural network based song generation method based on singing voice and accompany music to enrich audio generation and improve optimization effect (Xiong paragraph [0061] “Therefore, the
song generation method of this embodiment can optimize the user's singing voice differently based on different reference songs, effectively enriching the song generation method and improving the optimization effect.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Xiong with the method of Liu in view to Rey and Kim to improve optimization effect of audio source separation model.
Regarding claim 14, claim 14 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2.
Regarding claim 18, claim 18 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2.
Claim(s) 10 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL Liu et al. (“MusicFace: Music-driven Expressive Singing Face Synthesis”), hereinafter as Liu, in view of de la Rey et al. (US 20230130844 A1), hereinafter as Rey, further in view of Kim et al. (US 20240338874 A1), hereinafter as Kim, and NPL Zhuang et al. (“Music2Dance: DanceNet for Music-driven Dance Generation”), hereinafter as Zhuang.
Regarding claim 10, Liu in view of Rey and Kim teaches The method of Claim 1, but fail to teach wherein the one or more trained avatar animation models comprise a trained dance model, the method further comprising: generating, by a trained non-vocal encoder of the trained dance model, a set of encoded non-vocal features; generating, by a trained motion encoder of the trained dance model, a set of encoded motion features; and generating, by a trained dance-style classifier of the trained dance model, a dance classification based on the encoded non-vocal features; generating, by a motion decoder of the trained dance model, a dance animation for the avatar based on the set of encoded motion features and the dance classification. Zhuang teaches wherein the one or more trained avatar animation models comprise a trained dance model (Zhuang Page 1, abstract, “we propose a novel autoregressive generative model, DanceNet, to take the style, rhythm and melody of music as the control signals to generate 3D dance motions with high realism and diversity.”), the method further comprising: generating, by a trained non-vocal encoder of the trained dance model, a set of encoded non-vocal features (Zhuang teaches a musical encoder as non-vocal encoder, Page 6, Figure 2, “musical context-aware encoder”); generating, by a trained motion encoder of the trained dance model, a set of encoded motion features (Zhuang Page 6, Figure 2, “motion encoder” and Page 9, first paragraph, “Motion encoder. We stack two ”Conv1D+Relu” module as motion encoder to encode the past k frames. The convolution kernel is set to 1, which ensures that each frame motion code(512 channels) is independent.”); and generating, by a trained dance-style classifier of the trained dance model, a dance classification based on the encoded non-vocal features (Zhuang teaches a musical style classifier, it takes the input of music feature as an encoded non-vocal feature. Page 6, Figure 2, “First we extract the musical rhythm and melody (music features) and classify the musical style by the musical style classifier.”); generating, by a motion decoder of the trained dance model, a dance animation for the avatar based on the set of encoded motion features and the dance classification (Zhuang Page 6, Figure 2, “DanceNet takes the musical style, rhythm and melody as control signals to generate dance motion. DanceNet consists of four parts: musical context-aware encoder, motion encoder, residual motion control stacked module, motion decoder.”).
Liu, Rey, Kim and Zhuang are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Zhuang teaches music guided DanceNet to achieve realistic and diverse avatar animation (Zhuang Page 3, third paragraph, “The results show that our method can achieve SOTA result, and the dance motions generated by our method are not only realistic and diverse, but also are music-consistent.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Zhuang with the method of Liu, Rey and Kim to achieve realistic and diverse avatar animation.
Regarding claim 24, claim 24 has similar limitations as claim 10, therefore it is rejected under the same rationale as claim 10.
Claim(s) 11, 16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL Liu et al. (“MusicFace: Music-driven Expressive Singing Face Synthesis”), hereinafter as Liu, in view of de la Rey et al. (US 20230130844 A1), hereinafter as Rey, further in view of Kim et al. (US 20240338874 A1), hereinafter as Kim, NPL Zhuang et al. (“Text/Speech-Driven Full-Body Animation”), hereinafter as Zhuang2, and NPL Zhuang et al. (“Music2Dance: DanceNet for Music-driven Dance Generation”), hereinafter as Zhuang.
Regarding claim 11, Liu in view of Rey and Kim teach The method of Claim 1, and further teach wherein the trained one or more animation models comprise: ……a facial-expression model for animation a face of the avatar (Liu, Page 4, Figure 3, “The Decoder contains three downstream generators, including Expression Generation Network (EGN)”)……
Liu in view of Rey and Kim fail to explicitly teach a lip-sync model for animating a mouth of the avatar;…… and a dance model for animating a body of the avatar. Zhuang2 teaches a lip-sync model for animating a mouth of the avatar (Zhuang2 Page 2, Figure 1, “Given a section of text and speech, the human face and the body are synthesized through two branches respectively. One branch adopts a learning-based method to synthesize lip motions and expressions”);…… and a dance model for animating a body of the avatar (Zhuang2 Page 2, Figure 1, “while the other branch uses a database retrieval-based method to synthesize skeleton motion. Full-body animation is then obtained through skinning and rendering.”).
Liu, Rey, Kim and Zhuang2 are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Zhuang2 teaches a method to simultaneously synthesize face and body animation to achieve high quality avatar animation (Zhuang2 Page 1, left column, first paragraph, “We adopt a learning-based approach for synthesizing facial animation and a graph-based approach to animate the body, which generates high-quality avatar animation efficiently and robustly. Our results demonstrate the generated avatar animations are realistic, diverse and highly text/speech-correlated.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Zhuang2 with the method of Liu, Rey and Kim to achieve realistic and diverse avatar animation.
Liu in view of Rey, Kim and Zhuang2 fail to teach the trained one or more animation models… dance model. Zhuang teaches the trained one or more animation models… dance model (Zhuang Page 1, abstract, “we propose a novel autoregressive generative model, DanceNet, to take the style, rhythm and melody of music as the control signals to generate 3D dance motions with high realism and diversity.”).
Liu, Rey, Kim, Zhuang2 and Zhuang are in the same field of endeavor, namely computer graphics, especially in the field of audio driven avatar animation. Zhuang teaches music guided DanceNet to achieve realistic and diverse avatar animation (Zhuang Page 3, third paragraph, “The results show that our method can achieve SOTA result, and the dance motions generated by our method are not only realistic and diverse, but also are music-consistent.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the dance model of Zhuang2 with the DanceNet of Zhuang, with the method of Liu, Rey and Kim to achieve realistic and diverse avatar animation.
Regarding claim 16, claim 16 has similar limitations as claim 11, therefore it is rejected under the same rationale as claim 11.
Regarding claim 20, claim 20 has similar limitations as claim 11, therefore it is rejected under the same rationale as claim 11.
Allowable Subject Matter
Claims 3, 6-8, 12 and 21-22 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claims 3 and 21, the closest prior art of Rey teaches a self-iterative supervised audio source separation model. However, Rey fails to teach the combined limitation as a whole “wherein the trained audio source separation model is further defined by a supervised learning training process comprising, for each of the plurality of training audio inputs: providing a predetermined training avatar animation; determining, by a pretrained instance of the one or more trained avatar animation models and from the first training audio output and the second training output, a corresponding training avatar animation output; and updating the source separation model based on a similarity between the predetermined training avatar animation and the corresponding training avatar animation output.”. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claims 3 and 21 are considered to allowable.
Regarding claims 6 and 22, the closest prior art of Kim teaches an emotion classification based on text input, and further teaches the animation of avatar based on emotion features. However, Kim fails to teach the combined limitations as a whole “wherein the trained emotion encoder outputs one or more encoded emotion features for animating the facial expression of the avatar; and the trained dance encoder outputs or more encoded dance features for animating a body movement of the avatar”. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claims 6 and 22 are considered to allowable.
Claims 7-8 contain allowable subject matter because they depend on claim 6 that contains allowable subject matter.
Regarding claim 12, the closet prior art of Kim teaches an user interface to generate avatar animation using neural network. However, Kim fails to teach the combined limitation below as a whole “further comprising classifying a first portion of the user input as emotion-related input and a second portion of the user input as movement-related input”. The prior art of Zhuang2 teaches a text and audio based full body simulation with lip movement generation, facial expression generation and motion graph construction based body animation. However, Zhuang2 fails to teach the combined limitation above as a whole. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claim 12 is considered to allowable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gorumkonda et al. (US 20250238987 A1) teaches generating dance pose based on music input.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/XIAOMING WEI/Examiner, Art Unit 2611
Prosecution Insights