Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination based on the amendment filed on 05/11/2026.
Claims 1-3, 5-7, 11-13, 15-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,further in the view of Jose M. Alvarez and Mathieu Salzmann. 2016. Learning the number of neurons in deep networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16).
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,further in the view of NAEEM SYED, ADNAN ANWAR, ZUBAIR BAIG, and SHERALI ZEADALLY. "Artificial Intelligence as a Service (AlaaS) for Cloud, Fog and the Edge: State-of the- Art Practices." (2018).
Claims 9-10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, further in the view of H. H. Bothe, "Audio to audiovideo speech conversion with the help of phonetic knowledge integration," 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, Orlando, FL, USA, 1997, pp. 1632- 1637 vol.2, further in the view of Waterworth, Nicholas. "Speech Processing in Computer Vision Applications." (2020).
This action is Final rejection.
Priority
Acknowledgment is made of applicant's claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 2022112953012, filed on 10/21/2022.
Information Disclosure Statement
The information disclosure statement submitted on 11/11/2022 was failed. All documents are considered. See the attached file.
Response to Arguments
Following Applicants amendments to the claim, 35 USC 112 rejection is Withdrawn.
Following Applicants amendments to the claim, 35 USC 101 rejection is Withdrawn.
Applicants Argument: Applicant’s arguments directed the 103 rejection are based on newly amended subject matter.
Examiner’s Response: All arguments are addressed in the 103 rejection of the claims below.
Applicant’s argument: With regard to the §103 rejection of claims 1-3, 5 and 6, Applicant respectfully traverses on the ground that the teachings of Qu and Teerapittayanon fail to disclose or suggest each and every limitation of claims 1-3, 5 and 6, arranged as recited in those claims, and on the further ground that there is no suggestion or motivation to modify the collective teachings of Qu and Teerapittayanon in a manner that would reach these limitations. Lee and the other cited references are similarly deficient .
Examiner’s Response: Examiner agrees the newly amended claim limitation on claim 1, is not explicitly teach by the combined model of Qu and Teerapittayanon, so a newly 35 USC 103 rejection is written as shown below on 103 rejection section. Since a new prior art is integrated, a new motivation is also added.
While the combined model of Yi- Qu-Teerapittayanon teaches all the limitations of claims 1-3, 5 and 6 and the rest of the claims are also taught by the newly combined prior art as it is mapped below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, 11-13, 15-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,
As of claim 1, Yi teaches determining, in a processor-based machine learning system, a target model for processing data, the target model comprising at least a portion of a mixed reality machine learning model (abstract, a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V).
dividing, in the processor-based machine learning system, the target model into a plurality of modules that implement different tasks, the plurality of modules comprising an audio-based parameter generation module of the mixed reality machine learning model, a video-based parameter generation module of the mixed reality machine learning model, an initial rendering module of the mixed reality machine learning model, and a rendering tuning module of the mixed reality machine learning model; (Fig.1, abstract , In this paper, we address this problem by proposing a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V), expression and lip synchronization (by considering both A and V)
PNG
media_image1.png
389
763
media_image1.png
Greyscale
Yi teaches using of a deep neural network and it contains different models for different tasks. As it is shown 1,2,3, 4 is mapped to an audio-based parameter generation module, a video-based parameter generation module, an initial rendering module, and a rendering tuning module respectively.
configuring, in the processor-based machine learning system, the mixed reality machine learning model with the target module in the arrangement position, at least in part by coupling (i) an output of the audio-based parameter generation module implemented in the at least one server of the cloud, and (ii) an output of the video-based parameter generation module implemented in the at least one server of the cloud, to respective inputs of the initial rendering module implemented in the edge device, and coupling an output of the initial rendering module to an input of the rendering tuning module implemented in the at least one server of the cloud, wherein an output of the rendering tuning module provides at least a portion of a mixed reality output of the mixed reality machine learning model; ( Fig.1
PNG
media_image1.png
389
763
media_image1.png
Greyscale
). According to Fig. 5 of the claimed invention, Fig.1 is mappeds as follow, the output of label 1 and 2 is coupled to label 3, and used as input in label 3, then the output of label 3 is also used as input for the last label for final rendering( Fine tuning)
executing, in the processor-based machine learning system, the mixed reality machine learning model to generate the mixed reality output.( Abstract, we address this problem by proposing a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V), expression and lip synchronization (by considering both A and V)). As shown on Fig.1 above the it outputs generated personalized talking face video (label 5).
While Yi does not explicitly teach the processor-based machine learning system comprising an edge device and at least one server of a cloud; the plularity of modeles were performed in the edge device , server of the cloud, determining, in the processor-based machine learning system, a quantity of parameters of a target module in the plurality of modules and a size of transmission data related to the target module; determining, in the processor-based machine learning system, an arrangement position of the target module based on the quantity and the size.
Teerapittayanon teaches the processor-based machine learning system comprising an edge device and at least one server of a cloud; (Section 1, "Introduction, An example of one such distributed approach is to combine a
small NN1 model (less number of parameters) on end devices and a larger NN model (more number of parameters) in the cloud The small model at an end device can quickly perform initial feature extraction, and also classification if the model is confident. Otherwise, the end device can fall back to the large NN model in the cloud, which performs further processing and final classification).
Yi and Teerapittayanon considered to be analogous for the claimed invention since they focus on using a neural network model to create animation and task offloading. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filling date to include edge device and a server of a cloud into Yi model to partition tasks in either edge device or a server of a cloud.
The motivation would have been to perform task offloading of diffenet neural network model of Yi based on Teerapittayanon teaching of a single DNN properly trained can be mapped onto a distributed computing hierarchy to meet the accuracy, communication and latency requirements of a target application while gaining inherent benefits associated with distributed computing such as fault tolerance and privacy (Teerapittayanon, "conclusion")
While the modified model does not explicitly teach plularity of modeles were performed in the edge device, server of the cloud, determining, in the processor-based machine learning system, a quantity of parameters of a target module in the plurality of modules and a size of transmission data related to the target module; determining, in the processor-based machine learning system, an arrangement position of the target module based on the quantity and the size.
While Qu teaches determining , in the processor-based machine learning system, a quantity of parameters of a target module in the plurality of modules and a size of transmission data related to the target module(section 1, "introduction", The process of task offloading is generally affected by a variety of factors in different areas, e.g., user preferences, wireless communication channels, network connection quality, mobility of loT devices device and availability of edge/cloud servers. Therefore, making the optimal decision is the most critical issue for edge offloading. It needs to dynamically decide whether the task should be offloaded to the edge server or cloud server ... Application Partition: Since different tasks usually have different amounts of computation and communication, before performing task offloading operation, it is better to divide the task into a workflow with multiple associated subtasks or as a series of independent subtasks, and then offload the subtasks separately. Among them, some subtasks are executed on the loT devices, the others are executed on the relatively powerful server, making full use of the server resources, thereby greatly reducing the load of the loT devices and improving their endurance .Resource Allocation: After the offloading decision is made, resources need to be allocated, including computing power, communication bandwidth, and energy consumption.)
Determining, in the processor-based machine learning system, an arrangement position of the target module based on the quantity and the size( section 1 "Introduction", To tackle the above challenges, we design an edge-cloud offloading framework in this paper, where loT devices can choose to shift their computing tasks either to edge servers or cloud servers. Edge servers make offloading decisions based on task information for each device, reducing latency and energy consumption).
Examiner note: As it was cited above offloading decisions are affected by different parameters and the tasks are divide by their amount of computation and
communication. So, amount of computation is interpreted as size and other factors like amount of communication is interpreted as quality, so based on this a person of ordinary skill in art can try to perform the machine learning modules of initial rendering on the edge device an, the rest of the module on the server of the cloud based on their computation requirment as Qu teaches above.
Claims 11 and 20 are in the same scope to that of claim 1 with additional elements of a electronic device with a processor and memory, and clam 20 include computer program product comprising non-transitory computer-readable medium, to cause the machine to perform the actions, while Yi performs Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose, using a deep neural network model and it is obvious that a computer is used to perform generation of animation video from a audio and video input, and a computer inherelty contains a memory and processor. Therefore claims 11 and 20 is also rejected under the same rational to that of claim 1.
As of claim 2, the modified model teaches all the limitation of claim 1, and QU also teaches the target model comprises: determining a code of the target model; (Section B, " Intelligent Offloading Decision-Making", Besides, Neurosurgeon [30] was a fine-grained partitioning method that can find the optimal dividing point in DNNs according to different factors, and made full use of the resources of cloud servers and mobile devices to minimize the computational delays or energy consumption in loT environments ).The different factors are considered as a code to find the optimal dividing points in DNNS for model arrangement. Determining a group of neural network models in the target model by analyzing the code ( Section B," Intelligent
Offloading Decision-Making", Deep learning methods refer to the classification of
the input task information through the multi-layer neural network to determine the
final offloading position. Huang et al. [28] provided an algorithm that adopted
distributed deep learning to solve the offloading problem of mobile edge networks. It used parallel and distributed DNNs to produce offloading decisions
and achieved good results.)
Claim 12 is in the same scope as claim 2, so claim 12 is rejected under the same rational as of claim 2.
As of claim 3, the modified model teaches all the limitation of claim 2, and Teerapittayanon also teaches determining a task of each neural network model in the group of neural network models;(Section 1, "Introduction, An example of one such distributed approach is to combine a small NN1 model (less number of parameters) on end devices and a larger NN model (more number of parameters) in the cloud. The small model at an end device can quickly perform initial feature extraction, and also classification if the model is confident. Otherwise, the end device can fall back to the large NN model in the cloud, which performs further processing and final classification).
determining neural network models that implement the same task as a
module.( Section1, "introduction", Multiple models at the cloud, the edge and the
device need to be learned jointly to allow coordinated decision making. Computation already performed on end device models should be useful for further processing on edge or cloud models).
Claim 13 is in the same scope as claim 3, so claim 13 is rejected under the same rational as of claim 3.
As of claim 5 and 6 the modified model teaches all the limitation of claim 1, and Qu also teaches claim 5 and 6, as follow As of claim 5 and 6 contains a contingent limitations, Qu teaches claim 5 limitation of determining a ratio of the size to the quantity arranging, if the ratio is greater than a threshold, the target module in the edge device, and claim 6 limitation arranging, if the ratio is less than or equal to the threshold, the the target module in the at least one server ( section A, "Inner Model", As shown in Fig. 3, the inner model is based on a parallel Deep Reinforcement Learning (DRL) algorithm. We apply a classic reinforcement learning method named Q-learning, in which we input environmental parameters, labeled initial parameters and
workflow x into the inner model. We use ai to represent the offloading decision of
the i-th subtask of the workflow, which is defined as:
PNG
media_image2.png
115
628
media_image2.png
Greyscale
where ai = 0, 1, and 2 indicate that the i-th subtask is executed locally on the loT
device, the edge server, and the cloud server, respectively). Qu determine an
offloading decision for each DNN with different method but the same result of
arrange the tasks on edge or cloud server based on threshold).
Claims 15 and 16 are in the same scope as claims 5 and 6, so claims 15 and 16 are rejected under the same rational as of claim 5 and 6 resepectively.
As of claim 7, the modified model teaches all the limitations of claim 6, and Yi also teaches the target model is a voice-based avatar generation, the voice-based avatar generation model comprises an audio-based avatar parameter generation module, an avatar video-based avatar parameter generation module, an initial drawing module, and a fine-tuning drawing module(Fig. 3 and section 3, In this paper, we tackle the problem of generating high quality talking face video, when given an audio speech of a source person and a short video (about 10 seconds) of a target person. In addition to learn the transformation from the audio speech to lip motion and face expression, our talking face generation also considers the personalized talking behavior (i.e., head pose) of the target person
PNG
media_image1.png
389
763
media_image1.png
Greyscale
). According to Fig. 5 of the claimed invention, Fig.1 is mappeds as follow, the output of label 1 and 2 is coupled to label 3, and used as input in label 3, then the output of label 3 is also used as input for the last label for final rendering( Fine tuning).
Claim 17 is in the same scope as claim 7, so claim 17 is rejected under the same rational as of claim 7.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,further in the view of Jose M. Alvarez and Mathieu Salzmann. 2016. Learning the number of neurons in deep networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16).
As of claim 4, the modified model teaches all the limitation of claim 1, but they do not explicitly teach, determining a number of neurons in the target module and determining the quantity of the parameters based on the number of neurons.
While Alvarez teaches determining a number of neurons in the target module (Section 3, "Deep model selection: learning with structure sparsity", We now introduce our approach to automatically determining the number of neurons in each layer of a deep network while learning the network parameters).
determining the quantity of the parameters based on the number of neurons.(
Section 3, "Deep model selection: learning with structure sparsity", A general deep
network can be described as a succession of L layers performing linear operations on
their input, intertwined with non-linearities, such as Rectified Linear Units (ReLU) or
sigmoid, and, potentially, pooling operations. Each layer I consists of N1 neurons, each
of which is encoded by parameters 0n1 = [wn1, bn1 ], where wn1 is a linear operator acting on the layer's input and bn1 is a bias. Altogether, these parameters form the parameter set θ = { θ}1<=|<=L, with θ = { θ n } 1<=n<=NI ).
Alvarez is considered to be analogous to the claim invention because they use parameters to analyze models. Therefore it would be obvious to one of the ordinary skill in the art before the effective filing data to have applied Alvarez teaching of determining number of neurons as based on the number of neurons determining parameters on the target model of the modified model.
The motivation would have been to reduce redundant parameters and improves
the accuracy of network by automatically determining the number of neurons, which will
help to reduce the memory usage and computational cost in order to keep the model
arrangement reasonable (Abstract, Alvarez).
Claim 14 is in the same scope as claim 4, so claim 14 is rejected under the same rational as of claim 4.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448- 3459, Sept. 2021,further in the view of NAEEM SYED, ADNAN ANWAR, ZUBAIR BAIG, and SHERALI ZEADALLY. "Artificial Intelligence as a Service (AlaaS) for Cloud, Fog and the Edge: State-of the- Art Practices." (2018).
As of claim 8, the modified of Yi Teerapittayanon and Qu teaches all the limitations of claim 7, but the modified does not explicitly teach wherein the audio-based avatar parameter generation module, the avatar video-based avatar parameter generation module, and the finetuning drawing module are arranged in the server, and the initial drawing module is arranged in the edge device.
While Naeem teaches wherein the audio-based avatar parameter generation
module, the avatar video-based avatar parameter generation module, and the fine-tuning drawing module are arranged in the server, and the initial drawing module is arranged in the edge device (Section 5.2,
PNG
media_image3.png
510
530
media_image3.png
Greyscale
Fig. 8. AlaaS services for edge/fog computing in a data-driven offloading scenarios
or privacy preserving circumstances requiring FL hierarchical aggregation
The offloading algorithms should consider the factors mentioned previously to make
the optimal offloading decisions. For efficient processing of these Al related tasks at
the edge and fog, the AlaaS platform can be leveraged to provide services such as
model training, serving and monitoring (SLA/SLO parameters), pre-trained model
access, data storage and management, optimizing models for different processing
layers (edge, fog) and data visualization and analytics services for clients .... Some
examples of Al based offloading techniques are lightweight Al models that can be
trained at the cloud and optimized for edge and deployed at the edge to do simpler
inference tasks. For example, TinyML models which can run on constrained devices [102] can be first trained on the cloud AlaaS deployment and then optimized and
deployed at the edge to provide inference services).
Naeem is considered to be analogues to the claim inventions, because they focused on arranging tasks. Therefore it be obvious to try for some one of ordinary skill in the art before the effective failing date using Syed teaching of task of loading on the server and on the edge on the parameters like size, consumption cost as the combined model teaches for better latency and to make the optimal offloading decisions.
The motivation would have been to have optimal offloading decisions by
considering factors task complexity, latency, energy consumption, network
conditions, device resources and privacy concerns, so that effectively offloading
services to be performed either locally at the edges, fog or on the cloud
resource(Naeem, section 5.2).
Claim 18 is in the same scope as claim 8, so claim 18 is rejected under the same rational as of claim 8.
Claims 9-10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yi, Ran, et al. "Audio-driven talking face video generation with learning-based personalized head pose." arXiv preprint arXiv:2002.10137 (2020) in the view of in in the view of S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017.,further in the view of G. Qu, H. Wu, R. Li and P. Jiao, "OMRO: A Deep Meta Reinforcement Learning Based Task Offloading Framework for Edge-Cloud Computing," in IEEE Transactions on Network and Service Management, further in the view of H. H. Bothe, "Audio to audiovideo speech conversion with the help of phonetic knowledge integration," 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, Orlando, FL, USA, 1997, pp. 1632- 1637 vol.2, further in the view of Waterworth, Nicholas. "Speech Processing in Computer Vision Applications." (2020).
As of claim 9, the modified model of Yi, Teerapittayanon,Qu and Naeem
teach all the limitations of claim 8, and Yi also teaches receiving voice data (Fig.1 and section 3.2, We use both the audio information and the 3D face geometry information extracted from input video to establish a mapping from the input audio to the facial expression and head pose).
While the modified model does not explicitly teach converting the voice data into a video for an avatar through the voice-based avatar generation model; generating text information corresponding to the voice data based on the voice data; and displaying the video of the avatar and the text information.
While Bothe converting the voice data into a video for an avatar through the voice-based avatar generation model;( Section 1 figure 1. LIPPS converts acoustic speech signals from a microphone input into a facial animation on a computer screen with corresponding movements of the mouth region (lips, teeth, tongue). As it showed above on Fig. 1, a voice was changed to a facial animation.
displaying the video of the avatar and the text information.(section 1,
PNG
media_image4.png
238
283
media_image4.png
Greyscale
The LIPPS terminal is integrated on a multi-media Pentium@computeru nder Windows
9S%s shown in figure 2. It is served by keyboard or mouse. If used as a training aid for
lipreading, the user may enter the acoustic utterances to be processed with the help of
an integrated microphone or by keyboard, or may store the recorded or processed data
on hard disk for later display, ... section 2, a facial image of the actual speaker is
displayed on the screen).
Bothe is considered to be analogous to the claim invention, because they focus on model arrangement. Therefore it would be obvious to one of the ordinary skill in
the art before the effective filing data to have applied Bothe teaching of using a voice to
create animation on the combined model to visualize the video of avatar.
The motivation would have been by using a phoneme recognizer, converting
speech utterances into corresponding sequences of phoneme probability vectors, and a
connected phoneme sequence to video conversion, creating a realistic computer
animation(Bothe, section 1 ).
However the modifed model of Yi – Teerapittayanon- Qu -Naeem -Bothe does not explicitly teach generating text information corresponding to the voice data based on the voice data.
While Nicholas teaches generating text information corresponding to the
voice data based on the voice data;( section 1.1, Speech Recognition handles
listening to a given speaker and interpreting what words are being said and turning
them into their textual representation).
Nicholas is considered to be analogous to the claim invention, because they focus on model analysis of input data. Therefore it would be obvious to one of the
ordinary skill in the art before the effective filing data to have applied Nicholas teaching
of generating a text from a voice on the target model in order to find a video and voice
by using the target model.
The motivation would have been to create a better voice detection by removing
noise from the speech sample so that a better textual information can be extracted from
the voice using speech recognition ( Nicholas, Conclusion).
Claim 19 is in the same scope as claim 9, so claim 19 is rejected under the same rational as of claim 9.
As of claim 10, themodified model teach all the limitations of claim 9 and Nicholas also teaches wherein generating the text information comprises: determining a voice feature based on the voice data; and acquiring the text information based on the voice feature ( section 1.1, Speaker Recognition is a class
comparison problem to be able to identify a speaker from the characteristics in a
given spoken phrase. The work in Speaker Recognition revolves around creating
unique representations of speech samples and training networks to compare with
extracted features .... Speech Recognition handles listening to a given speaker
and interpreting what words are being said and turning them into their textual
representation).
Conclusion
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure.
Shang; Chong (US 20210027511 A 1, Date Published 2021-01-28), this application is similar to the claimed invention since it disclosed a system and method for animation generation using input audio data by generating an output based on the generated final prediction.
Tajima; Kaori (US 20190260827 A 1, Date Published 2019-08-22), this application is similar to the claimed invention since it disclosed model partitioning for To appropriately allocate program processing to edge servers in an edge computing system and a backend server.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABRHAM A. TAMIRU whose telephone number is (571)272-6987. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571 272 4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABRHAM ALEHEGN TAMIRU/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights