DETAILED ACTION
This Office Action is in response to the correspondence filed by the applicant on 2/3/2025.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 21-40 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of US PAT 12,205,577. Although the claims, at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are rejected as being unpatentable over the claims of the US PAT. Please see below for the mapping in the table, where the bolded limitations indicate the corresponding limitations between the US PAT and instant application.
Instant application
US PAT 12,205,577
21. A computer-implemented method, comprising:
receiving input data comprising a natural language user input;
determining natural language data responsive to the natural language user input;
generating video data corresponding to the natural language data;
generating output data comprising the video data; and
presenting the output data.
1. A computer-implemented method comprising:
receiving first input audio data corresponding to a first spoken natural language input;
determining the first spoken natural language input includes:
a first portion requesting first content be output; and
a second portion representing a first parameter to be used instead of a second parameter included in the first content;
generating, by a content generator component, first natural language data corresponding to the first content updated to include the first parameter instead of the second parameter;
generating, by an image generator component, first image data representing the first natural language data;
generating, by an avatar generator component, first video data including a representation of an avatar corresponding to the first natural language data; and
generating, by a response generator component, first output data including the first image data, first output audio data including first synthesized speech corresponding to the first natural language data, and the first video data, wherein the first output data synchronizes display of the representation of the avatar with output of the first synthesized speech.
Other independent claim 31 is also similar to the independent claim 10 of the US PAT.
With respect to the dependent claims, each claim maps to a corresponding dependent claim of the US PAT or are found within the scope of the independent claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21-40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The independent claims 21 and 31 recite, “receiving input data comprising a natural language user input; determining natural language data responsive to the natural language user input; generating video data corresponding to the natural language data; generating output data comprising the video data; and presenting the output data.”
The limitations of the recited steps, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than “at least one processor; and at least one memory”, nothing in the claim element precludes the step from practically being performed in the mind. For example, a person can receive a natural language input, determine how to generate a video/image/facial expression, and express the facial expression along with verbalizing the natural language input. The limitations, as drafted, are processes that, under its broadest reasonable interpretation, cover performance of the limitations in the mind.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claims only recite additional elements – “at least one processor; and at least one memory”. The additional elements in both steps is recited at a high-level of generality (i.e., as a generic processor performing a generic computer function of the recited steps) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a processor to perform the recited steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Regarding the dependent claims, claims 22 and 32 recite, “an identifier of a language”; claims 23 and 33 recite, “generating the video speaking the language”; claims 24 and 34 recite, “emotion identifier and generating the video based on the identifier”; claims 25 and 35 recite, “generating an avatar exhibiting the emotion”; claims 26 and 36 recite, “determining an emotion using a machine learning model”; claims 27 and 37 recite, “two-dimensional image for generating a video”; claims 28 and 38 recite, “using a three-dimensional model for mapping the facial image”; claims 29 and 39 recite, “generating audio data corresponding to the input data”; and claims 30 and 40 recite, “configurating the video and the audio data corresponding to an output time duration.”
Even though the disclosed invention is described in the specification as improving computer technology, the claim provides no meaningful limitations such that this improvement is realized. Therefore, the claim does not amount to significantly more than the abstract idea itself.
Accordingly, the limitations of the Claims, whether considered individually or as an ordered combination, are not sufficient to add significantly more to improve technological functionality. As such, claims 21-40 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 21, 24-26, 28-31, 34-36, and 38-40 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ROCHE (US 10,521,946 B1).
REGARDING CLAIM 21, ROCHE discloses a computer-implemented method, comprising:
receiving input data comprising a natural language user input (ROCHE Col 30:46-51 – “At 1102, the animation service 104A may receive text. The text may be received from a user or from another source, such as a prewritten document. The text may include words that are to be spoken by an avatar that is animated and performs gestures during speaking of the text, where the gestures are associated with a context of the text.”);
determining natural language data responsive to the natural language user input (ROCHE Col 31:4-7 – “At 1108, the animation service 104A may receive the speech markup data including at least the phonic symbols, the words, and the time codes. For example, the speech markup data may be received as structured data in database or other repository enabling extraction of components of the data.”);
generating video data corresponding to the natural language data (ROCHE Col 31:24-28 – “At 1110, the animation service 104A may generate one or more first animation sequences of facial features, such as a mouth, of the avatar based at least in part on the one or more phonic symbols and the time codes associated with the phonic symbols.”);
generating output data comprising the video data (ROCHE Col 31:44-57 – “At 1114, the animation service 104A may output the audio data synchronized with output of combined animation sequences of the avatar that include the one or more first animation sequences and the one or more second animation sequences. The output may be a file, such as a downloadable animation file that includes the animation, the sound, or both. In various examples, the combined animation sequences may include a blending of a first animation sequence and a second animation sequence to create at least part of the second gesture sequence. The output may include a virtual environment that includes at least one object in the virtual environment along with the avatar. The virtual environment may include another avatar and/or other objects, which may be associated with tags as discussed above.”); and
presenting the output data (ROCHE Col 31:44-57 – “At 1114, the animation service 104A may output the audio data synchronized with output of combined animation sequences of the avatar that include the one or more first animation sequences and the one or more second animation sequences.”).
REGARDING CLAIM 24, ROCHE discloses the computer-implemented method of claim 21, further comprising:
receiving an identifier of an emotion to be exhibited in the video data (ROCHE Col 33:8-25 – “At 1152, the animation service 104A may determine whether to apply an emotion value to the animation sequence. The emotion values may be an input for the audio data or an outcome of setting that generate the audio data. The animation service 104A may determine to apply an emotion value when an emotion value is available, such as being returned with the speech markup data. The emotional parameter may be input by a user for use in creation of the speech markup data or may be created as a product of the speech markup data, such as based on settings employed by text-to-speech service 104E or the ASR service 104F that creates the speech markup data. When applied by the animation service 104A, the emotion value may influence an animation sequence, movement the standard pose, and/or other attributes of the animation. When an emotion value is available or determined (following the “yes” route from the decision operation 1152, then the process 1150 may continue to an operation 1154.”); and
generating the video data based at least in part on the identifier (ROCHE Col 33:31-47 – “At 1156, the animation service 104A may generate the animation sequences of the body of the avatar based at least in part on the words and the emotion. For example, the animation manager 204A may modify movement range of skeletal components based on an emotional parameter, such as increasing movement range for emotional parameters associated with “happy” or “excited” and reducing movement range for emotional parameters associated with “sad” or “tired”. The emotion value may be used to determine general animation, such as by causing selection of first animations for a first word for a first emotion value, and causing selection of second animations for the first word for a second emotion value. In some examples, the emotion value may change during a duration of a speech, such as by starting as a first emotion value associated with a first group of words and later changing to a second emotion value associated with a second group of words.”).
REGARDING CLAIM 25, ROCHE discloses the computer-implemented method of claim 24, further comprising:
generating the video data to comprise an avatar exhibiting the emotion (ROCHE Col 33:31-47 – “At 1156, the animation service 104A may generate the animation sequences of the body of the avatar based at least in part on the words and the emotion. For example, the animation manager 204A may modify movement range of skeletal components based on an emotional parameter, such as increasing movement range for emotional parameters associated with “happy” or “excited” and reducing movement range for emotional parameters associated with “sad” or “tired”. ”).
REGARDING CLAIM 26, ROCHE discloses the computer-implemented method of claim 21, further comprising:
processing, using a machine learning model, the natural language data to determine an emotion corresponding to the natural language data (ROCHE Col 12:12-40 – “In some configurations, the animation manager 204A may receive or determine an emotion value associated with the text, and may use the emotion value (e.g., an parameter, weight, or other designator) to influence the animation sequence, movement, the standard pose, and/or other attributes of the animation. The emotion value may be a numeric value or a non-numeric value, such as a label (e.g., happy, sad, excited, etc.), or another designation. The emotion value may be received from the ASR service 104F, for example. The animation manager 204A may determine the emotion values when unpacking data from the ASR service 104F or another service that provides emotion values. For example, the ASR service 104F may include emotion values assigned via a look up table or other data repository, which may be assigned to individual words, phrases, idioms, sentences, and/or other portions of speech or entire speech. In various embodiments, the ASR service 104F may include emotion values associated with certain words. In some embodiments, the ASR service 104 may utilize machine learning algorithms to determine or generate emotion values. For example, a machine learning algorithm may analyze imagery of facial movements corresponding to actual spoken words to determine emotion values, possibly using training data and/or a feedback look to modify emotion values over time. The machine learning algorithm may analyze a context of words to determine emotional values for associated words, and may compare the determined emotional values to observation data obtained by analysis of real speakers, for example.”; Col 9:43-58 –“ The automatic speech recognition service 104F is configured to provide automatic speech recognition (ASR) functionality for converting speech to text, and to provide natural language understanding (NLU) to recognize the intent of the text.”; Col 12:41-47 – “In accordance to some embodiments, the animation manager 204A may determine emotion values using the look up table, machine learning, and/or from other services without requiring the emotion values to be provided by the ASR service 104F. For example, the animation manager 204A may, at times, generate and/or apply an emotion value to modify animation of an avatar.”);; and
generating the video data based at least in part on the emotion (ROCHE Col 12:12-40 – “In some configurations, the animation manager 204A may receive or determine an emotion value associated with the text, and may use the emotion value (e.g., an parameter, weight, or other designator) to influence the animation sequence, movement, the standard pose, and/or other attributes of the animation. The emotion value may be a numeric value or a non-numeric value, such as a label (e.g., happy, sad, excited, etc.), or another designation. The emotion value may be received from the ASR service 104F, for example.”; Col 33:31-47 – “At 1156, the animation service 104A may generate the animation sequences of the body of the avatar based at least in part on the words and the emotion. For example, the animation manager 204A may modify movement range of skeletal components based on an emotional parameter, such as increasing movement range for emotional parameters associated with “happy” or “excited” and reducing movement range for emotional parameters associated with “sad” or “tired”. The emotion value may be used to determine general animation, such as by causing selection of first animations for a first word for a first emotion value, and causing selection of second animations for the first word for a second emotion value. In some examples, the emotion value may change during a duration of a speech, such as by starting as a first emotion value associated with a first group of words and later changing to a second emotion value associated with a second group of words.”).
REGARDING CLAIM 28, ROCHE discloses the computer-implemented method of claim 21, where generating the video data comprises:
determining, using a three-dimensional model (ROCHE Col 12:61-13:14– “The animation manager 204A may be designed to interact with one or more users and/or interact with the virtual environment presented by the VR/AR application. The animation manager 204A may generate animations that cause the avatar to interact with objects within a virtual environment, appear to interact with objects in the real world (e.g., look a person or object identified at a specific location in the real world via image capture from one or more cameras, etc.), or both.”; Col 27:34-44 – “For example, the client data 118 can include graphics data such as an image taken from a camera, a graphical model such as a three-dimensional model of an object to place within the virtual environment, speech data indicating a name of an object, a category of an object, a type of an object, and the like. The client data 118 can also include text data that indicates the one or more objects to include in the virtual environment.”), a facial image for a sound represented in the natural language data (ROCHE Col 31:24-28 – “At 1110, the animation service 104A may generate one or more first animation sequences of facial features, such as a mouth, of the avatar based at least in part on the one or more phonic symbols and the time codes associated with the phonic symbols.”); and
mapping the facial image to an emotion corresponding to the sound (ROCHE Col 22:57-23:2 – “In some configurations, the animation manager 204A may receive or determine an emotion value associated with the text, and may use the emotion value (e.g., an parameter, weight, or other designator) to influence the animation sequence, the standard pose, and/or other attributes of the animation. The emotion value may be received from the ASR service 104F, for example, and/or be generated or provided as discussed above. The animation manager 204A may determine or modify movement range of skeletal components and/or facial features based on an emotional value, such as increasing movement range for emotional values associated with “happy” or “excited” and reducing movement range for emotional values associated with “sad” or “tired”.”; Col 33:31-47 – “In some examples, the emotion value may change during a duration of a speech, such as by starting as a first emotion value associated with a first group of words and later changing to a second emotion value associated with a second group of words.”).
REGARDING CLAIM 29, ROCHE discloses the computer-implemented method of claim 21, further comprising:
generating audio data including synthesized speech corresponding to the natural language data (ROCHE Col 30:52-31:3 – “At 1104, the animation service 104A may send text to the text-to-speech service 104E and/or the ASR service 104F to generate audio data and speech markup data. The audio data may include speech audio of the text. The audio data may be generated to create specific sound profiles of a speaker, such as a male voice, a female voice, a deep voice, a high pitched voice, a sad voice, an excited voice, and/or other variations or combinations thereof. The speech markup data may include at least one or more phonic symbols, one or more words, and one or more time codes. The one or more time codes may indicate a time of occurrence of the one or more phonic symbols and the one or more words during playback of the audio data. In some examples, the text may include and/or be associated with at least some markup data before the text is sent to the text-to-speech service 104E and/or the ASR service 104F. For example, markups and/or other information may be added to the text, which may be used by the animation service 104A and/or other services discussed herein.”); and
generating the output data to further comprise the audio data (ROCHE Col 31:44-57 – “At 1114, the animation service 104A may output the audio data synchronized with output of combined animation sequences of the avatar that include the one or more first animation sequences and the one or more second animation sequences. The output may be a file, such as a downloadable animation file that includes the animation, the sound, or both. In various examples, the combined animation sequences may include a blending of a first animation sequence and a second animation sequence to create at least part of the second gesture sequence. The output may include a virtual environment that includes at least one object in the virtual environment along with the avatar. The virtual environment may include another avatar and/or other objects, which may be associated with tags as discussed above.”).
REGARDING CLAIM 30, ROCHE discloses the computer-implemented method of claim 29, further comprising:
configuring the video data to correspond to an output time duration (ROCHE Fig. 6B – “Animation Sequence 648”; Col 25:54-26:5 – “The words may be analyzed by the ASR service 104F to create the speech markup data that includes the phonic symbols 646. The specific phonic symbols 654 may be associated with animation sequences of facial features and/or mouth movements 656. The phonic symbols 654 may be unique sounds, sounds associated with particular mouth or facial movements, or other subdivisions of words or sentences used to create or select animation sequences. Different levels of granularity may be used to describe the words other than the phonic symbols shown in FIG. 6B. For example the text may include more phonic symbols than the nine phonic symbols shown in FIG. 6B. The animation sequences 648 and corresponding phonic symbols 646 may be stored in the data store 202A. The animation sequences 648 may be associated with the third time codes and may be played back at occurrence of the third time code to synchronize movement of facial features and/or mouth movements with playback of the audio data to animate speaking by the avatar.”); and
configuring the audio data to correspond to the output time duration (ROCHE Fig. 6B; Col 25:43-53 – “The time codes 642 may span an amount of time (e.g., ti) used to playback the audio of the text or portion of the text. In the example provided in FIG. 6B, the text reads “There was once a really big monster.” The time codes 642 may include first time codes 650 that indicate a time of occurrence of each word in the text. The time codes 642 may include second time codes 652 that indicate a time of occurrence of each phonic symbol in the text. The combined time codes may represent third time codes, which may be used to cue the different animation sequences 648, such as skeletal animations and facial animations.”).
REGARDING CLAIM 31, ROCHE discloses a system comprising: at least one processor; and at least one memory comprising instructions that, when executed by the at least one processor (ROCHE Fig. 14 – “CPU, RAM, ROM, Storage Controller, Data Store”), cause the system to: perform the steps of claim 1; thus, it is rejected under the same rationale.
Claim 34 is similar to claim 24; thus, it is rejected under the same rationale.
Claim 35 is similar to claim 25; thus, it is rejected under the same rationale.
Claim 36 is similar to claim 26; thus, it is rejected under the same rationale.
Claim 38 is similar to claim 28; thus, it is rejected under the same rationale.
Claim 39 is similar to claim 29; thus, it is rejected under the same rationale.
Claim 40 is similar to claim 30; thus, it is rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22-23 and 32-33 are rejected under 35 U.S.C. 103 as being unpatentable over ROCHE (US 10,521,946 B1), and in further view of BAKIS (US 2005/0096909 A1).
REGARDING CLAIM 22, ROCHE discloses the computer-implemented method of claim 21, further comprising:
[receiving an identifier of a language of the natural language data] (ROCHE Col 33:59-34:2 – “At 1162, the animation service 104A may determine whether input text is to be translated from a first language to a second, different language by a translation service. For example, the system may enable creation of avatars that speak different languages for a particular purpose, such as to greet guests to a website in different languages. Thus, a user may request that input text be translated from a first language to a second language, and possibly to other languages. When the input text is to be translated (following the “yes” route from the decision operation 1162), then the process 1160 may advance to an operation 1164.”); and
generating the video data based at least in part on the identifier (ROCHE Col 34:14-21 – “At 1168, the animation service 104A may generate animation sequences of a face of the avatar based at least in part on the speech markup data in the second language. By using the speech markup language in the second language to drive the animations, the animation service 104A generates different animations for the audio data in the second language than animations generated for audio data in the first language.”).
ROCHE does not explicitly teach the [square-bracketed] limitation. In other words, ROCHE determine the input text is to be translated into another language, but does not explicitly teach an identifier of the language.
BAKIS discloses the [square-bracketed] limitation. BAKIS discloses generating synthesized data based on natural language input comprising: [receiving an identifier of a language of the natural language data] (BAKIS Par 42 – “In some embodiments, the low level markup 132 may further define which language the voice-type 130 is to be spoken in. For example, the voice-type “German Female” 130 b associated with the voice style labeled “serious” 120 n may include the low level markup 132 b. The low level markup 132 b specifies a language parameter as “DE”, which may indicate for example, that the associated text should be spoken in “Deutsch” (or German).”); and
generating the video data based at least in part on the identifier (BAKIS Par 42 – “In some embodiments, the low level markup 132 may further define which language the voice-type 130 is to be spoken in. For example, the voice-type “German Female” 130 b associated with the voice style labeled “serious” 120 n may include the low level markup 132 b. The low level markup 132 b specifies a language parameter as “DE”, which may indicate for example, that the associated text should be spoken in “Deutsch” (or German).”; Par 34 – “The low level markup may be read or otherwise interpreted by the TTS program and used to produce the speech in accordance with the styles, rules, qualities, and/or characteristics associated with the respective style sheet, voice style, and/or voice font. The identified text “flight departs from runway 22L” of the current example, for instance, is associated with a voice-type labeled “Female” and a voice style labeled “formal”. Processing at 108 for this example may include the application of the low level markups associated with the voice-type labeled “Female” and the voice style labeled “formal” to produce the phrase in a formal female voice.”; Note that ROCHE already teaches generating both video and audio based on the natural language data; thus, the combination teaches generating the video based on the language identifier.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of ROCHE to include a language identifier, as taught by BAKIS.
One of ordinary skill would have been motivated to include a language identifier, in order to generate more useful synthesized data to a user according to the user’s preference and needs.
REGARDING CLAIM 23, ROCHE in view of BAKIS discloses the computer-implemented method of claim 22, further comprising:
generating the video data to comprise an avatar speaking the language (ROCHE Col 34:14-21 – “At 1168, the animation service 104A may generate animation sequences of a face of the avatar based at least in part on the speech markup data in the second language. By using the speech markup language in the second language to drive the animations, the animation service 104A generates different animations for the audio data in the second language than animations generated for audio data in the first language.”).
Claim 32 is similar to claim 22; thus, it is rejected under the same rationale.
Claim 33 is similar to claim 23; thus, it is rejected under the same rationale.
Claims 27 and 37 are rejected under 35 U.S.C. 103 as being unpatentable over ROCHE (US 10,521,946 B1), and in further view of CHEN (US 2008/0259085 A1).
REGARDING CLAIM 27, ROCHE discloses the computer-implemented method of claim 21, further comprising:
querying a storage for a [two-dimensional] image corresponding to an avatar (ROCHE Col – “The animation manager 204A may generate combined animation sequences of the avatar that include the first animation sequences and the second animation sequences, which may be stored in the data store 202A, possibly as a file. In some examples, some animation sequences may be blended with other sequences to create and depict natural looking movement of the avatar. For example, when animation sequences conflict in a time period (such as the words “big” and “small” are used within a short period of time (a few seconds), then both example animation sequences discussed above may not be selected since they conflict and would not depict a natural looking animation.”; Col 27:34-44 – “For example, the client data 118 can include graphics data such as an image taken from a camera, a graphical model such as a three-dimensional model of an object to place within the virtual environment, speech data indicating a name of an object, a category of an object, a type of an object, and the like. The client data 118 can also include text data that indicates the one or more objects to include in the virtual environment.”); and
generating the video data by rendering the [two-dimensional] image into a video with the avatar appearing to speak the natural language data (ROCHE Col 31:44-57 – “At 1114, the animation service 104A may output the audio data synchronized with output of combined animation sequences of the avatar that include the one or more first animation sequences and the one or more second animation sequences. The output may be a file, such as a downloadable animation file that includes the animation, the sound, or both. In various examples, the combined animation sequences may include a blending of a first animation sequence and a second animation sequence to create at least part of the second gesture sequence. The output may include a virtual environment that includes at least one object in the virtual environment along with the avatar. The virtual environment may include another avatar and/or other objects, which may be associated with tags as discussed above.”).
ROCHE does not explicitly teach the [square-bracketed] limitation. In other words, ROCHE generating 3D avatar, but does not explicitly teach 2D images.
CHEN discloses the [square-bracketed] limitation. CHEN discloses a method/system for generating synthesized data based on natural language input comprising: querying a storage for a [two-dimensional] image corresponding to an avatar (CHEN Fig. 5; Par 37 – “Referring to FIG. 5, a schematic diagram illustrates rotation of a head portion comprising an upper facial part 205 and a lower facial part 210 of an avatar, according to an embodiment of the present invention. Such rotation of the head portion of an avatar can be performed using image warping technology, which generates a perception of image rotation-but without requiring any three dimensional model rendering. As is known to those skilled in the art, a Thin Plate Spline (TPS) deformation analysis can interpolate movement of fixed points on a surface. TPS deformation analysis uses an elegant algebraic expression for the dependence of a physical bending energy U of a thin metal plate constrained at various points. That can be visualized as a two-dimensional deformable plate that is pushed up from underneath at given points. Because a height of the plate is fixed at given locations, the plate will deform. The energy required to bend the plate can be defined according to Equation 2 below, which is known as the biharmonic equation.”); and
generating the video data by rendering the [two-dimensional] image into a video with the avatar appearing to speak the natural language data (CHEN Par 42 – “At step 710 the lower facial part 205 is animated based on speech data that are classified according to a reduced vowel set. At step 715 a coordinate transformation model, such as a Hotelling transform model, is used to cause gross head tilting movements, including the lower facial part 210 and the upper facial part 205 moving together. At step 720 an image warping model, such as a TPS model, is used to cause gross head rotation movements, including the lower facial part 210 and the upper facial part 205 moving together. At step 725, the limb part 215 is animated using a random model. Finally, at step 730 the upper facial part 205 is animated independently of the animation of the lower facial part 210.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of ROCHE to include generating an avatar from two-dimensional images, as taught by CHEN.
One of ordinary skill would have been motivated to include generating an avatar from two-dimensional images, in order to reduce a computational cost so that the avatar synthesis becomes feasible with limited processor and memory resources (Par 43).
Claim 37 is similar to claim 27; thus, it is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN C KIM whose telephone number is (571)272-3327. The examiner can normally be reached Monday to Friday 8:00 AM thru 4:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew C Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN C KIM/Primary Examiner, Art Unit 2655
Prosecution Insights