Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CPR 1.114, including the fee set forth in 37 CPR 1.17 (e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CPR 1.114, and the fee set forth in 37 CPR 1.17 (e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CPR 1.114. Applicant's submission filed on 07/08/2026 has been entered.
Response to Amendment
Applicant’s amendments and remarks submitted 07/08/2026 have been entered and considered, Claims 1, 5 are amended. Claim 2 is canceled. Claim 21 is new.
Response to Arguments
Applicant’s arguments filed on 07/08/2026 have been fully considered but are moot because they don’t apply to the reference(s)/combination(s) in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-5, 7-9, 12-16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993).
Regarding Claim 1. Perez teaches A computer-implemented method (Perez, abstract, the invention describes method of customizing avatar based on captured user data. Data captured with respect to a human may be analyzed and applied to a visual representation of a user such that the visual representation begins to reflect the behavioral characteristics of the user. For example, a system may have a capture device that captures data about the user in the physical space. The system may identify the user's characteristics, tendencies, voice patterns, behaviors, gestures, etc. Over time, the system may learn a user's tendencies and intelligently apply animations to the user's avatar such that the avatar behaves and responds in accordance with the identified behaviors of the user. The animations applied to the avatar may be animations selected from a library of pre-packaged animations, or the animations may be entered and recorded by the user into the avatar's avatar library.), comprising:
providing a virtual avatar associated with a user, wherein the virtual avatar
is associated with a plurality of poses and a plurality of animations (Perez, [0052] As shown, in FIG. 2, the computing environment 12 may include an avatar library 196 that comprises animations selectable for application to the user's avatar. The user profile 198 may include an avatar library or otherwise associated with an avatar library with animations specific to the user. The lookup tables may include the user's behaviors and tendencies with respect to various contextual circumstances. The profile may be use to apply animations to the user's avatar so that the avatar closely reflects the behaviors of the user.
[0053] The animations in the avatar library may comprise any characteristic that may be applied to an avatar. The characteristics may be indicative of a user's behavior. Thus, the animation selected for application to the user's behavior may be selected to correspond to the user's behaviors.);
receiving input data associated with the user from at least one input source (Perez, [0027] The capture device 20 may capture data representative of a user's behaviors. For example, the capture device may capture characteristics that are indicative of the user's behaviors. Such characteristics may include the user's body position, facial expressions, vocal commands, speech patterns, gestures, or the like.);
processing the input data, the input data comprising images of the user
and/or audio data captured from the user (Perez, [0027] The capture device 20 may capture data representative of a user's behaviors. For example, the capture device may capture characteristics that are indicative of the user's behaviors. Such characteristics may include the user's body position, facial expressions, vocal commands, speech patterns, gestures, or the like.
Although Perez didn’t explicitly indicate capturing images of the user, the capture device obtains data such as facial expressions and gestures which requires image being captured.);
determining a user mannerism from the processed input data by: (Perez, [0018] The system may track the user and any motion in the physical space and identify characteristics of the user that can be applied to the user's visual representation. The identified characteristics may be indicative of the user's behaviors. For example, the system may identify the user's physical characteristics, tendencies, voice patterns, gestures, etc. The system may continue to track the user over time and apply modifications or updates to the user's avatar based on the history of the tracked data. For example, the capture device may continue to identify behaviors and mannerisms, emotions, speech patterns, or the like, of a user and apply these to the user's avatar. The fidelity of the rendered avatar, with respect to the resemblance of the avatar to the avatar, increases over time as the system gathers history data for that user.);
Perez fails to explicitly teach, however, Chen teaches determining that the user characteristic is the user mannerism when a frequency of the user characteristic in the input data exceeds a predetermined limit; (Chen, abstract, the invention describes a method for determining output information. The method includes: acquiring at least one history behavior of a user after a first content is output, based on history behavior data of the user; determining a behavior habit of the user after the first content is output, based on the at least one history behavior of the user; and; determining output information for the user in response to the output of the first content, based on the behavior habit of the user. By obtaining different behavior habits corresponding to different users, personalized output information for the users is established.
[0051] S12 includes: determining the behavior habit of the user after the first content is output, based on a first history behavior of the at least one history behavior, wherein the first history behavior meets a preset condition, and wherein the preset condition characterizes a history behavior of the user with a play frequency equal to or larger than a preset threshold value.
[0054] If a preset condition is for a period of time, a preset threshold value can be set for a period of time. A play frequency can be set as a preset threshold value, for example, the play frequency of 5 times within 7 days is set as the preset threshold value.).
Perez and Chen are analogous art because they both teach method of tracking/capturing user characteristics/behavior/habit. Perez further teaches applying those captured mannerism to an avatar. Chen further teaches analyzing user’s characteristics/behavior/habit by calculating the frequency of behavior occurrences. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez), to further evaluate the frequency of the user behavior over a threshold value (taught in Chen), so as to accurately determine user’s behavior habits and personalize output information in line with user’s own needs which improves user’s experience (Chen, [0033]).
The combination of Perez and Chen further teaches and updating at least one of the plurality of poses and/or animations of the virtual avatar, or creating a new pose and/or animation, to apply the user mannerism to the virtual avatar (Perez, [0018] The system may track the user and any motion in the physical space and identify characteristics of the user that can be applied to the user's visual representation. The identified characteristics may be indicative of the user's behaviors. For example, the system may identify the user's physical characteristics, tendencies, voice patterns, gestures, etc. The system may continue to track the user over time and apply modifications or updates to the user's avatar based on the history of the tracked data. For example, the capture device may continue to identify behaviors and mannerisms, emotions, speech patterns, or the like, of a user and apply these to the user's avatar. The fidelity of the rendered avatar, with respect to the resemblance of the avatar to the avatar, increases over time as the system gathers history data for that user).
Regarding Claim 4. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, wherein determining the user mannerism comprises:
determining a link between the user mannerism and a trigger event or a
trigger input in the input data; and
associating the updated pose and/or animation, or the new pose and/or
animation, with the trigger event or trigger input (Perez, [0052] As shown, in FIG. 2, the computing environment 12 may include an avatar library 196 that comprises animations selectable for application to the user's avatar. The user profile 198 may include an avatar library or otherwise associated with an avatar library with animations specific to the user. The lookup tables may include the user's behaviors and tendencies with respect to various contextual circumstances. The profile may be use to apply animations to the user's avatar so that the avatar closely reflects the behaviors of the user.
[0053] The animations in the avatar library may comprise any characteristic that may be applied to an avatar. The characteristics may be indicative of a user's behavior. Thus, the animation selected for application to the user's behavior may be selected to correspond to the user's behaviors.
[0054] The animations in the avatar library 193 may be a stock library of animations. In an example embodiment, the animations applied to the avatar may be animated with an animations selected from a library of pre-packaged animations, such as those that come with a program, application, or a system, for example. The animation selected for application to the user's avatar may be that which correspond to the user's inputs learned by the system to reflect certain behaviors. For example, the system may identify that the user tends to jump up and down in a certain context, such as when achieving success in a game application. Thus, when the system identifies a similar set of contextual circumstances such as a similar state of the game (e.g., success), the system may select an animation that reflects jumping up and down and apply the animation to the user's avatar. The pre-canned animations may be defined for an application or for a system. For example, the jumping up and down animation may be applicable to a gaming application, but an open/close file animation applied to an avatar may be the same system-wide.
Therefore, specific contextual circumstances (trigger event) triggers specific movement of the user.).
Regarding Claim 5. The combination of Perez and Chen further teaches The computer-implemented method of claim 4, comprising using image recognition and/or audio recognition to determine the trigger event or trigger input from the input data (Perez, [0054] The animations in the avatar library 193 may be a stock library of animations. In an example embodiment, the animations applied to the avatar may be animated with an animations selected from a library of pre-packaged animations, such as those that come with a program, application, or a system, for example. The animation selected for application to the user's avatar may be that which correspond to the user's inputs learned by the system to reflect certain behaviors. For example, the system may identify that the user tends to jump up and down in a certain context, such as when achieving success in a game application. Thus, when the system identifies a similar set of contextual circumstances such as a similar state of the game (e.g., success), the system may select an animation that reflects jumping up and down and apply the animation to the user's avatar. The pre-canned animations may be defined for an application or for a system. For example, the jumping up and down animation may be applicable to a gaming application, but an open/close file animation applied to an avatar may be the same system-wide.
Therefore, the system uses captured image of user jumping up and down to determine that the circumstance such as game success triggers user’s cheering behavior.).
Regarding Claim 7. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, comprising using audio recognition and/or speech recognition to determine the user characteristic and/or user mannerism from the input data (Perez, [0121] The identity of characteristics indicative of a user's behaviors may include information that may be associated with the particular user 602 such as behavioral tendencies, speech patterns, facial expressions, skeletal movements, words spoken, history data, voice recognition information, or the like. The user's characteristics may comprise physical features of the user, such as: eye size, type, and color; hair length, type, and color; skin color; clothing and clothing colors. For example, colors may be identified based on a corresponding RGB image. Other target characteristics for a human target may include, for example, height and/or arm length and may be obtained based on, for example, a body scan, a skeletal model, the extent of a user 602 on a pixel area or any other suitable process or data. The computing system 610 may use body recognition techniques to interpret the image data and may size and shape the visual representation of the user 602 according to the size, shape and depth of the user's 602 appendages.
[0127] The system can learn any of the user's natural or idle behaviors in such circumstances and associate them to the user. For example, the system may identify how the player walks and save the motion as the walking animation in the avatar library for that user. The system can watch and listen to a user during activity under various circumstances and scenarios that may not involve a gesture or other active control of the system or the executing application. For example, when a user greets a friend in a remote game playing experience, the system may detect that the user typically greets friends with a typically greeting such as "Hi, buddy, how are you?" The same user may greet unknown players with a greeting such as "Hello, my name is ... " The system may use the captured data, including the voice fluctuations, words spoken, and any other motion, and add it to the avatar library for the user's avatar.).
Regarding Claim 8. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, further comprising outputting the updated pose and/or animation, or the new pose and/or animation (Perez, [0134] In both scenarios that the system or user adds, updates, or rewrites gesture or animation data, the system may record the user's inputs and validate the redefined gesture or animation data. For example, if the user is performing a "wave" gesture, the system can detect that the motion corresponds to the "wave" gesture via the gesture recognition engine as described above. The system may detect where the gesture starts and stops and prompt the user to perform a new motion in the physical space to overwrite the current "wave" gesture data and/or create an additional "wave" gesture that corresponds to the user's recorded motion.).
Regarding Claim 9. The combination of Perez and Chen further teaches The computer-implemented method of claim 8, wherein outputting the updated pose and/or animation, or the new pose and/or animation, comprises: modifying at least one of a live state of the virtual avatar or the updated or new pose and/or animation to avoid a potential conflict (Perez, [0134] In both scenarios that the system or user adds, updates, or rewrites gesture or animation data, the system may record the user's inputs and validate the redefined gesture or animation data. For example, if the user is performing a "wave" gesture, the system can detect that the motion corresponds to the "wave" gesture via the gesture recognition engine as described above. The system may detect where the gesture starts and stops and prompt the user to perform a new motion in the physical space to overwrite the current "wave" gesture data and/or create an additional "wave" gesture that corresponds to the user's recorded motion.
Therefore, the new gesture is updated in the system to overwrite the current gesture.).
Regarding Claim 12. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, wherein the user mannerism or the user characteristic includes one or more of: a facial expression; a sound; a word or a phrase spoken by the user; an emote; a pose; a breathing pattern; a gesture; and/or an action (Perez, [0027] The capture device 20 may capture data representative of a user's behaviors. For example, the capture device may capture characteristics that are indicative of the user's behaviors. Such characteristics may include the user's body position, facial expressions, vocal commands, speech patterns, gestures, or the like.)).
Regarding Claim 13. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, wherein the at least one input source comprises at least one primary input source configured to provide primary input data comprising images of the user and/or audio data captured from the user, and at least one secondary input source configured to provide secondary input data (Perez, [0018] The system may track the user and any motion in the physical space and identify characteristics of the user that can be applied to the user's visual representation. The identified characteristics may be indicative of the user's behaviors. For example, the system may identify the user's physical characteristics, tendencies, voice patterns, gestures, etc. The system may continue to track the user over time and apply modifications or updates to the user's avatar based on the history of the tracked data. For example, the capture device may continue to identify behaviors and mannerisms, emotions, speech patterns, or the like, of a user and apply these to the user's avatar. The fidelity of the rendered avatar, with respect to the resemblance of the avatar to the avatar, increases over time as the system gathers history data for that user.
Therefore, the system has history of the tracked data (primary input data) to preset the avatar initially. The continuous tracking data of the user is the secondary input source, which is also used to update the avatar.).
Regarding Claim 14. The combination of Perez and Chen further teaches The computer-implemented method of claim 13, further comprising characterizing the primary input data using the secondary input data, optionally wherein characterizing the primary input data using the secondary input data comprises associating the updated pose and/or animation, or the new pose and/or animation, with an event received in the secondary input data (Perez, [0052] As shown, in FIG. 2, the computing environment 12 may include an avatar library 196 that comprises animations selectable for application to the user's avatar. The user profile 198 may include an avatar library or otherwise associated with an avatar library with animations specific to the user. The lookup tables may include the user's behaviors and tendencies with respect to various contextual circumstances. The profile may be use to apply animations to the user's avatar so that the avatar closely reflects the behaviors of the user.
[0053] The animations in the avatar library may comprise any characteristic that may be applied to an avatar. The characteristics may be indicative of a user's behavior. Thus, the animation selected for application to the user's behavior may be selected to correspond to the user's behaviors.
[0054] The animations in the avatar library 193 may be a stock library of animations. In an example embodiment, the animations applied to the avatar may be animated with an animations selected from a library of pre-packaged animations, such as those that come with a program, application, or a system, for example. The animation selected for application to the user's avatar may be that which correspond to the user's inputs learned by the system to reflect certain behaviors. For example, the system may identify that the user tends to jump up and down in a certain context, such as when achieving success in a game application. Thus, when the system identifies a similar set of contextual circumstances such as a similar state of the game (e.g., success), the system may select an animation that reflects jumping up and down and apply the animation to the user's avatar. The pre-canned animations may be defined for an application or for a system. For example, the jumping up and down animation may be applicable to a gaming application, but an open/close file animation applied to an avatar may be the same system-wide.
Therefore, specific contextual circumstances (trigger event) triggers specific movement of the user.).
Regarding Claim 15. The combination of Perez and Chen further teaches The computer-implemented method of claim 14, wherein the secondary input data comprises gameplay data (Perez, [0052] As shown, in FIG. 2, the computing environment 12 may include an avatar library 196 that comprises animations selectable for application to the user's avatar. The user profile 198 may include an avatar library or otherwise associated with an avatar library with animations specific to the user. The lookup tables may include the user's behaviors and tendencies with respect to various contextual circumstances. The profile may be use to apply animations to the user's avatar so that the avatar closely reflects the behaviors of the user.
[0053] The animations in the avatar library may comprise any characteristic that may be applied to an avatar. The characteristics may be indicative of a user's behavior. Thus, the animation selected for application to the user's behavior may be selected to correspond to the user's behaviors.
[0054] The animations in the avatar library 193 may be a stock library of animations. In an example embodiment, the animations applied to the avatar may be animated with an animations selected from a library of pre-packaged animations, such as those that come with a program, application, or a system, for example. The animation selected for application to the user's avatar may be that which correspond to the user's inputs learned by the system to reflect certain behaviors. For example, the system may identify that the user tends to jump up and down in a certain context, such as when achieving success in a game application. Thus, when the system identifies a similar set of contextual circumstances such as a similar state of the game (e.g., success), the system may select an animation that reflects jumping up and down and apply the animation to the user's avatar. The pre-canned animations may be defined for an application or for a system. For example, the jumping up and down animation may be applicable to a gaming application, but an open/close file animation applied to an avatar may be the same system-wide.
Therefore, specific contextual circumstances (trigger event) triggers specific movement of the user. The trigger event is related to the state of game (e.g. success).).
Regarding Claim 16. The combination of Perez and Chen further teaches The computer-implemented method of claim 1, further comprising storing each updated pose and/or animation, and/or each new pose and/or animation, in a memory (Perez, [0127] The system can learn any of the user's natural or idle behaviors in such circumstances and associate them to the user. For example, the system may identify how the player walks and save the motion as the walking animation in the avatar library for that user. The system can watch and listen to a user during activity under various circumstances and scenarios that may not involve a gesture or other active control of the system or the executing application. For example, when a user greets a friend in a remote game playing experience, the system may detect that the user typically greets friends with a typically greeting such as "Hi, buddy, how are you?" The same user may greet unknown players with a greeting such as "Hello, my name is ... " The system may use the captured data, including the voice fluctuations, words spoken, and any other motion, and add it to the avatar library for the user's avatar.
Therefore, the captured new user behavior is added to the avatar library. It is obvious to a person with ordinary skill in the art that data library can be stored in location such as database server, memory, storage device and etc.).
Regarding Claim 19. The combination of Perez and Chen further teaches A computing device for controlling a virtual avatar on an electronic device, comprising:
a memory comprising computer readable instructions; and
a processor configured to read the computer readable instructions that
when executed causes the computing device to carry out the computer implemented method of claim 1 (Perez, [0083] In FIG. 4, the computing environment 220 comprises a computer 241, which typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer241 and includes both volatile and nonvolatile media, removable and non-removable media. The system memory 222 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 223 and
random access memory (RAM) 260. A basic input/output system 224 (BIOS), containing the basic routines that help to transfer information between elements within computer 241, such as during start-up, is typically stored in ROM 223. RAM 260 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 259. By way of example, and not limitation, FIG. 4 illustrates operating system 225, application programs 226, other program modules 227, and program data 228.).
Claims 3, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Zhang et al (CN114967937).
Regarding Claim 3. The combination of Perez and Chen fails to explicitly teach, however, Zhang teaches The computer-implemented method of claim 1, wherein determining the user mannerism comprises comparing at least a portion of the input data to input data received from one or more other users (Zhang, abstract, the invention teaches a virtual human motion generation method and system. The generation method and system comprise the steps of identifying identity information of a target object so as to generate an archive of the object, further, analyzing actions of the target object when the target object communicates with a virtual person, and particularly, setting small habitual actions which are made by focusing on the target object in an unconscious manner as habitual behaviors of the target object, recording the habitual behavior in a file of the target object; furthermore, whether the other target objects have positive attitudes when communicating with the virtual person and based on the set character of the virtual person, at least one habitual action of the virtual person for reproducing the target object at a proper time is calculated, so that the interaction scene of the two parties is more intimate.
[0003] On another front, related research indicates that in real-life human communication situations, if both parties are enthusiastic about communicating and maintain a positive attitude, they often unconsciously mimic the other person's gestures; for example, spreading their hands, shrugging their shoulders, or nodding rhythmically while speaking, thus creating a more friendly atmosphere.
Therefore, it is obvious to a person with ordinary skill in the art to further consider other user’s gestures when simulating the avatar of the user of interest.).
Perez, Chen and Zhang are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Zhang further teaches considering the effect of other user’s gesture when simulating the avatar for the use of interest. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further consider the effect of surrounding other user’s gesture (taught in Zhang), so as to provide a more accurate method to simulate real life communication scenarios (Zhang, [0001-0004]).
Regarding Claim 17. The combination of Perez, Chen and Zhang further teaches The computer-implemented method of claim 1, wherein the step of determining the user mannerism is at least partially implemented by a data model (Zhang, abstract, the invention teaches a virtual human motion generation method and system. The generation method and system comprise the steps of identifying identity information of a target object so as to generate an archive of the object, further, analyzing actions of the target object when the target object communicates with a virtual person, and particularly, setting small habitual actions which are made by focusing on the target object in an unconscious manner as habitual behaviors of the target object, recording the habitual behavior in a file of the target object; furthermore, whether the other target objects have positive attitudes when communicating with the virtual person and based on the set character of the virtual person, at least one habitual action of the virtual person for reproducing the target object at a proper time is calculated, so that the interaction scene of the two parties is more intimate.
[0111] Building the classification model requires implementing the machine learning steps of the classification model using a large amount of data; in a specific example, machine learning includes acquiring information on multiple different poses, which includes multiple consecutive action frames and multiple sets of (X, Y, Z) coordinate data; machine learning may include using two support vector machines, one using a linear kernel and the other using an RBF kernel, to build two classification models; both classification models can be trained using a tolerance of ε=0:00001 and a one-to-one method;).
Perez, Chen and Zhang are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Zhang further teaches using machine learning to set up classification data model to detect user mannerism. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use the machine learning data model to obtain user characteristics (taught in Zhang), so as to provide a more efficient method to simulate real life communication scenarios (Zhang, [0001-0004]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Du et al (US20140055554).
Regarding Claim 6. The combination of Perez and Chen fails to explicitly teach, however, Du teaches The computer-implemented method of claim 1, comprising using facial tracking and/or head tracking to determine the user characteristic and/or user mannerism from the input data (Du, abstract, the invention describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar, initiating communication, capturing an image, detecting a face in the image, determining facial characteristics from the face, including eye movement and eyelid movement of a user indicative of direction of user gaze and blinking, respectively, converting the facial features to avatar parameters, and transmitting at least one of the avatar selection or avatar parameters.
Perez, [0121] The identity of characteristics indicative of a user's behaviors may include information that may be associated with the particular user 602 such as behavioral tendencies, speech patterns, facial expressions, skeletal movements, words spoken, history data, voice recognition information, or the like. The user's characteristics may comprise physical features of the user, such as: eye size, type, and color; hair length, type, and color; skin color; clothing and clothing colors. For example, colors may be identified based on a corresponding RGB image. Other target characteristics for a human target may include, for example, height and/or arm length and may be obtained based on, for example, a body scan, a skeletal model, the extent of a user 602 on a pixel area or any other suitable process or data. The computing system 610 may use body recognition techniques to interpret the image data and may size and shape the visual representation of the user 602 according to the size, shape and depth of the user's 602 appendages.).
Perez, Chen and Du are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Du further teaches tracking user’s facial expression to determine user’s facial characteristics. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use the facial tracking method to capture user’s facial characteristics (taught in Du), so as to provide user with realistic avatar in interactive environment (Du, [0001-0003]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Khan et al (US10668382).
Regarding Claim 10. The combination of Perez and Chen fails to explicitly teach, however, Khan teaches The computer-implemented method of claim 9, wherein modifying at least one of the live state of the virtual avatar or the updated or new pose and/or animation to avoid the potential conflict comprises: supressing at least one feature of the live state virtual avatar; and/or overriding at least one feature of the updated or the new pose and/or animation (Khan, abstract, the invention describes Methods and systems are provided for augmenting a video game with an avatar of a real world person. A method provides for executing a video game being played by a user via a head mounted display (HMD). The method includes operations for identifying a generic spectator within the video game and determining virtual coordinates associated with the generic spectator. The method also provides receiving data associated with an avatar of a real world person that is usable to display the avatar within the video game in three-dimensional (3D) form. The method provides removing the generic spectator from view within the video game and inserting the avatar into the video game at the virtual coordinates associated with the generic spectator. The method further provides rendering a virtual reality (VR) presentation of the video game for the user having a view of the avatar and sending the VR presentation to an HMD of the user for display.
Col 7, line 54-67, col 8, line 1-4, FIG. 5 shows an overall flow of a method for augmenting a VR video game with an avatar associated with a real world spectator, the avatar changing in appearance in real-time based on real world states of the real world spectator. The method proceeds in operation 500 by executing a video game at a server for a user of an HMD, wherein the executing the video game includes augmentation of the video game with an avatar of a spectator. In operation 510, the method receives real-time sensor data relating to facial expression and body positioning of the spectator via a plurality of sensors associated with the spectator. The method the flows to operation 520, which serves to map the real-time sensor data to virtual states of the avatar related to facial expression and body position of the avatar. In operation 530, the method then modifies an appearance of the avatar of the spectator in real-time according to the virtual states of the avatar relating to facial expression and body positioning of the avatar.).
Perez, Chen and Khan are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Khan further teaches updating the avatar based on changes of the corresponding user in real-time. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use the real-time modification method to keep avatar behavior in sync with the corresponding user (taught in Khan), so as to provide user with more realistic and life-life video game scenes (Khan, col 1, line 22-31).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Kipman et al (WO2010088033).
Regarding Claim 11.The combination of Perez and Kipman fails to explicitly teach, however, Kipman teaches The computer-implemented method of claim 9, wherein modifying at least one of the live state of the virtual avatar or the updated or new pose and/or animation to avoid the potential conflict comprises: blending or interpolating between the live state of the virtual avatar and the updated or the new pose and/or animation (Kipman, abstract, the invention teaches a method of tracking a target includes receiving an observed depth image of the target from a source and analyzing the observed depth image with a prior-trained collection of known poses to find an exemplar pose that represents an observed pose of the target. The method further includes rasterizing a model of the target into a synthesized depth image having a rasterized pose and adjusting the rasterized pose of the model into a model-fitting pose based, at least in part, on differences between the observed depth image and the synthesized depth image. Either the exemplar pose or the model-fitting pose is then selected to represent the target.
[00136] As indicated at 146, a posed model acquired via model fitting can optionally be compared to a posed model acquired via exemplar. In particular, one or more confidence tests can be used to determine which pose is believed to be a more accurate representation of the target. When such a comparison is made, the pose that is believed to be more accurate can be selected while the other pose is discarded and/or saved to facilitate subsequent pose determinations. In some embodiments, high-confidence aspects of one pose may be combined with high-confidence aspects of the other pose to produce a combined pose that is believed to be a better representation of the target than either the model obtained via model fitting or the model obtained via exemplar. It is to be understood that in some embodiments, model fitting, as discussed with reference to steps 116-145 of FIG. 7, may be skipped if the relative confidence in the exemplar pose is above a predetermined threshold.).
Perez, Chen and Kipman are analogous art because they all teach method of tracking/capturing user characteristics/behavior/habit. Perez and Kipman both teach method of tracking/capturing user characteristics and apply to avatar. Kipman further teaches combining two high confidence poses for the avatar. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use the pose merging method (taught in Kipman), so as to create a better representation of the target (Kipman, [0136]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993), Zhang et al (CN114967937) further in view of Beith et al (US20240078731).
Regarding Claim 18. The combination of Perez, Chen and Zhang fails to explicitly teach, however, Beith teaches The computer-implemented method of claim 17, wherein the data model includes an artificial neural network, wherein the artificial neural network is configured to use machine learning to determine the user mannerism from the input data, optionally wherein the artificial neural network is a convolutional neural network (Beith, abstract, the invention describes a device includes a memory and one or more processors configured to process image data corresponding to a user's face to generate face data. The one or more processors are configured to process sensor data to generate feature data and to generate a representation of an avatar based on the face data and the feature data. The one or more processors are also configured to generate an audio output for the avatar based on the sensor data.
[0090] The face data adjuster 130 is configured to generate the adjusted face data 134 based on the feature data 124 and further based on the face data 132. For example, the face data adjuster 130 can include a deep learning architecture neural network. In an illustrative, non-limiting example, the face data adjuster 130 corresponds to a skin U-Net that includes a convolutional neural network contracting path or encoder followed by a convolutional network expanding path or decoder. The contracting path or encoder can include repeated applications (e.g., layers) of convolution, each followed by a rectified linear unit (ReLU) and a max pooling operation, which reduces spatial information while increasing feature information. The expanding path or decoder can include repeated applications (e.g., layers) of up-convolution and concatenations with high-resolution features from the contracting path, from the feature data 124, or both.).
Perez, Chen, Zhang and Beith are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Both Zhang and Beith further teaches using machine learning to detect user mannerism. Beith further teaches using convolutional neural network in feature data extraction. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez, Chen and Zhang), to further use the convolutional neural network in feature extraction (taught in Beith), so as to improve the accuracy with which the avatar conveys such expressions and emotions of the user (Beith, [0001-0004]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Beith et al (US20240078731).
Regarding Claim 20. The combination of Perez and Chen fails to explicitly teach, however, Beith teaches The computing device of claim 19, wherein the computing device comprises or is in communication with an artificial neural network, wherein the artificial neural network is configured to determine the user mannerism (Beith, abstract, the invention describes a device includes a memory and one or more processors configured to process image data corresponding to a user's face to generate face data. The one or more processors are configured to process sensor data to generate feature data and to generate a representation of an avatar based on the face data and the feature data. The one or more processors are also configured to generate an audio output for the avatar based on the sensor data.
[0005] According to one implementation of the present disclosure, a device includes a memory configured to store instructions. The device also includes one or more processors configured to process image data corresponding to a user's face to generate face data. The one or more processors are configured to process sensor data to generate feature data. The one or more processors are also configured to generate a representation of an avatar based on the face data and the feature data. The one or more processors are also configured to generate an audio output for the avatar based on the sensor data.
[0090] The face data adjuster 130 is configured to generate the adjusted face data 134 based on the feature data 124 and further based on the face data 132. For example, the face data adjuster 130 can include a deep learning architecture neural network. In an illustrative, non-limiting example, the face data adjuster 130 corresponds to a skin U-Net that includes a convolutional neural network contracting path or encoder followed by a convolutional network expanding path or decoder. The contracting path or encoder can include repeated applications (e.g., layers) of convolution, each followed by a rectified linear unit (ReLU) and a max pooling operation, which reduces spatial information while increasing feature information. The expanding path or decoder can include repeated applications (e.g., layers) of up-convolution and concatenations with high-resolution features from the contracting path, from the feature data 124, or both.).
Perez, Chen and Beith are analogous art because they all teach method of tracking/capturing user characteristics and apply to avatar. Beith further teaches using machine learning to detect user mannerism and using convolutional neural network in feature data extraction. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use the convolutional neural network in feature extraction (taught in Beith), so as to improve the accuracy with which the avatar conveys such expressions and emotions of the user (Beith, [0001-0004]).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Perez et al (US20110007079) in view of Chen et al (US20200409993) further in view of Chencinski et al (US20230384866).
Regarding Claim 21. The combination of Perez and Chen fails to explicitly teach, however, Chencinski teaches The computer-implemented method of claim 1, further comprising determining that the user characteristic is the user mannerism when the frequency exceeds the predetermined limit and when a predetermined threshold is met, the predetermined threshold being met when a parameter used to characterize an emotion or an action is achieved (Chencinski, abstract, the invention describes a head-wearable audio device with a motion sensor that outputs motion parameter values based on a detected motion gesture. A gesture type is recognized by comparing the motion parameter values and per-user per-gesture-type recognition parameter values. An action is selected based on the selected gesture type and is executed by the head-wearable audio device or an application in a remote computing device. The head-wearable audio device may also include a capacitive touch sensor that detects capacitive touch events and outputs capacitive touch parameter values. A gesture type may be recognized by comparing both of detected motion parameter values and detected capacitive touch parameter values with per-user per-gesture-type recognition parameter values. The per-user per-gesture-type recognition parameter values are improved over time using machine learning and/or automated logic based on historical detected motion or capacitive touch parameters and corresponding selected gestures, or user profile data.
[0032] Methods, systems, and computer program products disclosed herein may include applying learning algorithms (e.g., machine learning or rule based logic) to gesture identification processes in order to learn gesture patterns or gesture characteristics specific to each user (e.g., a user's gesture signature comprising sensor output parameter values triggered by the user's input). The learning algorithms allow for customized gesture recognition for the user. These gesture patterns or characteristics may be persistently stored in memory of the head-wearable audio device, or in a user profile stored in a host device or in the cloud. One or more machine learning model(s) may be created that is tailored for each user. Moreover, gesture characteristics (e.g., velocity, acceleration, force, distance, direction, duration) may be determined for each user for use in gesture recognition processes. For example, gesture recognition threshold values may be adjusted or determined based on detected gesture parameter values of a user (e.g., minimums, maximums, average values, etc.) that are output from a sensor. For example, gesture characteristics may be based on trajectories of one or more head gestures (e.g., relative movements of a user's head), or velocity of a head movement (e.g., the speed and/or direction of each gesture movement).).
Perez, Chen and Chencinski are analogous art because they all teach method of tracking/capturing user characteristics/habit/behavior. Chencinski further teaches gesture detecting method based on gesture parameter value evaluation. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the avatar modification method based on captured user mannerism (taught in Perez and Chen), to further use gesture detecting method (taught in Chencinski), so as to accurately capture user gesture pattern which can be applied to avatar.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Poot et al (US20100302138), abstract, the invention describes a system which tracks a user's motions or gestures performed in a physical space and map them to a visual representation of the user. The user's gestures may be translated to a control in a system or application space, such as to open a file or to execute a punch in a punching game. Similarly, the user's gestures may be translated to a control in the system or application space for making modifications to a visual representation. A visual representation may be a display of a virtual object or a display that maps to a target in the physical space. In another example embodiment, the system may track the
target in the physical space over time and apply modifications or updates to the visual representation based on the history data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/Primary Examiner, Art Unit 2619
Prosecution Insights