DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect amended claims 1, 2, 11, 17, 20 and canceled claims 3, 12, 13, and 19 filed on 09/24/2025 have been considered but they are not persuasive.
The examiner found some amended limitations are taught by references previous introduced.
In Remark page 9, first paragraph, applicant argued that Buzzelli, paragraph [0023]. This is fundamentally different from the claimed invention's requirement for receiving a video stream from a participant's device and then performing an analytical determination step to assess whether that received video stream contains non-facial content versus facial content of the participant”.
The examiner respectfully disagrees with Applicant’s argument. In fact, in Fig. 1, [0023] Buzzelli discloses “a video stream of an active portion of an active screen of the first user device 101. The communication session 100 further optionally includes representations of one or more other users, such as representations of a second user 122A and a third user 123A in the first portion 117 of the meeting application view 115, and optionally other participants or audience members in a second portion 118 of the meeting application view 115” Buzzelli teaches determining a video stream of an active portion, receive from the participant user’s device (1st user device 101), and providing the facial content of the participant user (e.g., 101A, 122A, 123A) in a first portion 117 and the non-facial content of the participant user in a second portion 118 of the meeting application view 115 (Fig. 1) as the amended claims.
In Remark page 11, second paragraph, applicant argued that Du's approach uses pre-fabricated avatar images that lack any connection to the actual appearance or characteristics of real participants. In contrast, the claimed invention requires generating avatar data specifically "based on the at least one image" of the actual participant user's face.
The examiner respectfully disagrees with Applicant’s argument. In fact, in paragraph [0011], Du discloses “A user's facial features are tracked in live video of the user, and facial feature parameters determined from the tracked features are mapped to predetermined avatar images”. Therefore, the avatar images that are mapped to the actual appearance or characteristics of real participants.
In Remark page 12, second paragraph, applicant argued that Tarquini does not disclose using video data as input to a second ML model as required by the claimed invention. While Tarquini mentions lip synchronization in paragraphs [0347] and [0348].
The examiner respectfully disagrees with Applicant’s argument. It should be note that in the103 rejecting this limitation, the examiner did not propose a modification to Tarquini to result “using video data as input to a second ML model” not being replicated as asserted by applicant because “using video data as input to a second ML model” is already discloses in the second’s reference of Harazi, paragraphs [0022], [0120]. Rather, the examiner rationalized that Tarquini suggests that what is being replicated depends on what problem is being solved. For example, Tarquini discloses using the participant user's own speech to mimic the participant user's specific lips movements and accent ([0362], [0347], [0348]). Doing so, it may provide a 3D avatar representing feeling, sentiments, mood (Tarquini, ([0015]).
The independent claim 11 and 17 are amended similar to claim 1 and rejected as the reason above.
Dependent claims 2-10, 12-16 and 18, 20 depend on independent claims 1, 11, 17 and the rejections to the claims are maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing
out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the
invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 20 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 20 is rejected under 35 U.S.C. § 112 (b) because it depends on the canceled claim 19. Proper amendment is suggested.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5, 8-11 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable by Buzzelli (U.S. 2024/0112389 A1) in view of Harazi et al. (U.S. 2023/0252972 A1) and further in view of Du et al. (U.S. 2014/0218371 A1) and further in view of Wei et al. (U.S. 2024/0312095 A1) and further in view of Tarquini et al. (U.S. 2015/0070351 A1).
Regarding Claim 1 (Currently amended), Buzzelli discloses a computer-implemented method (Buzzelli, [0003] “a method”) comprising:
receiving a facial data associated with at least one image of a face of a participant user (Buzzelli, Fig. 1 [0023] “sharing information in a first portion 117 of a meeting application view 115, such as a representation of the first user 101A with accompanying audio data of the first user or other shared information (e.g., a video stream of an active portion of an active screen of the first user device 101, a document)…The communication session 100 further optionally includes representations of one or more other users, such as representations of a second user 122A and a third user 123A in the first portion 117 of the meeting application view 115” Buzzelli teaches receiving one image of a face (facial image of first user 101A, or second user 122A or third user 123A, Fig. 1) associated with a participant user.
generating a static avatar data of the participant user based on the at least one image facial data using a first machine learning (ML) model (Buzzelli, [0020] “graphical representations of the user can include live video or static images of the user” and Fig.2, [0027] “the first representation of the user 222A can be a live video version of the user or an avatar representation mimicking one or more emotions, actions, or responses of the user” and [0033] “supervised machine-learning models such as the CNN or SVM utilize training data sets of facial images labeled with the user's emotions” Buzzelli teaches generating a static avatar data of the participant user (the static image of user 222A) based on the one image (e.g., the user’s emotion image) using a first ML model;
receiving video data associated with facial movement the participant user, (Buzzelli, [0020] “graphical representations of the user can include live video or static images of the user” [0027] “The select icons 220 in FIG. 2 include four emotions (e.g., happiness, sadness, surprise, anger, etc.). Emotions 1-3 are selected. Emotion 4 is not selected (e.g., anger)”. Emotion I (e.g., happiness) has been adjusted from the central mark to be more. The second representation of the user 222B is a representation of the user with more happiness response” Buzzelli teaches receiving live video data (selected happiness) associated with facial movement (open mouth of user 222B); and
retrieving the animated avatar data associated with the participant user (Buzzelli, [0017] “Meeting services and meeting applications additionally provide users control over how they are presented and perceived by others, such as by substituting, concealing, or obfuscating camera backgrounds, or providing customizable avatars that mimic their actions and emotions” and [0048] “facial animation models, or facial animation rules to render a modified emotional response on the representation of the user” Buzzelli teaches retrieving the animated avatar data (rendered by animation models/rules) such as customizable avatar mimic the participant user actions and emotion in the meeting; and
generating an avatar video stream of the participant user based on the animated avatar data mimicking received audio stream associated with the participant user (Buzzelli, Fig. 1, [0023], “a representation of the first user 101A with accompanying audio data of the first user or other shared information (e.g., a video stream of an active portion of an active screen of the first user device 101…) and providing such information to the animation service 111 for management and display” and Fig. 2, [0027] “the first representation of the user 222A can be a live video version of the user or an avatar representation mimicking one or more emotions, actions, or responses of the user” and [0048] “facial animation models, or facial animation rules to render a modified emotional response on the representation of the user” Buzzelli teaches generating an avatar video stream of the participant user (a first user 101A) via the animation service (111, Fig. 1, animated avatar data) based on the received audio stream (with accompanying audio data ) of the first user mimicking one or more emotions, actions, or responses of the user (Fig. 2);
determining that a video stream, received from the participant user's device, contains non-facial content of the participant user (Buzzelli, Fig. 1, [0023] “a video stream of an active portion of an active screen of the first user device 101. The communication session 100 further optionally includes representations of one or more other users, such as representations of a second user 122A and a third user 123A in the first portion 117 of the meeting application view 115, and optionally other participants or audience members in a second portion 118 of the meeting application view 115” Buzzelli teaches determining a video stream of an active portion, receive from the participant user’s device (1st user device 101), contains non-facial content of the participant user in a second portion 118 of the meeting application view 115.
Buzzelli discloses rendering avatar representation of emotion states associated with facial movement of a user by using machine learning model (Buzzelli, [0021], [0027], [0033], [0044])
However, Buzzelli does not explicitly teach wherein the video data facial movement includes lips movement, eyebrows movement, jaw movement, and eyes movement of the participant user; and
using a second ML model to generate animated avatar data for generating an avatar video stream, wherein input for the second ML model includes the static avatar data, the video data associated with facial movement, and a corresponding audio stream of the participant user, wherein the second ML model is trained using the participant user's own speech to mimic the participant user's specific lips movements and accent;
wherein the animated avatar data mimics appropriate facial movements associated with the corresponding audio stream;
identifying the participant user when the user becomes an active speaker at a meeting;
determining that a video stream, received from the participant user's device, contains non-facial content of the participant user;
Harazi teaches using a second ML model to generate trained avatar data for generating an avatar video stream, wherein input for the second ML model includes the static avatar data, the video data, and a corresponding audio stream of the participant user, wherein the trained avatar data mimics appropriate facial movements associated with the corresponding audio stream of the participant user (Harazi, [0044] “the map system 210 enables the display of user icons or avatars (e.g., stored in profile data 316) on a map” and Fig. 1, [0074] A transformation system can capture an image or video stream on a client device (e.g., the client device 102)” and [0120] “a second machine learning technique is trained to compute an emotion and intensity level for a given text string. In such cases, the training data includes pairs of text strings, and the corresponding ground truth emotion and/or intensity” and Fig. 1, [0051] “Avatar characteristics include different ways to customize a look and feel of an avatar, such as different poses, facial features” and [0043] “the facial tracking-based features…are modified based on movement of facial features detected in a received or captured image or video, the media overlays may include the audio clip-based features in which a sound clip or audio a clip” and [0052] “The text to speech system 230 generates an audio stream that includes one or more words spoken by a specified speaker with a given emotion” and [0022] “…generate audio that includes one or more words of the text string virtually spoken by the specified speaker with a specified emotion…generate the audio by splitting embeddings of various speakers into two components that include voice and style components…the new embedding—can be generated using a machine learning technique (e.g., a neural network)” Harazi teaches using a second ML model on 2nd machine learning for generating an avatar video stream (captured video stream) for the generated static data (stored in profile data), the input for the second ML model (received captured image or video) as training data for generating avatar characteristics (look and feel, facial features) based on the detected movement of facial features and a corresponding audio stream of the participant user e.g., generated audio stream that includes one or more words of the text string virtually spoken by the specified speaker (the participant user) with a specified emotion (happy, sad, angry) and voice.
Harazi teaches identifying the participant user when the user becomes an active speaker at the meeting (Harazi, [0022] “The messaging client 104 can also receive a selection of a speaker (a specified speaker) or can randomly or pseudo-randomly select the speaker from a list of speakers,…generate audio that includes one or more words of the text string virtually spoken by the specified speaker with a specified emotion” and [0033] “the messaging client 104 can provide participants in a conversation (e.g., a chat session) in the messaging client 104 with notifications…The external resource can provide participants in a conversation” Harazi teaches identifying the participant user when the user becomes an active speaker by a selection of a speaker (a specified speaker) spoke a text string with a specified emotion;
Buzzelli and Harazi are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Buzzelli to apply a second ML model to generate a trained avatar data (as taught by Harazi) in order to use a second ML model to generate a trained avatar data associated with facial movement and the facial movement associated with audio data because Harazi can provide using a second ML model on 2nd machine learning for generating an avatar video stream (captured video stream) for the generated static data (stored in profile data), the input for the second ML model (received captured image or video) as training data for generating avatar characteristics (look and feel, facial features) based on the detected movement of facial features and a corresponding audio stream of the participant user e.g., generated audio stream that includes one or more words of the text string virtually spoken by the specified speaker (the participant user) with a specified emotion (happy, sad, angry) and voice (Harazi, [0043], [0051], [0120]). Doing so, it may reduce the number of resources needed to perform routine social networking tasks and operations (Harazi, ([0017]).
However, Buzzelli and Harazi does not explicitly teach generate animated avatar data.
Wei teaches using a second ML model to generate animated avatar data for generating an stream, wherein input for the second ML model includes the static avatar data, the video data associated with facial movement (Wei, [0040] “The machine learning model 208 is specifically a two-stage model having a first stage 522 and a second stage 524” and Fig. 5, [0041] “That is, the first stage 522 is trained so that it extracts image features 526 on which basis the second stage 524 can accurately predict blendshape weights 506, facial expression 504, and 3D vertices 512” and [0017] “ A facial expression can be defined by a set of blendshape weights of a facial action coding system (FACS). A FACS taxonomizes human facial movement by their appearance on the face” and [0050] “FIG. 7, From the avatar training images 510, an avatar animation video 702 is generated (704). The avatar animation video 720 includes consecutive groups of frames 706 of the facial avatars 502 having the facial expression 504 at the specified blendshape weights 506” and [0068] “permitting sufficient training data to be generated for training the model without having to manually label the training images with blendshape weights” Wei teaches a 2 stages ML models and using a second ML model (the second stage 524, corresponding to blendshape weight 506, Fig. 5) to generate animated avatar data for generating a stream (the avatar animation video 720) includes the static avatar data (without having to manually label the training images), the data associated with facial movement (human facial movement by their appearance on the face).
Buzzelli, Harazi and Wei are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Buzzelli to apply a second ML model to generate an animated avatar data (as taught by Wei) in order to use a second ML model to generate an animated avatar data associated with facial movement because Wei can provide a 2 stages ML models and using a second ML model to generate animated avatar data for generating an stream (the avatar animation video 720) includes the static avatar data (without having to manually label the training images), the data associated with facial movement (human facial movement by their appearance on the face) (Wei, [0017], [0040], [0041], [0050]). Doing so, it may provide the accuracy of the machine learning model may depend on the quantity and diversity of the training data, acquiring large numbers of different HMD-captured training images of actual HMD wearers exhibiting different facial expressions can be paramount even if necessitating significant time and effort (Wei, ([0020]).
Du teaches the video data includes lips movement (Du, [0011] “A user's facial features are tracked in live video of the user, and facial feature parameters determined from the tracked features are mapped to predetermined avatar images” and [0027] “Fig. 3, selecting an avatar feature image from series 300 based on, for example, the distance between the upper and lower lips, and combining the selected avatar feature images with the avatar background 320) Du teaches the video data includes the lips movement between the upper and lower lips) eyebrows movement (Du, [0016] “facial features include the position of left and right eyebrows (inside end, middle and outside end) and [0023] “avatar images series can be used to make raise its eyebrows” Du teaches eyebrows movement (raise its eyebrows), jaw movement (Du, [0028] “a single avatar feature image series for animating an avatar's mouth can be used to make the mouth movements associated with a language's phonemes” and Fig. 3, [0029] “consider the avatar feature image series 300 used for animating the opening and closing of an avatar's mouth” Du teaches jaw movement (referred to as the opening and closing of an avatar’s mouth), and eyes movement (Du, [0020] “FIG. 2, series 210 can be used to animate an avatar blinking” Du teaches eyes movement (eyes is blinking);
A combination between Wei and Du can be used to teach using a second ML model to generate animated avatar data for generating an avatar video stream (as taught by Wei), wherein the video data includes lips movement, eyebrows movement, jaw movement, and eyes movement (as taught by Du).
Buzzelli, Harazi, Wei and Du are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Buzzelli to apply the detected animated avatar effects to user (as taught by Du) in order to use animating video of the detected effects of the user with an avatar because Du can provide the detected a animated user’s avatar (180) effects to the user’s facial features (170) on the display of the computing device (Du, Fig. 1, [0011], [0033]). Doing so, it may provide the battery life of mobile computing devices employing the disclosed technologies can be extended relative to devices that use more computationally intensive approaches (Du, ([0011]).
Tarquini teaches the second ML model is trained using the participant user's own speech to mimic the participant user's specific lips movements and accent (Tarquini, [0150] “the ASM model used is determined from a number of training sets to create avatar form” and [0362] “Each participant to the competition is represented graphically by his/her LP” and [0348] The virtual reader could be customized to speak with a given accent and voice, the lip movement being synchronized with the spoken text with the LP showing expressions and animations to make the reading more lifelike” Tarquini teaches a model is trained using the participant user’s speech to speak (mimic) with a given accent and the lip movement being synchronized with the spoken text.
Buzzelli, Harazi, Wei and Du and Tarquini are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Buzzelli to speak (mimic) with accent and user’s lips movements (as taught by Du) in order to speak (mimic) with accent and user’s lips movements in user’s speech because Tarquini can provide a model is trained using the participant user’s speech to speak (mimic) with a given accent and the lip movement being synchronized with the spoken text (Tarquini, [0150], [0348]). , [0362]). Doing so, it may provide a 3D avatar representing feeling, sentiments, mood (Tarquini, ([0015]).
Regarding Claim 2, a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 1, wherein the corresponding from the audio stream of the participant user is stream without video of the participant user determining that the video stream contains non-facial content of the participant user further comprises determining that the video stream contains a whiteboard or shared screen (Buzzelli, Fig. 1, [0023] “ a first user device 101 executing a meeting application and sharing information in a first portion 117 of a meeting application view 115, a video stream of an active portion of an active screen of the first user device 101. The communication session 100 further optionally includes representations of one or more other users, such as representations of a second user 122A and a third user 123A in the first portion 117 of the meeting
application view 115, and optionally other participants or audience members in a second portion 118 of the meeting application view 115” Buzzelli teaches determining a video stream of an active portion, receive from the participant user’s device (1st user device 101), contains non-facial content of the participant user in a shared screen e.g., a second portion 118 of the meeting application view 115.
Regarding Claim 3 (Canceled).
Regarding Claim 5, a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 4, wherein the injecting complements data being transmitted during the meeting session without video stream of the participant user (Buzzelli, Fig. 1, [0023] “optionally other participants or audience members in a second portion 118 of the meeting application view 115. The first and second portions 117, 118 of the meeting application view 115 can change sizes and organization depending on, for example, active participants of the communication session 100, participants sharing audio…in the communication session 100, hosts of the communication session 100, etc.” Buzzelli teaches the injecting complements data being transmitting (sharing audio data associated with participant users (in a second portion 118, Fig. 1) during a meeting session (e.g., the communication session 100) without a video stream of the participant user.
Regarding Claim 8 (Currently amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 1, wherein the at least one image facial data associated with the participant user comprises at least one static image of the participant user from an application that is different from an application that facilitates an online meeting for the participant user (Buzzelli, [0029] “[0029] FIG. 3 illustrates an example control area 319 illustrating a radar chart adjustment feature 321 providing different representations 322A-322D of a user (e.g., the second user 122A of FIG. 1) at different meeting application settings, such as illustrated by select icons 320A-320D and adjustment features 321A, 321B of the control area 319 of a meeting application” Buzzelli teaches one static image of the facial data of the participant user, second user 122A of FIG. 1 from an application that is different from a meeting application (on line) e.g., the representation of images 322A-322D of second user 122A in Fig. 3.
Regarding Claim 9 (Currently amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 1, wherein the at least one image facial data associated with the participant user comprises at least one static image of the participant user from an that facilitates an online meeting for the participant user (Buzzelli, Fig. 2, [0028] “the meeting application can provide one or more of the first and second representations 222B, 222C to the user as preview images before providing the rendered representation of the user to the one or more other participants or audience members of a communication session, etc.” Buzzelli teaches at least one static image of the participant user (images 222B, 222C) from an that facilitates an online meeting.
Regarding Claim 10 (Currently amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 1, wherein the at least one image facial data associated with the participant user comprises at least a portion of a video stream (Buzzelli, Fig. 1, [0023] “a video stream of an active portion of an active screen of the first user device 101. The communication session 100 further optionally includes representations of one or more other users, such as representations of a second user 122A and a third user 123A in the first portion 117 of the meeting application view 115, and optionally other participants or audience members in a second portion 118 of the meeting application view 115” Buzzelli teaches determining a video stream of an active portion, receive from the participant user’s device (1st user device 101), contains facial content of the participant user in a first portion 117 and non-facial content of the participant user in a second portion 118 of the meeting application view 115.
Regarding Claim 11 ( Currently amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose a computer-implemented method (Buzzelli, [0003] “a method”) comprising:
receiving a facial data associated with at least one image of a face of a participant user:
generating a static avatar data of the participant user based on the at least one image facial data using a first machine learning (ML) model:
receiving video data associated with facial movement the participant user, wherein the video data facial movement includes lips movement, eyebrows movement, jaw movement, and eyes movement of the participant user; and
using a second ML model to generate animated avatar data for generating
an avatar video stream, wherein input for the second ML model includes the static avatar data, the video data associated with facial movement, and a corresponding audio stream of the participant user, wherein the second ML model is trained using the participant user's own speech to mimic the participant user's specific lips movements and accent:
wherein the animated avatar data mimics appropriate facial movements
associated with the corresponding audio stream.
identifying the participant user when the user becomes an active speaker at a meeting;
determining that a video stream, received from the participant user's device, contains non-facial content of the participant user;
retrieving the animated avatar data associated with the participant user; and
generating an avatar video stream of the participant user based on the animated avatar data mimicking the received audio stream associated with the participant user.
Claim 11 is substantially similar to claim 1 and is rejected based on similar analyses.
Regarding Claim 12 (Canceled).
Regarding Claim 13 (Canceled).
Regarding Claim 16, a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 15, wherein the injecting complements data being transmitted during the meeting session without video stream of the participant user.
Claim 16 is substantially similar to claim 5 and is rejected based on similar analyses.
Regarding Claim 17 (Current amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose a system (Buzzelli, [0003] “a system”), comprising:
a processor (Buzzelli, [0051] “a processor 1002”);
a memory (Buzzelli, [0051] “a processor 1003”); operatively connected to the processor and storing instructions that, when executed by the processor (Buzzelli, [0052] “The instructions may be performed by the processor or by multiple processors within the computing device”), cause:
receiving a facial data associated with at least one image of a face of a participant user:
generating a static avatar data of the participant user based on the at least one image facial data using a first machine learning (ML) model:
receiving video data associated with facial movement the participant user, wherein the video data facial movement includes lips movement, eyebrows movement, jaw movement, and eyes movement of the participant user; and
using a second ML model to generate animated avatar data for generating
an avatar video stream, wherein input for the second ML model includes the static avatar data, the video data associated with facial movement, and a corresponding audio stream of the participant user, wherein the second ML model is trained using the participant user's own speech to mimic the participant user's specific lips movements and accent:
wherein the animated avatar data mimics appropriate facial movements
associated with the corresponding audio stream.
identifying the participant user when the user becomes an active speaker at a meeting;
determining that a video stream, received from the participant user's device, contains non-facial content of the participant user;
retrieving the animated avatar data associated with the participant user; and
generating an avatar video stream of the participant user based on the animated avatar data mimicking the received audio stream associated with the participant user.
Claim 17 is substantially similar to claim 1 and is rejected based on similar analyses.
Regarding Claim 18 (Currently amended), a combination of Buzzelli, Harazi, Wei, Du and Tarquini disclose the system of claim 17, wherein the corresponding from the audio stream of the participant user is stream without video stream of the participant user determining that the video stream contains non-facial content of the participant user further comprises determining that the video stream contains a whiteboard or shared screen.
Claim 18 is substantially similar to claim 2 and is rejected based on similar analyses.
Regarding Claim 19 (Canceled).
Claims 4, 6-7, 14-15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable by Buzzelli (U.S. 2024/0112389 A1) in view of Harazi et al. (U.S. 2023/0252972 A1) and further in view of Wei et al. (U.S. 20240312095 A1) and further in view of Du et al. (U.S. 2014/0218371 A1) and further in view of Tarquini et al. (U.S. 2015/0070351 A1) and further in view of Zhang et al. (U.S. 2024/0129437 A1).
Regarding Claim 4 (Currently amended), the computer-implemented method of claim 1, a combination of Buzzeli, Harazi, Wei, Du and Tarquini do not explicitly teach further comprising: injecting the generated avatar video stream to data being transmitted to participants of the meeting when the participant user is speaking as the active speaker.
However, Zhang teaches injecting the generated avatar video stream to data being transmitted to participants of the meeting when the participant user is speaking as the active speaker (Zhang, Figs. 1A, 1B, [0025] FIG. 1B shows the display 112 of the computing system 104 representing the user 102 (not shown in FIG. 1B) with the selected avatar 114 during the videoconference according to an example implementation. The display 112 can show the selected avatar 114, which can move and/or change appearance to correspond to the user's 102 speech, to show the user 102 how other participants of the videoconference see the user 102. The display 112 can also present and/or display other participants 150 of the videoconference. The computing system 104 can also generate audio output based on speech of users represented by the other participants 150, so that all participants in the videoconference, including the user 102, can hear each other and talk to each other” Zhang teaches injecting the generating avatar video stream (selected avatar 114, Fig. 1B) to data being transmitted to participants (participants 150) of the meeting when the participant user (120) is speaking as the active speaker.
Buzzelli, Harazi, Wei, Du, Tarquini and Zhang are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Buzzelli to (as taught by Zhang) in order to inject the generated avatar video stream to data to the participant user is speaking as the active speaker in the meeting because Zhang can provide injecting the generating avatar video stream (selected avatar 114, Fig. 1B) to data being transmitted to participants (participants 150) of the meeting when the participant user (120) is speaking as the active speaker (Zhang, Figs. 1A, 1B [0025]). Doing so, it may provide a stream video of the avatar, including audio of the user speaking. The avatar can be realistic, e.g., closely resembling video of an actual human speaking (Zhang, ([0020]).
Regarding Claim 6, the computer-implemented method of claim 1, a combination of Buzzeli, Harazi, Wei, Du and Tarquini do not explicitly teach wherein the identifying the participant user is through voice recognition processing of the audio stream associated with the participant user.
However, Zhang teaches the identifying the participant user is through voice recognition processing of the audio data associated with the participant user (Zhang, [0047] “the computing system 104 can select the avatar (502) based on remote audio input received during the videoconference. The remote audio input can include attendees other than the user 102 speaking during the videoconference. The computing system 104 can recognize voices of attendees based on the remote audio input and/or words or topics discussed based on the remote audio input. The computing system 104 can select the avatar (502) based on the recognized attendees and/or words or topics based on the remote audio input” Zhang teaches identifying the participant user (102) through voice recognition processing of the audio stream input associated with the participant user (102).
Buzzelli, Harazi, Wei, Du, Tarquini and Zhang are combinable see rationale in claim 4.
Regarding Claim 7 (Currently amended), the computer-implemented method of claim 1, a combination of Buzzeli, Harazi, Wei, Du and Tarquini do not explicitly teach wherein the video data associated with facial movement comprises video streams of one or more users speaking and wherein the one or more users are different from the participant user.
However, Zhang teaches the video data comprises video streams of one or more users speaking and wherein the one or more users are different from the participant user (Zhang, Figs. 1A, 1B, [0023] “The computing system 104 can receive audio input via the microphone 120, such as the user's 102 voice while the user 102 is speaking during the videoconference, generate the video representation of the user 102 based on the audio input. For example, while the user 102 is talking, the computing system 104 can cause the mouth of the selected avatar 114 to move within the generated video representation of the user 102…cause the shape of the mouth of the selected avatar 114 to change based on the sounds the user 102 makes, such as opening the mouth when making a long, “O′” sound, or puckering the lips when making a “P” sound” and [0025] “The display 112 can also present and/or display other participants 150 of the videoconference” Zhang teaches generating a video stream of one user speaking (user 102, represent via an avatar 114) and other users (participants 150, Fig. 1B) are different from the participant user 102.
Buzzelli, Harazi, Wei, Du, Tarquini and Zhang are combinable see rationale in claim 4.
Regarding Claim 14 (Currently amended), a combination of Buzzeli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 11, wherein the identifying the participant user is through voice recognition processing of the audio data associated with the participant user.
Claim 14 is substantially similar to claim 6 and is rejected based on similar analyses.
Regarding Claim 15, a combination of Buzzeli, Harazi, Wei, Du and Tarquini disclose the computer-implemented method of claim 11 further comprising:
injecting the generated avatar video stream to data being transmitted to participants of the meeting when the participant user is speaking as the active speaker.
Claim 15 is substantially similar to claim 4 and is rejected based on similar analyses.
Regarding Claim 20, a combination of Buzzeli, Harazi, Wei, Du, Tarquini and Zhang disclose the system of claim 19, wherein the instructions when executed by the process further cause:
injecting the generated avatar video stream to data being transmitted to participants of the meeting when the participant user is speaking as the active speaker.
Claim 20 is substantially similar to claim 4 and is rejected based on similar analyses.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHOA VU whose telephone number is (571)272-5994. The examiner can normally be reached 8:00- 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHOA VU/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611