DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
2. Applicant’s arguments with respect to claim(s) 1-8, 10-15 and 17-20 (note correction) have been considered but are moot in view of a new ground(s) of rejection. The amendments to the claims necessitated the new ground(s) of rejection discussed below.
With respect to the last office action, Applicant amends claims discusses the claims limitations and the prior art of record and further argues that the PAR does not meet the claims limitations (see Applicant’s Remarks).
In response, the Examiner note Applicant’s arguments/amendment, however the amends do not overcome the PAR for these reasons: the PAR discloses, (Expressive Speech Translation System “ESTS” figs.1A+], that: in some implementations, as to facial movement: mouth movements, emotions, etc., takes into account speaker and/or facial recognition to determine additional data for image processing to combine to robustly correlate facial movement with speech detected in the audio data to generate more accurate duration data, performs facial measurements and calculates aspect ratios, uses various parameters associated with various biometric features, uses various image processing component techniques to reconstruct the real anchor or speaker (see [0100-0115-facial movements and [0130-0137]]), which meets the amended claims limitations; furthermore as further discloses, WANG, determines (ESTS) a first live video that is time-aligned with the live audio and obtaining a second live video comprising the live audio by matching facial movements of a real anchor who is speaking in the first live video with the live audio; comprising: determining a first parameter about the face and mouth of the real anchor in the first live video; determining a relation between a voice parameter of the live audio at each moment and facial movements of the real anchor who is speaking, and obtaining a second parameter about face when the real anchor is speaking in the live audio at the corresponding moment; obtaining a fusion parameter by fusing the first parameter and the second parameter; obtaining a reconstructed image by reconstructing the real anchor of the first live video according to the fusion parameter; and generating the second live video according to the reconstructed image; and playing the second live video in a live broadcast room (figs.1A+, [0024-0032], [0037-0045], [0053-0068], [0076-0084], [0100-0115-facial movements and [0130-0137]]); ESTS receives transcript data or text data from a source or live broadcast and generates audio/video data or segments associated specific durations of the text or transcript data of a speaker(s) (actors, performers, anchors, animated characters, etc.) and other metadata data or parameters associated with the speakers head or facial: specific biometric mouth movements; respective segments associated with a specific durations of the speech (phrase, sentence or other segments) can be aligned and played accordingly; WANG further discloses that: in some implementations, as to facial movement: mouth movements, emotions, etc., takes into account speaker and/or facial recognition to determine additional data for image processing to combine to robustly correlate facial movement with speech detected in the audio data to generate more accurate duration data, performs facial measurements and calculates aspect ratios, uses various parameters associated with various biometric features, uses various image processing component techniques to reconstruct the real anchor or speaker; in response to a question input by an audience received in the live broadcast room and other user(s) intent, (pausing) the second live video, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered and wherein the obtaining the second live video comprising the live audio by matching facial movements of the real anchor who is speaking in the first live video with the live audio comprises: determining a first parameter about the face and mouth of the real anchor in the first live video; determining a relation between a voice parameter of the live audio at each moment and facial movements of the real anchor who is speaking, and obtaining a second parameter about face when the real anchor is speaking in the live audio at the corresponding moment; obtains a fusion parameter by fusing the first parameter and the second parameter obtaining a reconstructed image by reconstructing the real anchor of the first live video according to the fusion parameter (uses various parameters and other trained models to generate or reconstruct the source speaker or a human voice response speech of the target response); and generating the second live video according to the reconstructed image ([0024-0035], [0062-0073], [0076-0085], [0100-0115-facial movements, [0130-0137] and [0140-0147]), ESTS further receives as inputs and other user(s) intent(s): generates specific segments: news, sports, weather, etc., from the source data: live-broadcast, streaming services, etc., that includes additional data and/or metadata including script data (transcript of the audio data, closed captioning, etc.), including information related to the source and/;or target language(s), identification of the speaker (e.g., the source audio for selecting a speaker embedded data for the speech and/or other information; the script data may be in the source language or the target language; if the script is in the source language the ESTS generates a speaker(s) speech accordingly; user(s) may interrogate in real-time or during live broadcast with the system and receive real-time response (tailored to the specific geographic area of the request) in the target language: speech of speaker(s), actor(s), character(s), etc., interrogation may be general or specific, e.g., “what is the weather currently like?”; a response is generate that takes into account specific facial movements: mouth, lips, etc.,; user(s) may chat with the system and share content with other user groups as desired; Further generates trained models based on the comparison of the transcript data embedded data and other performance representation generated and/or other data to reconstructed to generate the output audio or speech of the target speaker; WANG further discloses identifies specific scripts associated with the speaker speech, identifies pauses or breaks in phrase an/or sentence, response to question or queries from user(s) or group of users, to generate relevant segments for specific source speech or speaker for specific durations and play: audio-only, without video, video-only and/or combination, as desired accordingly further uses users or group profiles to process various input for specific segments outputs of specific durations and sharing as desired ([0024-0035], [0062-0073], [0076-0085], [0100-0107] and [0140-0147]), BUT appears silent as to where in response to setting duration to receive all questions input by an audience received in the live broadcast room, pausing the second live video, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered; However, in the same field of endeavor, THOMAS discloses systems and methods for navigating media assets, generating a playlist based on messages or queries (strings of text: particular characters, actors, etc., and/or other customized parameters), associated with an entity; e.g., “Romeo and Juliet in Paris”; “show me scenes involving Top Man”, “who is that man…” etc., to the system by user(s), and further sets up duration to receive all questions input by an audience received in the live broadcast room, pausing and/or trick playing various portions or set segments associated with one or more entities of the live broadcast, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered; pausing and trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users; pausing an/or trick playing a first playlist to play the second playlist and resuming, is user(s) controlled or automatically processing the desired segments or playlist accordingly (figs.1+, Abstract, [0002-0013], [0015-0036], [0064-0081] and [0086-0099]), first user(s) inputs messages: text, comments, questions, chats, etc., generates set of segments or one or more playlist for one or more entities; second user inputs, pauses set of segments to play other segments including trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users, as discussed below. Hence the amended claims do not overcome, the PARs. The amendments to the claims necessitated the new ground(s) of rejection. This office action is made FINAL.
Claim Rejections - 35 USC § 103
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
4. Claim(s) ) 1-8, 10-15 and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al (2024/0274122) in view of THOMAS et al (2018/0288490).
As to claims 1-2, WANG discloses speech translation with performance characteristics and further discloses a live broadcast method or a computer (Expressive Speech Translation System “ESTS” figs.1A+, Abstract, [0027-0033]), comprising: a storage device; at least one processor; and the storage device storing one or more programs that, when executed by the at least one processor, cause the at least one processor (ESTS) to:
obtaining (Expressive Speech Translation System “ESTS” figs.1A+, Abstract, [0027-0033]) a live broadcast text and generating a live audio based on the live broadcast text (text, script and/or closed caption of the speech); determining (ESTS) a first live video that is time-aligned with the live audio and obtaining a second live video comprising the live audio by matching facial movements of a real anchor who is speaking in the first live video with the live audio; comprising: determining a first parameter about the face and mouth of the real anchor in the first live video; determining a relation between a voice parameter of the live audio at each moment and facial movements of the real anchor who is speaking, and obtaining a second parameter about face when the real anchor is speaking in the live audio at the corresponding moment; obtaining a fusion parameter by fusing the first parameter and the second parameter; obtaining a reconstructed image by reconstructing the real anchor of the first live video according to the fusion parameter; and generating the second live video according to the reconstructed image; and playing the second live video in a live broadcast room (figs.1A+, [0024-0032], [0037-0045], [0053-0068], [0076-0084], [0100-0115-facial movements and [0130-0137]]); ESTS receives transcript data or text data from a source or live broadcast and generates audio/video data or segments associated specific durations of the text or transcript data of a speaker(s) (actors, performers, anchors, animated characters, etc.) and other metadata data or parameters associated with the speakers head or facial: specific biometric mouth movements; respective segments associated with a specific durations of the speech (phrase, sentence or other segments) can be aligned and played accordingly; WANG further discloses that in some implementations, as to facial movement: mouth movements, emotions, etc., takes into account speaker and/or facial recognition to determine additional data for image processing to combine to robustly correlate facial movement with speech detected in the audio data to generate more accurate duration data, performs facial measurements and calculates aspect ratios, uses various parameters associated with various biometric features, uses various image processing component techniques to reconstruct the real anchor or speaker
in response to a question input by an audience received in the live broadcast room and other user(s) intent, (pausing) the second live video, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered and wherein the obtaining the second live video comprising the live audio by matching facial movements of the real anchor who is speaking in the first live video with the live audio comprises: determining a first parameter about the face and mouth of the real anchor in the first live video; determining a relation between a voice parameter of the live audio at each moment and facial movements of the real anchor who is speaking, and obtaining a second parameter about face when the real anchor is speaking in the live audio at the corresponding moment; obtaining a fusion parameter by fusing the first parameter and the second parameter obtaining a reconstructed image by reconstructing the real anchor of the first live video according to the fusion parameter (uses various parameters and other trained models to generate or reconstruct the source speaker or a human voice response speech of the target response); and generating the second live video according to the reconstructed image ([0024-0035], [0062-0073], [0076-0085], [0100-0107] and [0140-0147]), ESTS further receives as inputs and other user(s) intent(s): generates specific segments: news, sports, weather, etc., from the source data: live-broadcast, streaming services, etc., that includes additional data and/or metadata including script data (transcript of the audio data, closed captioning, etc.), including information related to the source and/;or target language(s), identification of the speaker (e.g., the source audio for selecting a speaker embedded data for the speech and/or other information; the script data may be in the source language or the target language; if the script is in the source language the ESTS generates a speaker(s) speech accordingly; user(s) may interrogate in real-time or during live broadcast with the system and receive real-time response (tailored to the specific geographic area of the request) in the target language: speech of speaker(s), actor(s), character(s), etc., interrogation may be general or specific, e.g., “what is the weather currently like?”; a response is generate that takes into account specific facial movements: mouth, lips, etc.,; user(s) may chat with the system and share content with other user groups as desired; Further generates trained models based on the comparison of the transcript data embedded data and other performance representation generated and/or other data to reconstructed to generate the output audio or speech of the target speaker
WANG further discloses identifies specific scripts associated with the speaker speech, identifies pauses or breaks in phrase an/or sentence, response to question or queries from user(s) or group of users, to generate relevant segments for specific source speech or speaker for specific durations and play: audio-only, without video, video-only and/or combination, as desired accordingly further uses users or group profiles to process various input for specific segments outputs of specific durations and sharing as desired ([0024-0035], [0062-0073], [0076-0085], [0100-0107] and [0140-0147]), BUT appears silent as to where in response to setting duration to receive all questions input by an audience received in the live broadcast room, pausing the second live video, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered.
However, in the same field of endeavor, THOMAS discloses systems and methods for navigating media assets, generating a playlist based on messages or queries (strings of text: particular characters, actors, etc., and/or other customized parameters), associated with an entity; e.g., “Romeo and Juliet in Paris”; “show me scenes involving Top Man”, “who is that man…” etc., to the system by user(s), and further sets up duration to receive all questions input by an audience received in the live broadcast room, pausing and/or trick playing various portions or set segments associated with one or more entities of the live broadcast, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered; pausing and trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users; pausing an/or trick playing a first playlist to play the second playlist and resuming, is user(s) controlled or automatically processing the desired segments or playlist accordingly (figs.1+, Abstract, [0002-0013], [0015-0036], [0064-0081] and [0086-0099]), first user(s) inputs messages: text, comments, questions, chats, etc., generates set of segments or one or more playlist for one or more entities; second user inputs, pauses set of segments to play other segments including trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users.
Hence it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of THOMAS into the system of WANG to enable the user to control or play the various segments or playlist as desired.
As to claim 3, WANG further discloses wherein before playing the first response video that replies to the question, the method further comprises: obtaining a question text of the audience and a preset knowledge base of question-and answer, wherein the preset knowledge base comprises a plurality of standard questions and a plurality of standard answers corresponding to the plurality of standard questions; performing a semantic match between the question text and each standard question of the preset knowledge base and determining a target question set; obtaining a standard answer set with the standard answer corresponding to each target question of the target question set; and generating a target response of the question text according to the standard answer set and the question text by using a large language model; generating a human voice response speech of the target response and determining a second response video that is time-aligned with the human voice response speech, and obtaining a first response video comprising the human voice response speech by reconstructing facial movements of the real-person who is speaking in the live broadcast room in the second response video to correspond to the human voice response speech ([0024-0035], [0062-0073], [0076-0085], [0100-0107] and [0140-0147]), note remarks to claims 1-2, further uses templates and trained or learned model(s) to response to various questions.
As to claim 4, WANG further disclose wherein after performing the semantic match between the question text and each standard question of the preset knowledge base, the method further comprises: obtaining a manual response from a manual customer service representative to the question text in response that the question text does not match any standard question of the preset knowledge base; generating a human voice response speech for the manual response; determining a second response video that is time-aligned with the human voice response speech, and obtaining the first response video comprising the human voice response speech by reconstructing the facial movements of the real-person who is speaking in the live broadcast room in the second response video to correspond to the human voice response speech ([0024-0035], [0062-0073], [0076-0085], [0100-0107] and [0140-0147]), note remarks to claims 1-2.
As to claim 5, WANG further discloses wherein playing one or more sets of segments associated with an entities as desired and further discloses processing using speech and/or other input modalities to identify an intent and/or entity including chatting sessions ([0059-0064] and [0124-0147]) discussed above, BUT appears silent as to performing trick plays or playing the second live video in the live broadcast room, the method further comprises: generating a chat text of an audience in the live broadcast room; determining whether the chat text comprises an inquiry intention by identifying the chat text using a preset intention recognition model; setting the chat text as the question text in response that the chat text comprises the inquiry intention.
However, THOMAS further discloses pausing, performing trick plays or playing the second live video in the live broadcast room, the method further comprises: generating a chat text of an audience in the live broadcast room; determining whether the chat text comprises an inquiry intention by identifying the chat text using a preset intention recognition model; setting the chat text as the question text in response that the chat text comprises the inquiry intention ([0060-0064] and [0121-0129]), set up chat sessions including sharing of segments as desired.
Hence it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of THOMAS into the system of WANG to include additional applications to enable the users to control or play the various segments or playlist and share comments during playback of the segments with other users as desired.
As to claim 6, WANG further discloses obtaining a preset sample library of verbal tricks; generating a first audio-driven live broadcast video based on an opening script of the preset sample library, a second audio-driven live broadcast video based on a welcome script of the preset sample library, a third audio-driven live broadcast video based on a promotional script of the preset sample library, a fourth audio-driven live broadcast video based on a thank-you script of the preset sample library, and a fifth audio-driven live broadcast video based on an offline script of the preset sample library ([0024-0035], [0076-0085], [0100-0107] and [0140-0147]), note remarks to claims 1-2.
As to claim 7, WANG further discloses playing the first audio-driven live broadcast video before playing the second live video in the live broadcast room as desired, and sharing portions of segments with other groups or users; and further generating specific segments based on user profiles and/or preferences and sharing with other user accordingly including using trained models to update profile based on environment and/or the received inputs segment generations if user(s) or users profile changes; the number of new audiences joining the live broadcast room within a first preset duration is greater than a second preset threshold during the second live video is playing in the live broadcast room; pausing the second live video and playing the third audio-driven live broadcast video in response that a time duration from a start of the live broadcast meets a first preset condition; pausing the second live video and playing the fourth audio-driven live broadcast video in response that an interactive operation between the audience and the anchor meets a second preset condition; pausing the second live video and playing the fifth audio-driven live broadcast video in response that remaining time of the live broadcast is equal to a second preset duration. ([0024-0035], [0076-0085], [0100-0107] and [0140-0147]) room effect and/or other voices; BUT appears silent as to pausing the second live video and playing the second audio-driven live broadcast video based on a first predetermined condition, the first predetermined condition comprising a duration of a new audience joining the live broadcast room is greater than a first preset threshold during the second live video is playing in the live broadcast room, and the number of new audiences joining the live broadcast room within a first preset duration is greater than a second preset threshold during the second live video is playing in the live broadcast room; pausing the second live video and playing the third audio-driven live broadcast video in response that a time duration from a start of the live broadcast meets a first preset condition; pausing the second live video and playing the fourth audio-driven live broadcast video in response that an interactive operation between the audience and the anchor meets a second preset condition; pausing the second live video and playing the fifth audio-driven live broadcast video in response that remaining time of the live broadcast is equal to a second preset duration.
However, in the same field of endeavor, THOMAS discloses setting up a duration questions and setting up a chat sessions in the live broadcast room, pausing and/or trick playing various portions or set segments associated with one or more entities of the live broadcast, and playing a first response video that replies to the question, the first response video comprising a human voice reply and continuous image frames showing the corresponding facial movements of the real anchor; continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered; pausing and trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users; pausing an/or trick playing a first playlist to play the second playlist and resuming, is user(s) controlled or automatically processing the desired segments or playlist accordingly (figs.1+, Abstract, [0002-0013], [0015-0036], [0064-0081] and [0086-0099]), first user(s) inputs messages: text, comments, questions, chats, etc., generates set of segments or one or more playlist for one or more entities; second user inputs, pauses set of segments to play other segments including trick playing set of segments associated with one or more entities as desired based on the received inputs: text, comments, chats, etc., in the live broadcast room from the user or users or groups of users.
Hence it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of THOMAS into the system of WANG to enable the user to control or play the various segments or playlist as desired.
As to claim 8, the claimed “A computer comprising…” is composed of the same structural elements that were discussed with respect to claims 1-2.
Claim 10 is met as previously discussed in claim 3.
Claim 11 is met as previously discussed in claim 4.
Claim 12 is met as previously discussed in claim 5.
Claim 13 is met as previously discussed in claim 6.
Claim 14 is met as previously discussed in claim 7.
As to claim 15, the claimed “A non-transitory…comprising…” is composed of the same structural elements that were discussed with respect to claims 1-2.
Claim 17 is met as previously discussed in claim 4.
Claim 18 is met as previously discussed in claim 5.
Claim 19 is met as previously discussed in claim 6.
Claim 20 is met as previously discussed in claim 7.
Conclusion
5. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
6. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANNAN Q SHANG whose telephone number is (571)272-7355. The examiner can normally be reached Monday-Friday 7-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BRUCKART BENJAMIN can be reached at 571-272-3982. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNAN Q SHANG/ Primary Examiner, Art Unit 2424
ANNAN Q. SHANG
