CTNF 18/342,705 CTNF 98988 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1 – 2, 5 – 6, 10 – 12, 15 – 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent No. 11,990,132 B2 to Chenguang Zhu et al. (hereinafter Zhu) in view of U.S. Patent Application Publication No. 2021/0375291 A1 to Nanshan Zeng et al. (hereinafter Zeng) . Regarding claim 1, Zhu teaches an apparatus comprising: interface circuitry; machine readable instructions; and programmable circuitry to at least one of instantiate or execute the machine readable instructions to: (Zhu teaches a system implemented on a variety of computer systems including processors, interfaces, and the capability to execute machine-readable instructions. Zhu at 42:1 - 42:28.) adjust a language model based on a terminology utilized in a first context data; (Zhu at 27:23 - 28:19 teaches fine-tuning training of all components to enhance summarization performance by incorporating contextual information.) generate a conversation summary from a transcription and a human controlled variable; (Zhu at 24:55 - 25:23 teaches generating a meeting summary including user-identified portions of a transcript (i.e., a human controlled variable.)) Zhu alone, however, does not teach machine readable instructions to “extract a semantic entity from the conversation summary and a second context data, the second context data indicative of an input associated with a conferencing environment; and summarize the semantic entity and the second context data using the adjusted language model.” In a similar field of endeavor (e.g., generation of a summary of a meeting/conversation using semantic analysis), Zeng teaches [extracting] a semantic entity from the conversation summary and a second context data, the second context data indicative of an input associated with a conferencing environment; (Zeng teaches extracting keywords, topics, and entities (i.e., semantic entities) and enriching the transcript using external knowledge. Further, Zeng teaches summarization of specific individuals or participants including summarization across multiple related meetings including meeting documents, emails, etc. that are external to the transcript (i.e., a second context indicative of an input associated with a meeting/conference). Zeng at ¶¶ [0065] - [0067].) and [summarizing] the semantic entity and the second context data using the adjusted language model. (Zeng teaches generating a summary of the specific entities using the transcript and external knowledge (i.e., second context.) Zeng at ¶¶ [0065] - [0067].) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu with the teachings of Zeng to provide the limitations of claim 1. Doing so would have provided improved outputs for summaries as recognized by Zeng at ¶ [0033]. Further, a person of ordinary skill in the art would have found it obvious to combine Zeng and Zhu due to their significantly overlapping fields of endeavor and similar goals. Regarding claim 2, Zhu in view of Zeng (hereinafter Zhu-Zeng) teaches all the limitations of claim 1 as laid out above. Further, Zeng teaches the apparatus of claim 1, wherein the terminology utilized in the first context data has a theme and is extracted from the first context data to re-learn representations of the language model. (Zeng teaches using a token-level understanding within each turn of a meeting summarization process (i.e., terminology is extracted) wherein each turn corresponds to a speaker and an utterance (i.e., a theme) and the training process incorporates the information of the turns in. Zeng at ¶¶ [0478] - [0483]. Further, Zeng teaches summaries are generated based on a specific theme, topic, task, or project wherein the summaries are generated across a plurality of meetings (i.e., terminology extracted meeting a theme across a plurality of meetings.) Zeng at ¶¶ [0512] - [0530].) Regarding claim 5, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Further, Zeng teaches the apparatus of claim 1, wherein the human controlled variable is at least one of a window of time, a word to focus on, a phrase to focus on, or an entity to focus on. (Zeng teaches summaries are generated based on a specific theme, topic, task, or project wherein the summaries are generated across a plurality of meetings (i.e., a theme, topic, task, or project is, in essence, a word, phrase, or entity to focus on) Zeng at ¶¶ [0512] - [0530].) Regarding claim 6, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Further, Zeng teaches the apparatus of claim 1, wherein, to generate the summary of the semantic entity and the second context data using the adjusted language model, the programmable circuitry is to: collect transcriptions from a window of time of the conferencing environment; analyze the conferencing environment using the adjusted language model, the conversation summary, and the extracted semantic entity; and generate a summary of the conferencing environment using the analysis. (Zeng teaches a portion of the audio recording where multiple speakers are speaking at the same time (i.e., a window of time of the conferencing environment) wherein the audio is split into multiple streams, multiple transcripts, and summarized (i.e., multiple transcripts are collected, analyzed, and the conference (i.e., the conference environment) is summarized based on the collected transcripts.) Zeng at ¶¶ [0558] - [0559].) Regarding claim 10, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Further, Zeng teaches the apparatus of claim 1, wherein the programmable circuitry is to: retrieve a keyword or phrase from an input; (Zeng teaches a user defining keywords or phrases that are tagged and used within the summarization of content. Zeng at ¶¶ [0073] - [0079].) and monitor usage of the retrieved keyword or phrase when generating the conversation summary. (Zeng teaches the keyword/phrase is highlighted or given some level of importance (i.e., the usage is monitored in generation.). Zeng at ¶¶ [0073] - [0079].) Regarding claim 11, Zhu teaches A non-transitory computer readable medium comprising instructions that, when executed, cause a machine to at least: adjust a language model based on a terminology utilized in a first context data; (Zhu at 27:23 - 28:19 teaches fine-tuning training of all components to enhance summarization performance by incorporating contextual information.) generate a conversation summary from a transcription and a human controlled variable; (Zhu at 24:55 - 25:23 teaches generating a meeting summary including user-identified portions of a transcript (i.e., a human controlled variable.)) Zhu alone, however, does not teach machine readable instructions to “extract a semantic entity from the conversation summary and a second context data, the second context data indicative of an input associated with a conferencing environment; and summarize the semantic entity and the second context data using the adjusted language model.” In a similar field of endeavor (e.g., generation of a summary of a meeting/conversation using semantic analysis), Zeng teaches [extracting] a semantic entity from the conversation summary and a second context data, the second context data indicative of an input associated with a conferencing environment; (Zeng teaches extracting keywords, topics, and entities (i.e., semantic entities) and enriching the transcript using external knowledge. Further, Zeng teaches summarization of specific individuals or participants including summarization across multiple related meetings including meeting documents, emails, etc. that are external to the transcript (i.e., a second context indicative of an input associated with a meeting/conference). Zeng at ¶¶ [0065] - [0067].) and [summarizing] the semantic entity and the second context data using the adjusted language model. (Zeng teaches generating a summary of the specific entities using the transcript and external knowledge (i.e., second context.) Zeng at ¶¶ [0065] - [0067].) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu with the teachings of Zeng to provide the limitations of claim 11. Doing so would have provided improved outputs for summaries as recognized by Zeng at ¶ [0033]. Further, a person of ordinary skill in the art would have found it obvious to combine Zeng and Zhu due to their significantly overlapping fields of endeavor and similar goals. Regarding claim 12, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Further, Zeng teaches the non-transitory computer readable medium of claim 11, wherein the terminology utilized in the first context data has a theme and is extracted from the first context data to re-learn representations of the language model. (Zeng teaches using a token-level understanding within each turn of a meeting summarization process (i.e., terminology is extracted) wherein each turn corresponds to a speaker and an utterance (i.e., a theme) and the training process incorporates the information of the turns in. Zeng at ¶¶ [0478] - [0483]. Further, Zeng teaches summaries are generated based on a specific theme, topic, task, or project wherein the summaries are generated across a plurality of meetings (i.e., terminology extracted meeting a theme across a plurality of meetings.) Zeng at ¶¶ [0512] - [0530].) Regarding claim 15, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Further, the non-transitory computer readable medium of claim 11, wherein the human controlled variable is at least one of a window of time, a word to focus on, a phrase to focus on, or an entity to focus on. (Zeng teaches summaries are generated based on a specific theme, topic, task, or project wherein the summaries are generated across a plurality of meetings (i.e., a theme, topic, task, or project is, in essence, a word, phrase, or entity to focus on) Zeng at ¶¶ [0512] - [0530].) Regarding claim 16, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Further, the non-transitory computer readable medium of claim 11, wherein, to generate the summary of the semantic entity and the second context data using the adjusted language model, the programmable circuitry is to: collect transcriptions from a window of time of the conferencing environment; analyze the conferencing environment using the adjusted language model, the conversation summary, and the extracted semantic entity; and generate a summary of the conferencing environment using the analysis. (Zeng teaches a portion of the audio recording where multiple speakers are speaking at the same time (i.e., a window of time of the conferencing environment) wherein the audio is split into multiple streams, multiple transcripts, and summarized (i.e., multiple transcripts are collected, analyzed, and the conference (i.e., the conference environment) is summarized based on the collected transcripts.) Zeng at ¶¶ [0558] - [0559].) Regarding claim 20, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Further, the non-transitory computer readable medium of claim 11, wherein the programmable circuitry is to: retrieve a keyword or phrase from an input; (Zeng teaches a user defining keywords or phrases that are tagged and used within the summarization of content. Zeng at ¶¶ [0073] - [0079].) and monitor usage of the retrieved keyword or phrase when generating the conversation summary. (Zeng teaches the keyword/phrase is highlighted or given some level of importance (i.e., the usage is monitored in generation.). Zeng at ¶¶ [0073] - [0079].) 07-22-aia AIA Claim s 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu-Zheng as applied to claim s 1 and 11 above, and further in view of U.S. Patent Application Publication No. 2022/0215052 A1 to Vikram Chalana (hereinafter Chalana) . Regarding claim 3, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Further, Zeng teaches the apparatus of claim 1, wherein, to adjust the language model, the programmable circuitry is to: create a copy of the first context data; add noise to the copy of the first context data; (Zeng teaches synthesizing examples for training the system which includes grammatical error correction data (i.e., the data is noisy by including grammatical errors, and their corrections) to address a lack of task-specific data. Zeng at ¶¶ [0066] - [0069].) Zhu-Zeng, however, do not alone teach all the limitations of claim 3. In a similar field of endeavor (e.g., generation of summaries using semantic analysis), Chalana teaches [retraining] the language model using the first context data and the copy of the first context data including the noise. (Chalana teaches retraining a neural network for summarizing transcripts by including a user change to a transcript and retraining the neural network accordingly. (i.e., retraining the copy of the transcript including a user change (noise)). Chalana at ¶¶ [0062] - [0070].) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Chalana to provide the limitations of claim 3. Doing so would have made the generation of the summary more efficient, faster, and require fewer computation resources as recognized by Chalana at ¶ [0039]. Further, a person of ordinary skill in the art would have found it obvious to combine Zhu-Zeng and Chalana due to their significantly overlapping fields of endeavor and similar goals. Regarding claim 13, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Further, Zeng teaches the non-transitory computer readable medium of claim 11, wherein, to adjust the language model, the programmable circuitry is to: create a copy of the first context data; add noise to the copy of the first context data; (Zeng teaches synthesizing examples for training the system which includes grammatical error correction data (i.e., the data is noisy by including grammatical errors, and their corrections) to address a lack of task-specific data. Zeng at ¶¶ [0066] - [0069].) Zhu-Zeng, however, do not alone teach all the limitations of claim 3. In a similar field of endeavor (e.g., generation of summaries using semantic analysis), Chalana teaches [retraining] the language model using the first context data and the copy of the first context data including the noise. (Chalana teaches retraining a neural network for summarizing transcripts by including a user change to a transcript and retraining the neural network accordingly. (i.e., retraining the copy of the transcript including a user change (noise)). Chalana at ¶¶ [0062] - [0070].) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Chalana to provide the limitations of claim 13. Doing so would have made the generation of the summary more efficient, faster, and require fewer computation resources as recognized by Chalana at ¶ [0039]. Further, a person of ordinary skill in the art would have found it obvious to combine Zhu-Zeng and Chalana due to their significantly overlapping fields of endeavor and similar goals . 07-22-aia AIA Claim s 4, 7 – 9, 14, and 17 - 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu-Zeng as applied to claim s 1 and 11 above, and further in view of U.S. Patent No. 12,277,767 B2 to Hailin Jin et al. (hereinafter Jin) . Regarding claim 4, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Zhu-Zeng, however, do not teach all the limitations of claim 4. In a similar field of endeavor, (e.g., generation of summaries using semantic analysis of segmented video portions and transcripts), Jin teaches the apparatus of claim 1, wherein, to generate the conversation summary, the programmable circuitry is to: embed sentences from the transcription into a model; run a clustering algorithm on the model to identify clusters; and find the sentences closest to a centroid of each cluster. (Jin teaches generating text embeddings (i.e., sentence embeddings) and clustering the embeddings to finder the embeddings closest to the centroid of the cluster as part of a process of summarization. Jin at 14:50 - 15:14.) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Jin to provide the limitations of claim 4. Doing so would have allowed for processing and summarization of long-form videos such as livestreams (or meetings, for example) which allows the users to understand and consume videos efficiently as recognized by Jin at 2:48 - 3:33. Regarding claim 7, Zhu-Zeng teaches all the limitations of claim 1 as laid out above. Zhu-Zeng, however, do not teach all the limitations of claim 7. In a similar field of endeavor, (e.g., generation of summaries using semantic analysis of segmented video portions and transcripts), Jin teaches the apparatus of claim 1, wherein the programmable circuitry is to: sample a visual sequence associated with the conferencing environment; (Jin teaches extracting information from a video for summarization along with a transcription component (i.e., sampling a visual sequence.) Jin at 13:17 - 14:37 and Fig. 6.) encode the visual sequence and the transcription; and resample the encoded visual sequence. (Further, Jin teaches performing cross correlation between a generated transcript and video features (i.e., encoding the visual sequence and the transcription) wherein the cross correlation is used to perform the final summarization (i.e., the encoded sequence is resampled for summarization). Jin at 13:17 - 14:37 and Fig. 6.) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Jin (hereinafter Zhu-Zheng-Jin) to provide the limitations of claim 7. Doing so would have allowed for processing and summarization of long-form videos such as livestreams (or meetings, for example) which allows the users to understand and consume videos efficiently as recognized by Jin at 2:48 - 3:33. Regarding claim 8, Zhu-Zeng-Jin teaches all the limitations of claim 7 as laid out above. Further, Jin teaches the apparatus of claim 7, wherein the programmable circuitry is to sample the visual sequence via K-means clustering. (Jin teaches performing visual summarization by extracting frames using K-means clustering. Jin at 15:15-15:24.) Regarding claim 9, Zhu-Zeng-Jin teaches all the limitations of claim 7 as laid out above. Further, Jin teaches the apparatus of claim 7, wherein, to resample the encoded visual sequence, the programmable circuitry is to: obtain a variable number of features from the encoded visual sequence and the encoded transcription; (Jin teaches extracting sentence embeddings (i.e., features) where N is the number of sentences (i.e., the number of sentences depends on the input transcript, and therefore the number of features from the generated transcript cross-correlated with the video sequence is variable.) Jin at 15:1 - 15:24.) and select a representative fixed number of frames as outputs. (Further, Jin teaches extracting key frames of the video (i.e., visual sequence) as representative of the video wherein the frames are selected each for a plurality of visual feature clusters. (i.e., the number of frames selected as outputs is fixed to the number of feature clusters.) Jin at 15:15 - 15:24.) Regarding claim 14, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Zhu-Zeng, however, do not teach all the limitations of claim 14. In a similar field of endeavor, (e.g., generation of summaries using semantic analysis of segmented video portions and transcripts), Jin teaches the non-transitory computer readable medium of claim 11, wherein, to generate the conversation summary, the programmable circuitry is to: embed sentences from the transcription into a model; run a clustering algorithm on the model to identify clusters; and find the sentences closest to a centroid of each cluster. (Jin teaches generating text embeddings (i.e., sentence embeddings) and clustering the embeddings to finder the embeddings closest to the centroid of the cluster as part of a process of summarization. Jin at 14:50 - 15:14.) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Jin to provide the limitations of claim 14. Doing so would have allowed for processing and summarization of long-form videos such as livestreams (or meetings, for example) which allows the users to understand and consume videos efficiently as recognized by Jin at 2:48 - 3:33. Regarding claim 17, Zhu-Zeng teaches all the limitations of claim 11 as laid out above. Zhu-Zeng, however, do not teach all the limitations of claim 17. In a similar field of endeavor, (e.g., generation of summaries using semantic analysis of segmented video portions and transcripts), Jin teaches the non-transitory computer readable medium of claim 11, wherein the programmable circuitry is to: sample a visual sequence associated with the conferencing environment; (Jin teaches extracting information from a video for summarization along with a transcription component (i.e., sampling a visual sequence.) Jin at 13:17 - 14:37 and Fig. 6.) encode the visual sequence and the transcription; and resample the encoded visual sequence. (Further, Jin teaches performing cross correlation between a generated transcript and video features (i.e., encoding the visual sequence and the transcription) wherein the cross correlation is used to perform the final summarization (i.e., the encoded sequence is resampled for summarization). Jin at 13:17 - 14:37 and Fig. 6.) It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date to combine the teachings of Zhu-Zeng with the teachings of Jin (hereinafter Zhu-Zheng-Jin) to provide the limitations of claim 17. Doing so would have allowed for processing and summarization of long-form videos such as livestreams (or meetings, for example) which allows the users to understand and consume videos efficiently as recognized by Jin at 2:48 - 3:33. Regarding claim 18, Zhu-Zeng-Jin teaches all the limitations of claim 17 as laid out above. Further, Jin teaches the non-transitory computer readable medium of claim 17, wherein the programmable circuitry is to sample the visual sequence via K-means clustering. (Jin teaches performing visual summarization by extracting frames using K-means clustering. Jin at 15:15-15:24.) Regarding claim 19, Zhu-Zeng-Jin teaches all the limitations of claim 17 as laid out above. Further, Jin teaches the non-transitory computer readable medium of claim 17, wherein, to resample the encoded visual sequence, the programmable circuitry is to: obtain a variable number of features from the encoded visual sequence and the encoded transcription; (Jin teaches extracting sentence embeddings (i.e., features) where N is the number of sentences (i.e., the number of sentences depends on the input transcript, and therefore the number of features from the generated transcript cross-correlated with the video sequence is variable.) Jin at 15:1 - 15:24.) and select a representative fixed number of frames as outputs. (Further, Jin teaches extracting key frames of the video (i.e., visual sequence) as representative of the video wherein the frames are selected each for a plurality of visual feature clusters. (i.e., the number of frames selected as outputs is fixed to the number of feature clusters.) Jin at 15:15 - 15:24.) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAMERON KENNETH YOUNG whose telephone number is (703)756-1527. The examiner can normally be reached Mon - Fri, 9:00 AM - 5:00 PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /CAMERON KENNETH YOUNG/Examiner, Art Unit 2655 /ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655 Application/Control Number: 18/342,705 Page 2 Art Unit: 2655 Application/Control Number: 18/342,705 Page 3 Art Unit: 2655 Application/Control Number: 18/342,705 Page 4 Art Unit: 2655 Application/Control Number: 18/342,705 Page 5 Art Unit: 2655 Application/Control Number: 18/342,705 Page 6 Art Unit: 2655 Application/Control Number: 18/342,705 Page 7 Art Unit: 2655 Application/Control Number: 18/342,705 Page 9 Art Unit: 2655 Application/Control Number: 18/342,705 Page 10 Art Unit: 2655 Application/Control Number: 18/342,705 Page 11 Art Unit: 2655 Application/Control Number: 18/342,705 Page 12 Art Unit: 2655 Application/Control Number: 18/342,705 Page 13 Art Unit: 2655 Application/Control Number: 18/342,705 Page 14 Art Unit: 2655
Prosecution Insights