DETAILED ACTION
This action is in response to communication filed on 06 July 2026. Claims 1-5, 8-9, 11-12, 15-16 and 18-19 are amended. No claim has been added or canceled. Claims 1-20 are pending in the application and have been considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06 July 2026 has been entered.
Response to Arguments
Applicant argues that [“As such, claim 1 and its dependent claims should be found patentable over the cited references. For similar reasons, claims 9 and 16, and their dependent claims, should also be found patentable over the cited references” (Page 12)].
The argument described above has been considered and are persuasive. Therefore, rejection has been withdrawn. However, upon further search and consideration, a new ground of rejection is made, citing the new reference BUCKLEY et al. (US11334622B1) (see new claim 1 rejection below).
Applicant argues that [“As such, claim 1 [Examiner assumes a typo, and that the applicant is referring to claim 8] and its dependent claims should be found patentable over the cited references. For similar reasons, claims 4 and 18 should also be found patentable over the cited references” (Page 13)].
The argument described above has been considered and are persuasive. Therefore, rejection has been withdrawn. However, upon further search and consideration, a new ground of rejection is made, citing the new reference GILSON et al. (US20120059954A1) (see claim 8 rejection below).
Applicant argues that [“As such, claim 1 [Examiner assumes a typo, and that the applicant is referring to claim 15] and its dependent claims should be found patentable over the cited references. For similar reasons, claims 5 and 12 should also be found patentable over the cited references” (Page 14].
The argument described above has been considered and are persuasive. Therefore, rejection has been withdrawn. However, upon further search and consideration, a new ground of rejection is made, citing the new reference MANN et al. (US20210342785A1) (see claim 15 rejection below).
Applicant argues that [“As such, claim 6 should be found patentable over the cited references. For similar reasons, claims 13 and 20 should also be found patentable over the cited references” (Page 16]. Examiner respectfully disagrees.
JO teaches metadata created from video and video script contents: “More specifically, in this example, in an N-screen environment, TV screen 810 plays, using a video playback program, a video from which system 100 previously automatically generated annotation and narrative metadata. As the video plays on TV screen 810, the video is also simultaneously or concurrently played on mobile device 820. Mobile device 820 can be configured to synchronize the video playing on TV screen 810 with the video playing on mobile device 820 in real time using socket communication 830. According to some embodiments of the present invention, mobile device 820 can be configured to display story information related to the synchronized video on the mobile device in synchronism with the video on TV screen 810” (see JO, fig. 8, par. 0053). Thus, the video playing on the mobile device will show the story information as the video is playing (inserting the information). JO then further teaches face bounding boxes overlaid on the faces of the important persons in the story, and then displaying (inserting) story information and relationship between people, away from the boxes overlaid on the faces of the important people “In one example of such displayed story information related to the synchronized video, face bounding boxes 840 can be overlaid on the video displayed on mobile device 820 to mark a face of a person who is important in the story in a particular scene between characters appearing in the video, as indicated in the annotation metadata for that video scene. Another example of such displayed story information related to the synchronized video includes summaries of video story information from the narrative metadata, such as character relationships 850 and scene story summary 860. Display of character relationships 850, particularly of characters of importance in the video's story, permits easy recognition of the story of the video by a human viewer and understanding of relationships between different characters by this viewer. Scene story summary 860 summarizes a particular scene using the narrative metadata that was created based on that particular scene. By directly providing access to additional story information related to the synchronized video on mobile device 820, a human viewer may be permitted to learn such additional information without having stop the video from playing and then search the Internet for the particular information. Instead, directly providing access to story information related to the synchronized video on mobile device 820 creates a seamless viewing experience for the user” (JO, fig. 8, par. 0054). Thus, the story summary is not inserted on the TV screen playing the video, but it is inserted on the mobile device where it is synchronized to play the video.
Thus, the combination of LEE, JUAN, BUCKLEY and JO adequately discloses applicant's claimed limitation. Examiner respectfully reminds Applicants that during examination, the claims must be interpreted as broadly as their terms reasonably allow. In re American Academy of Science Tech Center, 367 F.3d 1359, 1369, 70 U.S.P.Q.2d 1827, 1834 (Fed. Cir. 2004).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1).
As to claim 1, LEE teaches one or more non-transitory computer storage media storing computer-useable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations (see fig. 4, par. 0006, wherein FIG. 4 shows an example system 400 that is configured to generate a video summary for a video recording of a virtual conference; as taught by LEE)
comprising: generating, based on applying a representation of an input video to a language model (see par. 0073, The chat and video conference provider 310 includes a model store 420. The model store 420 stores different artificial intelligence or machine learning (AI/ML) models that can be used during the process of generating a video summary for a video recording of a virtual conference. Various types of models or artificial intelligence algorithms may be used in example systems … Some AI/ML models are generative AI models, such as generative pre-trained transformer (GPT), Text-to-Text Transfer Transformer (T5), Bidirectional and Auto-Regressive Transformer (BART), Bidirectional Encoder Representations from Transformer (BERT), their variations, or other large language models (LLMs) or foundation models (FM). The AI/ML models in the model store 420 can be supervised or unsupervised learning models; as taught by LEE);
a trimmed version of the input video comprising a plurality of video segments of the input video (see par. 0020, wherein the communication platform determines portions in the transcript that match or are most relevant to the text or audio summary. Since the transcript includes time stamps, time ranges for the transcript portions can be identified. A first set of video portions corresponding to the identified time ranges are also identified. The first set of video portions identified via transcript can be mapped to the multiple portions of the audio summary. The communication platform thus identifies a first set of correspondences between the first set of video portions from the video recording and the text or audio summary; see also par. 0021, wherein in parallel or in series, the communication platform also identifies a second set of correspondences between a second set of video portions of the video recording and the text or audio summary. The communication platform analyzes the image data in the video recording and identifies the video images that match or are most relevant to the text or audio summary. Video portions that include these images are the second set of video portions mapped to the text or audio summary to form the second set of correspondences; see also par. 0022, wherein the communication platform identifies key moments or highlights from the first set of correspondences and the second set of correspondences; see also par. 0023, wherein this example provides a video summary of a video recording. The video summary includes an audio narrating the summary of the meeting along with key moments or highlights displaying in video; as taught by LEE);
and a transcript of the plurality of video segments (see par. 0019, wherein the communication platform includes a summarization model for generating a text summary for the video recording; as taught by LEE).
LEE does not expressly teach generating, based on applying a representation of at least a portion of the transcript to the language model, a representation of a caption from the transcript for a video segment of the plurality of video segments, the caption comprising selected words for emphasis selected from a larger portion of the transcript corresponding to the video segment; and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video.
In similar field of endeavor, JUAN teaches generating, based on applying a representation of at least a portion of the transcript to the language model, a representation of a caption from the transcript for a video segment of the plurality of video segments (see figs. 4-6, par. 0091, wherein using one or more of the techniques provided herein, one or more tasks may be performed automatically. The one or more tasks may comprise: (i) generation of closed captions, of the first video 502, in the first language (e.g., original language of the first video 502) using the transcript 506, wherein the closed captions may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; (ii) generation of subtitles, of the first video 502, in one or more languages different from the first language, wherein the subtitles may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; as taught by JUAN);
and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video (see fig. 5I, 0090, based upon a selection of the fifth selectable input 515 associated with the subtitle viewing option, the video interface may present the first video 502 (in conjunction with original audio of the first video 502, for example) in conjunction with subtitles in the target language. For example, the subtitles may be generated based upon the translated transcript 516 (in the target language); as taught by JUAN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE apparatus to include the teachings of JUAN for generating, based on applying a representation of at least a portion of the transcript to the language model, a representation of a caption for a video segment of the plurality of video segments; and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video. Such a person would have been motivated to make this combination as it is beneficial for the user to be able to easily add relevant information about video contents to the video as metadata or subtitles as it can be difficult to generate that solely from a video in a video format. For example, even if characters or objects relevant to a story told in a video have been recognized, it remains difficult to designate a title for such characters or object or to infer a relationship between characters, objects, or events (see also JUAN, par. 0001).
LEE and JUAN do not expressly teach the caption comprising selected words for emphasis selected from a larger portion of the transcript corresponding to the video segment.
In similar field of endeavor, BUCKLEY teaches the caption comprising selected words for emphasis selected from a larger portion of the transcript corresponding to the video segment (see col. 2, ll. 35-39, wherein in some embodiments, subtitles/captions can be generated automatically by the software system from transcripts or amalgamated transcripts; as taught by BUCKLEY).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE and JUAN apparatus to include the teachings of BUCKLEY wherein the caption comprising selected words for emphasis selected from a larger portion of the transcript corresponding to the video segment. Such a person would have been motivated to make this combination as such a phase (or phases) is undertaken for varying purposes, among which is to make audio, video, and/or audio/video content easier to work with, more navigable, more easily searched, and more orderly during the editing process, such that it enables as much efficiency, ease, swiftness, and/or clarity as possible during the editing process, and being of particular importance to the part of the editing process that involves the identification of moments in audio, video, and/or audio/video content that are of interest or disinterest to any person or persons undertaking the editing process. Such a phase (or phases) is also helpful to and supports the execution of any part of the editing process that involves the subtitling/captioning of audio, video, and/or audio/video content (see BUCKLEY, col. 1, ll. 32-45).
Claims 2, 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of ITO et al. (WO2015033448A1).
As to claim 2, LEE, JUAN and BUCKLEY teach the limitations of claim 1. LEE, JUAN and BUCKLEY do not expressly teach applying a prompt to the language model to identify the selected words for emphasis as the representation of the caption for the video segment of the plurality of video segments; and applying the caption to the video segment as the selected words are spoken during at least a corresponding portion of the video segment.
In similar field of endeavor, ITO teaches applying a prompt to the language model to identify the selected words for emphasis as the representation of the caption for the video segment of the plurality of video segments; and applying the caption to the video segment as the selected words are spoken during at least a corresponding portion of the video segment (see fig. 6, page 7, ll. 7-19, wherein The subtitle scene keyword creation unit 232 extracts at least the subtitle data from the program being played back or being received (mainly the program being viewed and selected by the user by the operation) and extracts the character string of the subtitle data as the subtitle scene keyword. The retrieved subtitle scene keyword is, for example, a "noun" unit, and is stored in the database file 244 in the memory 240. The scene list creation unit 233 executes an operation when a user selects one of the keywords by manipulation while a plurality of the subtitle scene keywords described above are displayed on the display unit 134. At this time, the scene list creation unit 233 accesses the database file 243 based on the selected keyword. A scene selection item of the subtitle scene corresponding to the selected keyword is created and outputted to the display unit 134 as a scene list via the display control unit 234. Selection items at this time can be created for each program, and a plurality of selection items can be created within one program; see also page 13, ll. 27-32 to page 14, ll. 1-22; see also page 8, ll. 1-4; as taught by ITO).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of ITO for applying a prompt to the language model to identify the selected words for emphasis as the representation of the caption for the video segment of the plurality of video segments; and applying the caption to the video segment as the selected words are spoken during at least a corresponding portion of the video segment. Such a person would have been motivated to make this combination as it is beneficial for the user to be able to search the video based on a customized keyword, especially when there are many segments to the video. This narrows the user’s search result and makes it more accurate to directly go to the point of interest (see also ITO, page 2, ll. 10-30).
Claim 9 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 2. Accordingly, claim 9 is rejected for substantially the same reasons as presented above for claim 2 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim 16 amounts to the computer system comprising one or more processors and memory configured to execute computer program instructions stored on the computer storage media of claim 2. Accordingly, claim 16 is rejected for substantially the same reasons as presented above for claim 2 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of ITO et al. (WO2015033448A1) and further view of HUANG et al. (US20220353469A1).
As to claim 3, LEE, JUAN, BUCKLEY and ITO teach the limitations of claim 2. LEE, JUAN, BUCKLEY and ITO do not teach applying a highlighting effect to a subset of words of the selected words.
In similar field of endeavor, HUANG teaches applying a highlighting effect to a subset of words of the selected words (see fig. 8, par. 0111, wherein at 802, the technique 800 includes generating a highlighted transcript as a copy of the transcript with a subset of the strings highlighted. A selected string is highlighted. For example, the selected string may have been selected as important or relevant using techniques described in relation to FIG. 5, 6, 7, 11, or 13. For example, the highlighted transcript may be displayed as a copy of the transcript with the selected strings color coded with a different color (e.g., yellow or red) than the strings of the transcript that have not been selected. Other visual indications may be used to indicate which strings in the transcript have been selected; as taught by HUANG).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN, BUCKLEY and ITO apparatus to include the teachings of HUANG for applying a highlighting effect to a subset of words of the selected words. Such a person would have been motivated to make this combination as it is beneficial for the user to have the keywords highlighted so it is easier to attract the attention of the user (see HUANG, par. 0044).
Claim 10 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 3. Accordingly, claim 10 is rejected for substantially the same reasons as presented above for claim 3 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim 17 amounts to the computer system comprising one or more processors and memory configured to execute computer program instructions stored on the computer storage media of claim 3. Accordingly, claim 17 is rejected for substantially the same reasons as presented above for claim 3 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 4, 8, 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of GILSON (US20120059954A1).
As to claim 4, LEE, JUAN and BUCKLEY teach the limitations of claim 1. LEE, JUAN and BUCKLEY do not expressly teach applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments wherein the representation of the caption for the video segment of the plurality of video segments further comprises a section heading of the plurality of section headings.
In similar field of endeavor, GILSON teaches applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments wherein the representation of the caption for the video segment of the plurality of video segments further comprises a section heading of the plurality of section headings (see figs 4A-4B, pars. 0030-0033, for example, par. 0031, wherein enhanced caption 404 may include the title of the scene currently playing in the video content and/or a short description of the scene. Additionally or alternatively, as the video content continues to be played back at the alternate speed, enhanced caption 404 may be updated to indicate, for example, the current scene title and/or the current scene description; as taught by GILSON).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of GILSON wherein applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments wherein the representation of the caption for the video segment of the plurality of video segments further comprises a section heading of the plurality of section headings. Such a person would have been motivated to make this combination as in currently existing multimedia content distribution systems and multimedia content consumption systems, captioning functionalities may be implemented to provide various ways of including and/or displaying captions representing transcripts of the audio portions of various multimedia programs. As technology continues to develop, however, it may be desirable to provide more convenient, usable, and/or advanced captioning functionalities (see GILSON, par. 0002).
As to claim 8, LEE teaches a method comprising: generating, based on applying a representation of an input video to a language model (see fig. 4, par. 0006, wherein FIG. 4 shows an example system 400 that is configured to generate a video summary for a video recording of a virtual conference; see also par. 0073, wherein the chat and video conference provider 310 includes a model store 420. The model store 420 stores different artificial intelligence or machine learning (AI/ML) models that can be used during the process of generating a video summary for a video recording of a virtual conference. Various types of models or artificial intelligence algorithms may be used in example systems … Some AI/ML models are generative AI models, such as generative pre-trained transformer (GPT), Text-to-Text Transfer Transformer (T5), Bidirectional and Auto-Regressive Transformer (BART), Bidirectional Encoder Representations from Transformer (BERT), their variations, or other large language models (LLMs) or foundation models (FM). The AI/ML models in the model store 420 can be supervised or unsupervised learning models; as taught by LEE);
a trimmed version of the input video comprising a plurality of video segments of the input video (see par. 0020, wherein the communication platform determines portions in the transcript that match or are most relevant to the text or audio summary. Since the transcript includes time stamps, time ranges for the transcript portions can be identified. A first set of video portions corresponding to the identified time ranges are also identified. The first set of video portions identified via transcript can be mapped to the multiple portions of the audio summary. The communication platform thus identifies a first set of correspondences between the first set of video portions from the video recording and the text or audio summary; see also par. 0021, wherein in parallel or in series, the communication platform also identifies a second set of correspondences between a second set of video portions of the video recording and the text or audio summary. The communication platform analyzes the image data in the video recording and identifies the video images that match or are most relevant to the text or audio summary. Video portions that include these images are the second set of video portions mapped to the text or audio summary to form the second set of correspondences; see also par. 0022, wherein the communication platform identifies key moments or highlights from the first set of correspondences and the second set of correspondences; see also par. 0023, wherein this example provides a video summary of a video recording. The video summary includes an audio narrating the summary of the meeting along with key moments or highlights displaying in video; as taught by LEE);
and a transcript of the plurality of video segments (see par. 0019, wherein the communication platform includes a summarization model for generating a text summary for the video recording; as taught by LEE).
LEE does not expressly teach generating, based on processing a representation of at least a portion of the transcript using the language model, a representation of a caption from the transcript for a video segment of the plurality of video segments, the caption comprising a section heading for a subset of video segments of the plurality of video segments, the section heading directed to a particular theme or topic of the subset of video segments that is different from other video segments of the plurality of video segments; and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video.
In similar field of endeavor, JUAN teaches:
generating, based on processing a representation of at least a portion of the transcript using the language model, a representation of a caption from the transcript for a video segment of the plurality of video segments (see figs. 4-6, par. 0091, wherein using one or more of the techniques provided herein, one or more tasks may be performed automatically. The one or more tasks may comprise: (i) generation of closed captions, of the first video 502, in the first language (e.g., original language of the first video 502) using the transcript 506, wherein the closed captions may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; (ii) generation of subtitles, of the first video 502, in one or more languages different from the first language, wherein the subtitles may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; as taught by JUAN);
and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video (see fig. 5I, 0090, based upon a selection of the fifth selectable input 515 associated with the subtitle viewing option, the video interface may present the first video 502 (in conjunction with original audio of the first video 502, for example) in conjunction with subtitles in the target language. For example, the subtitles may be generated based upon the translated transcript 516 (in the target language); as taught by JUAN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE apparatus to include the teachings of JUAN for generating, based on applying a representation of at least a portion of the transcript to the language model, a representation of a caption for a video segment of the plurality of video segments; and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video. Such a person would have been motivated to make this combination as it is beneficial for the user to be able to easily add relevant information about video contents to the video as metadata or subtitles as it can be difficult to generate that solely from a video in a video format. For example, even if characters or objects relevant to a story told in a video have been recognized, it remains difficult to designate a title for such characters or object or to infer a relationship between characters, objects, or events (see also JUAN, par. 0001).
LEE and JUAN do not expressly teach the caption comprising a section heading for a subset of video segments of the plurality of video segments, the section heading directed to a particular theme or topic of the subset of video segments that is different from other video segments of the plurality of video segments.
In similar field of endeavor, GILSON teaches the caption comprising a section heading for a subset of video segments of the plurality of video segments, the section heading directed to a particular theme or topic of the subset of video segments that is different from other video segments of the plurality of video segments (see figs 4A-4B, pars. 0030-0033, for example, par. 0031, wherein enhanced caption 404 may include the title of the scene currently playing in the video content and/or a short description of the scene. Additionally or alternatively, as the video content continues to be played back at the alternate speed, enhanced caption 404 may be updated to indicate, for example, the current scene title and/or the current scene description; as taught by GILSON).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE and JUAN apparatus to include the teachings of GILSON wherein the caption comprising a section heading for a subset of video segments of the plurality of video segments, the section heading directed to a particular theme or topic of the subset of video segments that is different from other video segments of the plurality of video segments. Such a person would have been motivated to make this combination as in currently existing multimedia content distribution systems and multimedia content consumption systems, captioning functionalities may be implemented to provide various ways of including and/or displaying captions representing transcripts of the audio portions of various multimedia programs. As technology continues to develop, however, it may be desirable to provide more convenient, usable, and/or advanced captioning functionalities (see GILSON, par. 0002).
As to claim 11, LEE, JUAN and BUCKLEY teach the limitations of claim 8. LEE, JUAN and BUCKLEY do not expressly teach applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments, wherein the section heading is selected from f the plurality of section headings.
In similar field of endeavor, GILSON teaches applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments, wherein the section heading is selected from f the plurality of section headings (see figs 4A-4B, pars. 0030-0033, for example, par. 0031, wherein enhanced caption 404 may include the title of the scene currently playing in the video content and/or a short description of the scene. Additionally or alternatively, as the video content continues to be played back at the alternate speed, enhanced caption 404 may be updated to indicate, for example, the current scene title and/or the current scene description; as taught by GILSON).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of GILSON wherein applying a prompt to the language model to identify a plurality of section headings for each subset of video segments of the plurality of video segments, wherein the section heading is selected from f the plurality of section headings. Such a person would have been motivated to make this combination as in currently existing multimedia content distribution systems and multimedia content consumption systems, captioning functionalities may be implemented to provide various ways of including and/or displaying captions representing transcripts of the audio portions of various multimedia programs. As technology continues to develop, however, it may be desirable to provide more convenient, usable, and/or advanced captioning functionalities (see GILSON, par. 0002).
Claim 18 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 4. Accordingly, claim 18 is rejected for substantially the same reasons as presented above for claim 4 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 5, 12, 15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of MANN et al. (US20210342785A1).
As to claim 5, LEE, JUAN and BUCKLEY teach the limitations of claim 1. LEE, JUAN and BUCKLEY teach applying the caption to the video segment. However, LEE, JUAN and BUCKLEY do not expressly teach applying a prompt to the language model to identify a list of items spoken during the video segment, the representation of the caption further comprising the list of items; as the list of items are spoken during at least a corresponding portion of the video segment.
In similar field of endeavor, MANN teaches applying a prompt to the language model to identify a list of items spoken during the video segment, the representation of the caption further comprising the list of items; as the list of items are spoken during at least a corresponding portion of the video segment (see par. 1700, wherein the characteristics of the communication further include at least one participant identification, start and end time stamps, a conversation transcript, a conversation duration, a list of key words spoken in the communication, or statistics related to participants; see also par. 1706, wherein characterizing the text messages includes recording key words from the text messages; see also pars. 0529-0532 and 1098; as taught by MANN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of MANN for applying a prompt to the language model to identify a list of items spoken during the video segment, the representation of the caption further comprising the list of items; as the list of items are spoken during at least a corresponding portion of the video segment. Such a person would have been motivated to make this combination as it is beneficial for the user to provide focus on objects that are discussed in the video and are deemed important to be presented distinctly (see also MANN, pars. 0003-0010).
Claim 12 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 5. Accordingly, claim 12 is rejected for substantially the same reasons as presented above for claim 5 and based on the references’ disclosure of the necessary supporting hardware and software.
As to claim 15, LEE teaches a computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors, the computer program instructions comprising: generating, based on applying a representation of an input video to a language model (see fig. 4, par. 0006, wherein FIG. 4 shows an example system 400 that is configured to generate a video summary for a video recording of a virtual conference; see also par. 0073, wherein the chat and video conference provider 310 includes a model store 420. The model store 420 stores different artificial intelligence or machine learning (AI/ML) models that can be used during the process of generating a video summary for a video recording of a virtual conference. Various types of models or artificial intelligence algorithms may be used in example systems … Some AI/ML models are generative AI models, such as generative pre-trained transformer (GPT), Text-to-Text Transfer Transformer (T5), Bidirectional and Auto-Regressive Transformer (BART), Bidirectional Encoder Representations from Transformer (BERT), their variations, or other large language models (LLMs) or foundation models (FM). The AI/ML models in the model store 420 can be supervised or unsupervised learning models; as taught by LEE);
a trimmed version of the input video comprising a plurality of video segments of the input video (see par. 0020, wherein the communication platform determines portions in the transcript that match or are most relevant to the text or audio summary. Since the transcript includes time stamps, time ranges for the transcript portions can be identified. A first set of video portions corresponding to the identified time ranges are also identified. The first set of video portions identified via transcript can be mapped to the multiple portions of the audio summary. The communication platform thus identifies a first set of correspondences between the first set of video portions from the video recording and the text or audio summary; see also par. 0021, wherein in parallel or in series, the communication platform also identifies a second set of correspondences between a second set of video portions of the video recording and the text or audio summary. The communication platform analyzes the image data in the video recording and identifies the video images that match or are most relevant to the text or audio summary. Video portions that include these images are the second set of video portions mapped to the text or audio summary to form the second set of correspondences; see also par. 0022, wherein the communication platform identifies key moments or highlights from the first set of correspondences and the second set of correspondences; see also par. 0023, wherein this example provides a video summary of a video recording. The video summary includes an audio narrating the summary of the meeting along with key moments or highlights displaying in video; as taught by LEE);
and a transcript of the plurality of video segments (see par. 0019, wherein the communication platform includes a summarization model for generating a text summary for the video recording; as taught by LEE).
LEE does not expressly teach trigger-generating, based on applying a representation of at least a portion of the transcript to a language model a representation of a caption from the transcript for a video segment of the plurality of video segments, the caption comprising a selected list of items spoken during the video segment determined from a larger portion of the transcript corresponding to the video segment; and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video.
In similar field of endeavor, JUAN teaches:
trigger-generating, based on applying a representation of at least a portion of the transcript to a language model a representation of a caption from the transcript for a video segment of the plurality of video segments (see figs. 4-6, par. 0091, wherein using one or more of the techniques provided herein, one or more tasks may be performed automatically. The one or more tasks may comprise: (i) generation of closed captions, of the first video 502, in the first language (e.g., original language of the first video 502) using the transcript 506, wherein the closed captions may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; (ii) generation of subtitles, of the first video 502, in one or more languages different from the first language, wherein the subtitles may be displayed during playback of the first video 502 to assist a viewer in understanding dialog of the first video 502; as taught by JUAN);
and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video (see fig. 5I, 0090, based upon a selection of the fifth selectable input 515 associated with the subtitle viewing option, the video interface may present the first video 502 (in conjunction with original audio of the first video 502, for example) in conjunction with subtitles in the target language. For example, the subtitles may be generated based upon the translated transcript 516 (in the target language); as taught by JUAN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE apparatus to include the teachings of JUAN for trigger-generating, based on applying a representation of at least a portion of the transcript to a language model a representation of a caption from the transcript for a video segment of the plurality of video segments, and applying the caption to the video segment by inserting the caption from the transcript into frames of the video segment for display of the trimmed version of the input video. Such a person would have been motivated to make this combination as it is beneficial for the user to be able to easily add relevant information about video contents to the video as metadata or subtitles as it can be difficult to generate that solely from a video in a video format. For example, even if characters or objects relevant to a story told in a video have been recognized, it remains difficult to designate a title for such characters or object or to infer a relationship between characters, objects, or events (see also JUAN, par. 0001).
LEE and JUAN do not expressly teach the caption comprising a selected list of items spoken during the video segment determined from a larger portion of the transcript corresponding to the video segment.
In similar field of endeavor, MANN teaches the caption comprising a selected list of items spoken during the video segment determined from a larger portion of the transcript corresponding to the video segment (see par. 1700, wherein the characteristics of the communication further include at least one participant identification, start and end time stamps, a conversation transcript, a conversation duration, a list of key words spoken in the communication, or statistics related to participants; see also par. 1706, wherein characterizing the text messages includes recording key words from the text messages; see also pars. 0529-0532 and 1098; as taught by MANN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE and JUAN apparatus to include the teachings of MANN wherein the caption comprising a selected list of items spoken during the video segment determined from a larger portion of the transcript corresponding to the video segment. Such a person would have been motivated to make this combination as it is beneficial for the user to provide focus on objects that are discussed in the video and are deemed important to be presented distinctly (see also MANN, pars. 0003-0010).
Claim 19 amounts to the computer system comprising one or more processors and memory configured to execute computer program instructions stored on the computer storage media of claim 5. Accordingly, claim 19 is rejected for substantially the same reasons as presented above for claim 5 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 6, 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of JO (US20200125600A1).
As to claim 6, LEE, JUAN and BUCKLEY teach the limitations of claim 1. LEE, JUAN and BUCKLEY do not expressly teach applying the caption to the video segment by inserting the caption from the transcript into a region of the frames of the video segment that is different than a detected region comprising a detected face in the video segment.
In similar field of endeavor, JO teaches applying the caption to the video segment by inserting the caption from the transcript into a region of the frames of the video segment that is different than a detected region comprising a detected face in the video segment (see fig. 8, pars. 0052-0055, for example par. 0054, wherein in one example of such displayed story information related to the synchronized video, face bounding boxes 840 can be overlaid on the video displayed on mobile device 820 to mark a face of a person who is important in the story in a particular scene between characters appearing in the video, as indicated in the annotation metadata for that video scene. Another example of such displayed story information related to the synchronized video includes summaries of video story information from the narrative metadata, such as character relationships 850 and scene story summary 860. Display of character relationships 850, particularly of characters of importance in the video's story, permits easy recognition of the story of the video by a human viewer and understanding of relationships between different characters by this viewer. Scene story summary 860 summarizes a particular scene using the narrative metadata that was created based on that particular scene. By directly providing access to additional story information related to the synchronized video on mobile device 820, a human viewer may be permitted to learn such additional information without having stop the video from playing and then search the Internet for the particular information. Instead, directly providing access to story information related to the synchronized video on mobile device 820 creates a seamless viewing experience for the user; as taught by JO).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of JO applying the caption to the video segment by inserting the caption from the transcript into a region of the frames of the video segment that is different than a detected region comprising a detected face in the video segment. Such a person would have been motivated to make this combination as it is beneficial for the user to be able to easily add relevant information about video contents to the video as metadata, as it can be difficult to generate that solely from a video in a video format. For example, even if characters or objects relevant to a story told in a video have been recognized, it remains difficult to designate a title for such characters or object or to infer a relationship between characters, objects, or events (see also JO, pars. 0003-0005).
Claim 13 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 6. Accordingly, claim 13 is rejected for substantially the same reasons as presented above for claim 6 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim 20 amounts to the computer system comprising one or more processors and memory configured to execute computer program instructions stored on the computer storage media of claim 6. Accordingly, claim 20 is rejected for substantially the same reasons as presented above for claim 6 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over LEE et al. (US20250039336A1) in view of DE JUAN et al. (US20240038271A1) [hereinafter JUAN] and further in view of BUCKLEY et al. (US11334622B1) and further in view of FANCELLU (US20210326643A1).
As to claim 7, LEE, JUAN and BUCKLEY teach the limitations of claim 1. LEE, JUAN and BUCKLEY do not expressly teach identifying an image corresponding to the caption; and applying the caption to the video segment with the image corresponding to the caption.
In similar field of endeavor, FANCELLU teaches:
identifying an image corresponding to the caption (see figs. 5A-5D and 8, par. 0082, wherein as shown in FIG. 5D, the user device 210 may provide, for display via the UI 500, text information 518 in the text information area 504. In this case, the text information 518 may corresponds to caption information of the image information 516 a-516 d based on the selection of the captions icon 510; as taught by FANCELLU);
and applying the caption to the video segment with the image corresponding to the caption (see fig. 9, par. 0153, The user device 210 may provide, for display, a response that includes image information and text information that is responsive to the query. As an example, the response may be a subset of a video that corresponds to the query. As mentioned above elsewhere herein, a user of the user device 210 might not be interested in watching an entire video, and may desire to more quickly identify a pertinent section of the video. In this case, the user device 210 may provide a response that identifies the pertinent portion of the video; as taught by FANCELLU).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the LEE, JUAN and BUCKLEY apparatus to include the teachings of FANCELLU identifying an image corresponding to the caption; and applying the caption to the video segment with the image corresponding to the caption. Such a person would have been motivated to make this combination as it is beneficial for the user to have an image related to the caption to make it easier to recognize the video segment with a visual cue (see FANCELLU, par. 0002-0004).
Claim 14 amounts to the method, which is executed by the computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors of claim 7. Accordingly, claim 14 is rejected for substantially the same reasons as presented above for claim 7 and based on the references’ disclosure of the necessary supporting hardware and software.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Publication Number
Filing Date
Title
US20140222834A1
2014-02-03
Content summarization and/or recommendation apparatus and method
US20070171303A1
2004-05-17
System and method for generating audio-visual summaries for audio-visual program content
US20170169853A1
2015-12-09
Automatic Media Summary Creation Systems and Methods
US20220215052A1
2022-01-05
Summarization of video artificial intelligence method, system, and apparatus
US12231745B1
2019-01-23
Automated video summary generation using textual quotes
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KOOROSH NEHCHIRI whose telephone number is (408)918-7643. The examiner can normally be reached M-F, 11-7 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William L. Bashore can be reached at 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KOOROSH NEHCHIRI/Examiner, Art Unit 2174
/WILLIAM L BASHORE/ Supervisory Patent Examiner, Art Unit 2174
Prosecution Insights