DETAILED ACTION
Response to Arguments
Applicants’ arguments filed with respect to claims 1-8 and 21-32 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
The 101 rejection is overcome by the claim amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 21 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Gupta et al. (Pub. No. US 2024/0155071).
Regarding claim 1, Chhaya teaches a system comprising: a memory component [Para. 35, 73, and 75]; and a processing device coupled to the memory component, the processing device to perform operations comprising [Para. 35, 73, and 75]: obtaining a video (digital video 114) comprising a plurality of images (frames 210) [Para. 45 “a digital video input module 202 is configured to input a plurality of digital videos 114 that are to be used as a basis to generate the digital document 120”; Para. 48 “The information references the selected digital videos 114 having frames 210 and corresponding digital audio 212”]; selecting a pivot image (key frame 512) from the plurality of images (frames from respective action clips) [Para. 29 “The selected path is utilized by a frame location module to find key frames by mapping the nodes back to the action clips. The frame location module, for instance, locates a key frame from a collection of frames from respective action clips using a clustering technique” and “A centroid is computed for each of the frames in the action clip, and a frame that is closest to the centroid is selected as a frame that is representative of the action client, i.e., is the “key frame”]; causing a first machine learning model (model 524 trained using machine learning) to generate a descriptor (textual components 518 including a sequence of entity 520 and respective action descriptions 522) based at least in part on the pivot image (key frame 512) by at least providing the pivot image as an input to the first machine learning model (model 524 trained using machine learning), where the descriptor includes a natural language description (textual components describe entities and corresponding action descriptions) of the pivot image (key frame) including a movement (fold or folding) attribute or location (into four) attribute of an object (entity 520) depicted in the pivot image [Para. 59 “The decoding module 516 is configured to form textual components 518 that include a sequence of entity 520 and respective action descriptions 522 using a model 524 trained using machine learning (block 920). To do so, the decoding module 516 is configured to obtain the key frames 512 that are representative of the clusters along with content associated with the digital videos 114, e.g., titles, content outline, synopsis”; Para. 30 “A decoding module is then utilized by the digital document generation system to generate textual components based on the frames. The textual components describe entities and corresponding action descriptions” And “In a baking scenario, for instance, the entities are ingredients and the action descriptions are instructions involving those ingredients, e.g., “fold eggs into flour.” (movement)]; and providing the pivot image (frames) and the descriptor (entities identified for each of the frames) to a decoder (action description decoding module) [Para. 30 “The entities are identified by an entity decoding module from portions of transcripts corresponding to the frames and/or from the frames themselves, e.g., using image processing and machine-learning classifiers”; and Para. 31 “The entities identified for each of the frames are processed using machine learning along with the frames using an action description decoding module to generate action descriptions for each of the entities”].
Chhaya doesn’t explicitly teach the rest of claim limitations.
Cupta teaches wherein the decoder provides the descriptor as a prompt to a text encoder that generates, based on the prompt, a representation including an image encoding [Para. 36, and 38] capturing semantic information in the descriptor [Para. 43] that, as a result of being provided as an input to an image decoder (306) [Para. 39], causes the image decoder to generate image frames of a reconstructed video based at least in part on the image encoding and the pivot image [Para. 39, 43, 62, and 63].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya’s digital video processing system by routing Chhaya’s generated textual components 518 and selected key frame 512 to Gupta’s text-video framework as, respectively, input text 302 and the input image, so that the pre-trained text encoder and prior network generate text and image embeddings and the decoder and image-animation network generate remaining video frames conditioned on the image embedding and key frame. This modification improves Chhaya by enabling its key-frame-and-text representation to generate semantically faithful and coherent video frames while preserving direct visual control through the selected key frame, thereby predictably reconstructing video content from the descriptor and pivot image.
Claims 21 and 27 are rejected for the same reasons as claim 1.
Claims 1, 2, 21 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179).
Regarding claim 1, Chhaya teaches a system comprising: a memory component [Para. 35, 73, and 75]; and a processing device coupled to the memory component, the processing device to perform operations comprising [Para. 35, 73, and 75]: obtaining a video (digital video 114) comprising a plurality of images (frames 210) [Para. 45 “a digital video input module 202 is configured to input a plurality of digital videos 114 that are to be used as a basis to generate the digital document 120”; Para. 48 “The information references the selected digital videos 114 having frames 210 and corresponding digital audio 212”]; selecting a pivot image (key frame 512) from the plurality of images (frames from respective action clips) [Para. 29 “The selected path is utilized by a frame location module to find key frames by mapping the nodes back to the action clips. The frame location module, for instance, locates a key frame from a collection of frames from respective action clips using a clustering technique” and “A centroid is computed for each of the frames in the action clip, and a frame that is closest to the centroid is selected as a frame that is representative of the action client, i.e., is the “key frame”]; causing a first machine learning model (model 524 trained using machine learning) to generate a descriptor (textual components 518 including a sequence of entity 520 and respective action descriptions 522) based at least in part on the pivot image (key frame 512) by at least providing the pivot image as an input to the first machine learning model (model 524 trained using machine learning), where the descriptor includes a natural language description (textual components describe entities and corresponding action descriptions) of the pivot image (key frame) including a movement (fold or folding) attribute or location (into four) attribute of an object (entity 520) depicted in the pivot image [Para. 59 “The decoding module 516 is configured to form textual components 518 that include a sequence of entity 520 and respective action descriptions 522 using a model 524 trained using machine learning (block 920). To do so, the decoding module 516 is configured to obtain the key frames 512 that are representative of the clusters along with content associated with the digital videos 114, e.g., titles, content outline, synopsis”; Para. 30 “A decoding module is then utilized by the digital document generation system to generate textual components based on the frames. The textual components describe entities and corresponding action descriptions” And “In a baking scenario, for instance, the entities are ingredients and the action descriptions are instructions involving those ingredients, e.g., “fold eggs into flour.” (movement)]; and providing the pivot image (frames) and the descriptor (entities identified for each of the frames) to a decoder (action description decoding module) [Para. 30 “The entities are identified by an entity decoding module from portions of transcripts corresponding to the frames and/or from the frames themselves, e.g., using image processing and machine-learning classifiers”; and Para. 31 “The entities identified for each of the frames are processed using machine learning along with the frames using an action description decoding module to generate action descriptions for each of the entities”].
Chhaya doesn’t explicitly teach the rest of claim limitations.
Min teaches wherein the decoder provides the descriptor as a prompt (text condition y) to a text encoder (trained language model) that generates, based on the prompt (text condition y), a representation (embedding e) including an image encoding (new latent map sequence) capturing semantic information in the descriptor (text condition y) that, as a result of being provided as an input to an image decoder (trained image decoder Ω ), causes the image decoder to generate image frames (new frame) of a reconstructed video (output video) based at least in part on the image encoding (new latest map sequence) and the pivot image (input image) [Para. 36 and 37].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya’s digital-document generation system by incorporating Min’s teaching of encoding a prompt (text condition y) with a text encoder (trained language model) into a semantic representation (embedding e), conditioning an image-encoding sequence (new latent map sequence) on that representation and an input image, and decoding the sequence with an image decoder (trained image decoder), such that Chhaya’s descriptor and pivot image feed the text conditioned video generation pipeline. This medication improves Chhaya by converting its compact key frame and action description representation into reconstructed video frames while preserving pivot-image appearance and descriptor specified motion, thereby providing decoder-side video reconstruction from compressed semantic information.
Regarding claim 2, Chhaya teaches wherein the pivot image (key frame 512) depicts a conceptual element of the video (actions from the plurality of digital video 114) [Para. 49 “The digital document generation system 118 begins by locating action clips. The action clips includes frames that depict actions from the plurality of digital videos 114 (block 906)”; Para. 29 “The selected path is utilized by a frame location module to find key frames by mapping the nodes back to the action clips.” And “The frame location module, for instance, locates a key frame from a collection of frames from respective action clips using a clustering technique”].
Claims 21 and 27 are rejected for the same reasons as claim 1.
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179) in view of LEE et al. (Pub. No. US 2023/0306056).
Regarding claim 3, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, LEE teaches wherein selecting the pivot image (key frame) further comprises selecting the pivot image from the plurality of images detecting a change between two or more images of the plurality of images [Para. 100 and 113].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitation, feature as taught by LEE; because the modification enables the system to improve video compression efficiency and conceptual fidelity by using a machine learning model to pick pivot/key frames only when there is a meaningful change between video images.
Regarding claim 4, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, LEE teaches wherein the change comprises a modification to an object depicted in the two or more images that is detected, by the second machine learning model [Para. 59, 116 and121].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitation, feature as taught by LEE; because the modification enables the system to improve video compression efficiency and conceptual fidelity by using a machine learning model to pick pivot/key frames only when there is a meaningful change between video images.
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179) in view of LEE et al. (Pub. No. US 2023/0306056) further in view of LI et al. (Pub. No. US 2022/0207750).
Regarding claim 5, Chhaya in view of Min in view of LEE doesn’t explicitly teach the claim limitation.
LI teaches wherein the change comprises detecting, by the second machine learning model, an additional object relative to at least one image of the two or more images [Para. 73, and 85].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min in view of LEE to teach the claim limitation, feature as taught by LI; because the modification enables the system to improve automated visual change detection by using a second machine learning model to identify newly appearing objects when comparing one image against other images in a set.
Regarding claim 6 Chhaya in view of Min in view of LEE doesn’t explicitly teach the claim limitation.
LI teaches wherein causing the first machine learning model to generate the descriptor further comprises prompting the first machine learning model to describe a conceptual element of the video relative to the pivot image and at least one other image of the plurality of images [fig. 8 , 9 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min in view of LEE to teach the claim limitation, feature as taught by LI; because the modification enables the system to improve automated visual change detection by using a second machine learning model to identify newly appearing objects when comparing one image against other images in a set.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179) in view of Kreis et al. (Pub. No. US 2024/0171788).
Regarding claim 7, Chhaya teaches wherein the processing device further performs operations causing [Claim 11 and corresponding description].
However, Chhaya in view of Min doesn’t explicitly teach the rest of the claim limitations.
Kreis teaches, at the decoder, a third machine learning model to generate a reconstructed video by at least providing as a first input to the third machine learning model the pivot image and the descriptor, where the third machine learning model uses the pivot image and at least a portion of the descriptor to output a second plurality of images that are combined to generate the reconstructed video [Para. 17 “The method can include updating a neural network model to align a plurality of images into frames of a first video by updating at least one first temporal attention layer of the neural network model” Para. 8 “the neural network model is to generate a third video and the first video by generating at least one frame between two consecutive frames of the third video according to relative time step embedding.” Para. 9-11].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Kreis; because the modification enables the system to improve scalability at high resolutions.
Regarding claim 8, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Kreis teaches wherein the first machine learning model comprises a large language model, the second machine learning model comprises a neural network, and the third machine learning model comprises a diffusion model [Para. 2, and 11].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Kreis; because the modification enables the system to improve scalability at high resolutions.
Claims 22 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of in view of Min et al. (Pub. No. US 2024/0087179) YIN et al. (Pub. No. US 2025/0088675).
Regarding claim 22, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, YIN teaches wherein the medium further stores executable instructions, that, cause the processing device to perform operations causing a decoder executed by the endpoint to generate a reconstructed video by at least providing the descriptor and the pivot image as an input to a generative model [Para. 73 and 74].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by YIN; because the modification enables the system to improve video streaming efficiently by letting the decoder reconstruct the video from a key image plus a compact descriptor using a generative model, reducing the amount of data that must be transmitted while maintaining visual quality.
Regarding claim 23 Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, YIN teaches wherein the generative model generates intermediate frames of the reconstructed video between the pivot image and a second pivot image based at least in part on the descriptor [fig. 5 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by YIN; because the modification enables the system to improve video streaming efficiently by letting the decoder reconstruct the video from a key image plus a compact descriptor using a generative model, reducing the amount of data that must be transmitted while maintaining visual quality.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179) in view of Liu et al. (Pub. No. US 2020/0012940).
Regarding claim 24, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Liu teaches wherein obtaining the pivot image further comprises sampling frames of the video over an interval of time [Para. 4 and 50].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Liu; because the modification enables the system to improve video streaming efficiently by letting the decoder reconstruct the video from a key image plus a compact descriptor using a generative model, reducing the amount of data that must be transmitted while maintaining visual quality.
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of in view of Min et al. (Pub. No. US 2024/0087179) Shetty et al. (Pub. No. US 2016/0070962).
Regarding claim 25, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Shetty teaches wherein obtaining the pivot image further comprises causing a second machine learning model to determine the pivot image includes a conceptual element of the video [Para. 36 and 43].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Shetty; because the modification enables the system to solving how to use a few pivot frames plus rich natural language descriptions to drive a generative model that reconstructs a video.
Claims 26, 28, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of Min et al. (Pub. No. US 2024/0087179) in view of Yu et al. (Pub. No. US 2017/0127016).
Regarding claim 26, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Yu teaches wherein the medium further stores executable instructions, that, cause the processing device to perform operations causing the machine learning model to generate a second descriptor that includes a second natural language description of a relationship between the set of pivot images and at least one other pivot image obtained from the video, where the pivot image of the at least one other pivot image is provided to the machine learning model as an input [Para. 27, and 70].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Yu; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Regarding claim 28, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Yu teaches wherein the descriptor further includes a second natural language description of objects within the pivot image [fig. 1, 3 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Yu; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Regarding claim 29, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Yu teaches wherein causing the second machine learning model to generate the reconstructed video further comprises causing the second machine learning model to reconstruct a first version of the video [fig. 2, 3 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Yu; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Claims 30, 31, and 32 are rejected under 35 U.S.C. 103 as being unpatentable over Chhaya et al. (Pub. No. US 2023/0290146) in view of in view of Min et al. (Pub. No. US 2024/0087179) and Yu et al. (Pub. No. US 2017/0127016), and further in view of YIN et al. (Pub. No. US 2025/0088675).
Regarding claim 30, Chhaya in view of Min and Yu doesn’t teach the claim limitation.
However, YIN teaches wherein causing the second machine learning model to generate the reconstructed video further comprises combining a plurality of images generated by the second machine learning model based at least in part on the pivot image and the descriptor [Para. 82].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min and Yu to teach the claim limitations, feature as taught by YIN; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Regarding claim 31, Chhaya in view of Min doesn’t explicitly teach the claim limitation.
However, Yu teaches wherein the descriptor further includes a second natural language description of objects within the pivot image [fig. 1, 3 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min to teach the claim limitations, feature as taught by Yu; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Regarding claim 32, Chhaya in view of Min and Yu doesn’t teach the claim limitation.
However, YIN teaches wherein causing the second machine learning model to generate the reconstructed video further comprises providing, as an input, a plurality of pivot images and a plurality of descriptors to the second machine learning model [fig. 2, 4 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Chhaya in view of Min and Yu to teach the claim limitations, feature as taught by YIN; because the modification enables the system to improve the quality, efficiency, and semantic controllability of video reconstruction by using a small set of intelligently chosen pivot frames plus rich natural language descriptors instead of needing the full original video.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
Prosecution Insights