Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim 1, see remarks page 9 filed on 05/27/2026 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Applicant’s amendment did not quiet incorporate all the limitations of the previously objected claim 10 into currently amended claim 1. The rejection is presented below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-8, 11 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over NPL1 (CoCa: Contrastive Captioners are Image-Text Foundation Models, Jiahui Yu et al., arXiv, 14 Jun 2022, Pages 1-19) hereafter NPL1 (Single reference 103 as a computing system comprising one or more computing devices would be obvious in view of “abstract discloses “By sharing the same computational graph, the two training objectives are computed efficiently with minimal overhead” and “compute two training losses considered efficiently” on pages 5 section “Pretraining efficiency”.)
1. Regarding claim 1, NPL1 discloses a computer-implemented method for performing a video understanding task with improved computational efficiency (see introduction and Fig 1), the method comprising:
accessing, by a computing system comprising one or more computing devices, a pre- trained image-text processing model (fig 1., Section 3.2, 3.3 and 4.1 “a computing system comprising one or more computing devices would be obvious in view of “abstract discloses “By sharing the same computational graph, the two training objectives are computed efficiently with minimal overhead” and “compute two training losses considered efficiently” on pages 5 section “Pretraining efficiency”), wherein the pre-trained image-text processing model comprises one or more pre-trained attentional pooling layers having a number of parameters (figs 1-2, sections 3.2 and 3.3 where λCon and λCap are loss weighting hyper-parameters), and wherein the pre-trained image-text processing model has been pre-trained on a joint contrastive and generative image captioning loss function (See abstract, Figs 1, 2 and section 1, sections 4.1 and 4.2.3 shows and discloses We propose a simple model family named Contrastive Captioners (CoCa) with a modified encoder-decoder architecture trained with both contrastive loss and captioning (generative image captioning) loss), wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been finetuned after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function (fig 1 shows fine-tuning after the pretraining and Figs 1-2, section 2, 3.3, discloses wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been finetuned after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function), wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been finetuned after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function (fig 1 shows fine-tuning after the pretraining and Figs 1-2, section 2, 3.3, discloses wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been finetuned after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function) applied to video data (Fig 3 shows an input video that comprises a plurality of image frames meeting the limitations of applied to video data, section 4.2.1 discloses fine-tunning applied to video data meeting the above claim limitations);
obtaining, by the computing system, an input video that comprises a plurality of image frames (Fig 3 shows an input video that comprises a plurality of image frames); processing, by the computing system, the input video with the pre-trained image-text processing model having the one or more pre-trained attentional pooling layers (figs 1-2 shows pre-trained model with one or more pre-trained attentional pooling layers) having the same number of parameters to generate, as an output of the pre-trained image-text processing model, a prediction for the video understanding task (figs 1-3, section 1, section 3.2, 3.3 and 4.2.3 output (i.e prediction) for video recognition task “i.e a learned CoCa model for video action recognition tasks”); and providing, by the computing system, the prediction for the video understanding task as an output (figs 1-3, section 1, section 3.2, 3.3 and 4.2.3 shows and discloses the prediction for the video understanding task as an output “i.e a learned CoCa model for video action recognition tasks”). Before the effective filing date of the invention was made, a computing system comprising one or more computing devices would be obvious and within one of ordinary skill in the art in view of the disclosure in NPL1 “a computing system comprising one or more computing devices would be obvious in view of “abstract discloses “By sharing the same computational graph, the two training objectives are computed efficiently with minimal overhead” and “compute two training losses considered efficiently” on pages 5 section “Pretraining efficiency”. The suggestion/motivation would be an efficient and minimal overhead method/system (see page 1 abstract).
2. Regarding claim 2, NPL1 discloses the computer-implemented method of claim 1, wherein: the pre-trained image-text processing model comprises a pre-trained unimodal image encoder configured to process an input image to generate one or more frame embeddings (abstract, figs 1, 2 shows and discloses the pre-trained image-text processing model comprises a pre-trained unimodal image encoder configured to process an input image to generate one or more frame embeddings); the one or more pre-trained attentional pooling layers are configured to process the one or more frame embeddings to generate one or more contrastive embeddings and one or more generative embeddings (Abstract, fig 2 and page 12 shows and discloses the one or more pre-trained attentional pooling layers are configured to process the one or more frame embeddings to generate one or more contrastive embeddings and one or more generative embeddings); and processing, by the computing system, the input video with the pre-trained image-text processing model comprises: separately processing each of the plurality of image frames with the pre- trained unimodal image encoder to generate a plurality of frame embeddings respectively for the plurality of image frames (figs 1-3 shows and discloses the input video with the pre-trained image-text processing model comprises: separately processing each of the plurality of image frames with the pre- trained unimodal image encoder to generate a plurality of frame embeddings respectively for the plurality of image frames); combining the plurality of frame embeddings to form a set of combined frame embeddings; and processing the set of combined frame embeddings with the one or more attentional layers to generate one or more generative embeddings and one or more contrastive embeddings (figs 1- 3 and pages 2-4, 6 shows and discloses combining the plurality of frame embeddings to form a set of combined frame embeddings; and processing the set of combined frame embeddings with the one or more attentional layers to generate one or more generative embeddings and one or more contrastive embeddings).
3. Regarding claim 5, NPL1 discloses the computer-implemented method of claim 1, wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been held fixed after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function (fig 2, algorithm 1, page 3 section 3.1 shows and discloses wherein the parameters of the one or more pre-trained attentional pooling layers of the pre-trained image-text processing model have been held fixed after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function).
4. Regarding claim 6, NPL1 discloses the computer-implemented method of claim 5, wherein the video understanding task comprises a zero-shot video understanding task (figs 1-3, pages 1-4, and 6, abstract, section 1, 3, 3.3 shows and discloses wherein the video understanding task comprises a zero-shot video understanding task).
5. Regarding claim 7, NPL1 discloses the computer-implemented method of claim 5, wherein the pre-trained image- text processing model has been trained only on training data comprising only still images (fig 1, 3 and section 3.3 shows the pretraining using still images meeting the claim limitations).
6. Regarding claim 8, NPL1 discloses the computer-implemented method of claim 1, wherein an entirety of parameters of the pre-trained image-text processing model have been held fixed after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function (fig 2, algorithm 1, page 3 section 3.1 shows and discloses wherein an entirety of parameters of the pre-trained image-text processing model have been held fixed after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function).
7. Regarding claim 11, NPL1 discloses the computer-implemented method of claim 1, wherein an entirety of parameters of the pre-trained image-text processing model have been further finetuned after said pre-training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function (fig 1, 3 shows fine-tuning after the pretraining and Figs 1-2, section 2, 3.3 wherein an entirety of parameters of the pre-trained image-text processing model have been further finetuned after said pre- training of the pre-trained image-text processing model on the joint contrastive and generative image captioning loss function).
8. Regarding claim 15, NPL1 discloses the computer-implemented method of claim 1, wherein the pre-trained image- text processing model comprises a decoder configured to process embeddings generated by the one or more attentional layers to generate a text output (figs 1-2 shows and discloses wherein the pre-trained image- text processing model comprises a decoder configured to process embeddings generated by the one or more attentional layers to generate a text output).
9. Regarding claim 16, NPL1 discloses the computer-implemented method of claim 1, wherein the pre-trained image- text processing model further comprises a unimodal text decoder and wherein processing the input video comprises processing a set of input text associated with the input video using the unimodal text decoder (figs 1-3 shows and discloses wherein the pre-trained image- text processing model further comprises a unimodal text decoder and wherein processing the input video comprises processing a set of input text associated with the input video using the unimodal text decoder).
10. Regarding claim 17, NPL1 discloses the computer-implemented method of claim 1, wherein the video understanding task comprises a video classification task (figs 1-3, table 5 (Zero-shot Video-text retrieval (i.e video classification task)).
11. Regarding claim 18, NPL1 discloses the computer-implemented method of claim 1, wherein the video understanding task comprises a video question answering task (section 4.2.3 discloses VQA (visual or video) question answering meeting the claim limitations).
12. Regarding claim 19, NPL1 discloses the computer-implemented method of claim 1, wherein the video understanding task comprises a video captioning task (figs1, 3 discloses and shows image/video captioning task meeting the claim limitations).
Allowable Subject Matter
Claims 3-4 and 12-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 21 is allowed. Regarding claim 21, none of the cited arts alone or in combination disclose or suggests at least the “wherein the one or more pre-trained attentional pooling layers comprise a generative pooling layer configured to generate one or more generative embeddings and a contrastive pooling layer configured to generate one or more contrastive embeddings;” along with other limitations, therefore claim 21 is allowed.
Claim 22 is allowed. Regarding independent claim 22, none of the cited arts alone or in combination disclose or suggests at least the “prior to processing the input video, appending, by the computing system, an additional encoder model to at least one of the one or more attentional pooling layers;” along with other limitations, therefore claim 22 is allowed.
Claim 23 is allowed. Regarding independent claim 23, none of the cited arts alone or in combination disclose or suggests at least the “wherein combining the plurality of frame embeddings to form the set of combined frame embeddings comprises concatenating the plurality of frame embeddings along a temporal dimension to generate a set of flattened frame embeddings or reshaping the plurality of frame embeddings into a joint space-time representation.” along with other limitations, therefore claim 23 is allowed.
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAYESH PATEL whose telephone number is (571)270-1227. The examiner can normally be reached IFW Mon-FRI.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAYESH A PATEL/Primary Examiner, Art Unit 2677
/JAYESH PATEL/
Primary Examiner
Art Unit 2677
Prosecution Insights