Prosecution Insights
Last updated: November 04, 2026
Application No. 19/039,719

SYSTEMS AND METHODS OF USING ARTIFICIAL INTELLIGENCE TO UNDERSTAND VIDEO CONTENT

Non-Final OA §103
Filed
Jan 28, 2025
Priority
Dec 13, 2024 — CIP of 18/981,227
Examiner
DUONG, JOHNNYKHOI BAO
Art Unit
2667
Tech Center
2600 — Communications
Assignee
Cynapse Pte. Ltd.
OA Round
1 (Non-Final)
64%
Grant Probability
Moderate
1-2
OA Rounds
1y 6m
Est. Remaining
97%
With Interview

Examiner Intelligence

Grants 64% of resolved cases
64%
Career Allowance Rate
40 granted / 63 resolved
+1.5% vs TC avg
Strong +33% interview lift
Without
With
+33.3%
Interview Lift
resolved cases with interview
Typical timeline
3y 3m
Avg Prosecution
24 currently pending
Career history
94
Total Applications
across all art units

Statute-Specific Performance

§101
6.1%
-33.9% vs TC avg
§103
48.6%
+8.6% vs TC avg
§102
38.6%
-1.4% vs TC avg
§112
4.9%
-35.1% vs TC avg
Black line = Tech Center average estimate • Based on career data from 63 resolved cases

Office Action

§103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Information Disclosure Statement The information disclosure statement (IDS) submitted on 08/04/2026 was filed and is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Interpretation This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "query module is configured to" in claim 1 described in paragraph [0095] and implemented on hardware disclosed in paragraphs [0022, 0025-0034, 0057]. "query parsing module…to validate" in claim 1 described in paragraph [0100-0104] and implemented on hardware disclosed in paragraphs [0022, 0025-0034, 0057]. "guardrail module to validate" in claim 1 described in paragraph [0095] and implemented on hardware disclosed in paragraphs [0022, 0025-0034, 0057]. “scene understanding module configured to" in claim 1 described in paragraph [0091] and implemented on hardware disclosed in paragraphs [0022, 0025-0034, 0057]. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Specification The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim(s) 1-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brienza (“Multi-agent Planning using Visual Language Models”, 2024), in view of OpenAI (“GPT-4 Technical Report”, 2024). Regarding claims 1, 6, 11, Brienza teaches An alert generating system, comprising: a user interface configured to receive a user query about a scene (Brienza, pg 2, column 1, first full paragraph, reproduced below: PNG media_image1.png 368 774 media_image1.png Greyscale . “picture of the scene” and “a textual query” are being interpreted as involving a user query. These inputs are being interpreted as using a user interface to accomplish this as they do not appear out of thin air); a query module (Brienza, see nearest image above, “picture of the scene” and “a textual query” are being interpreted as involving a user query) comprising … wherein the query module is configured to receive the user query from the user interface (Brienza, see pg 2, column 1 image above, “picture of the scene” and “a textual query” are being interpreted as involving a user query. These inputs are being interpreted as using a user interface to accomplish this as they do not appear out of thin air) a scene understanding module configured to process the scene to generate a frame document that comprises information about an object in the scene; a reasoning engine configured to carry out a rule-based check (Brienza, pg 3, column 2, first full paragraph: PNG media_image2.png 300 762 media_image2.png Greyscale . “The SKM Agent identifies object classes within the image and establishes the scene’s ontology”, which is being interpreted as involving a rule-based check) and a VLLM reasoning check (Brienza, see nearest image above, “The GKM Agent grounds these objects, providing short descriptions that include their relationships with surrounding objects”. Which is being interpreted as involving VLLM reasoning check) based on the validated user query (Brienza, see pg 2 image above, “picture of the scene” and “a textual query” are being interpreted as involving a user query. These inputs are being interpreted as being validated, as the non-limiting example of GPT-4, PHOSITA would know GPT-4 has guardrails to validate user query, illustrative reference here: https://www.searchenginejournal.com/research-gpt-4-jailbreak-easily-defeats-safety-guardrails/498386/ ) and the frame document (Brienza, see nearest image above, “scene description” and “relationships between objects” are being interpreted as non-limiting examples involving “frame document”); the reasoning engine configured to use at least one of rule-based reasoning (Brienza, see nearest image above, “The SKM [Semantic-Knowledge Miner, cited on pg 3] Agent identifies object classes within the image and establishes the scene’s ontology”, which is being interpreted as involving a rule-based) and VLLM reasoning (Brienza, see nearest image above, “The GKM [Grounded-Knowledge Miner, cited on pg 3] Agent grounds these objects, providing short descriptions that include their relationships with surrounding objects”. Which is being interpreted as involving VLLM reasoning) to generate an alert (Brienza, see nearest image above, “The P Agent [Planner Agent, cited on pg 2] then generates a plan using the information from the SKM and GKM Agents”. The plan is being interpreted as a non-limiting example involving an alert) based on the validated user query () and the frame document (Brienza, see nearest image above, “scene description” and “relationships between objects” are being interpreted as non-limiting examples involving “frame document”); the reasoning engine further configured to transmit an alert packet (Brienza, see nearest image above, “The P Agent [Planner Agent, cited on pg 2] then generates a plan using the information from the SKM and GKM Agents”. The plan is being interpreted as a non-limiting example involving an alert) to a computing device (Brienza, pg 2, Figure 2, “Output plan” being sent to for Evaluation is being interpreted as a non-limiting example of “to a computing device”); and wherein the alert packet causes the computing device to generate a visual notification of the alert (Brienza, pg 2, Figure 2, “Output plan” being sent to for Evaluation is being interpreted as a non-limiting example of “to a computing device”. The output plan example is being interpreted as a non-limiting example of a visual notification, as you can see the text). However, Brienza does not appear to explicitly teach validated user query. Although, Brienza does teach GPT-4, which has text parsing and user query validation built-in, as shown in the next reference. Pertaining to the same field of endeavor, OpenAI teaches a query parsing module (OpenAI, pg 12-13, Section Model-Assisted Safety Pipeline, reproduced below: PNG media_image3.png 679 924 media_image3.png Greyscale . “Then on the set of safety-relevant training prompts, which request harmful content such as illicit advice, we can reward GPT-4 for refusing these requests”. Which shows the text is parsed to look for harmful content. For a visual example of text parsing, see Figure 1 at the following illustrative reference: https://aclanthology.org/2024.acl-long.384/.) and to use the query parsing module to validate the user query to generate a validated user query (OpenAI, see nearest image above, “we can reward GPT-4 for not refusing requests on a subset of prompts guaranteed to be safe and answerable”. Which is being interpreted as involving validating the user query); Brienza and OpenAI are considered to be analogous art because they are directed to large language models. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for multi-agent vision language model scene understanding (as taught by Brienza) to include validated user queries (as taught by OpenAI) because the combination provides an improvement for model output safety (OpenAI, pg 12-13, Section Model-Assisted Safety Pipeline). Further, Brienza cited GPT-4, and the OpenAI technical report is on GPT-4, the validate user query is inherent to GPT-4. Regarding claims 7, Brienza teaches The alert generating system of claim 6, wherein the query module further comprises a guardrail module (OpenAI, see nearest image below, “Model-Assisted Safety Pipeline” is being interpreted as involving a guardrail module), and wherein the guardrail module is configured to filter prohibited user queries (OpenAI, pg 12-13, Section Model-Assisted Safety Pipeline, reproduced below: PNG media_image3.png 679 924 media_image3.png Greyscale . “Then on the set of safety-relevant training prompts, which request harmful content such as illicit advice, we can reward GPT-4 for refusing these requests”. “Refusing these requests” are being interpreted as involving “filter prohibited user queries”). Regarding claim 8, Brienza teaches The alert generating system of claim 7, wherein the prohibited user queries are filtered based on a prohibited word or a prohibited context (OpenAI, pg 12-13, Section Model-Assisted Safety Pipeline, reproduced below: PNG media_image3.png 679 924 media_image3.png Greyscale . “Then on the set of safety-relevant training prompts, which request harmful content such as illicit advice, we can reward GPT-4 for refusing these requests”. “Refusing these requests” are being interpreted as involving “filtered based on a prohibited word or a prohibited context”. Look at Table 6 and Table 7 for non-limiting examples of prohibited context). Regarding claim 9, Brienza teaches The alert generating system of claim 6, wherein the query parsing module is configured to determine a set of models that are necessary (Brienza, pg 3, column 2, lines 1-2: “GPT-4V is used for agents that process images, while GPT-4 is used for the planning agent [1].” GPT-4V and GPT-4 are being interpreted as non-limiting examples “a set of models” that are determined as necessary) to process the user query (Brienza, pg 2, column 1, first full paragraph: “All we need is a picture of the scene that captures the most relevant aspects of the environment, and a textual query on the goal we want to achieve.”). Regarding claim 10, Brienza teaches The alert generating system of claim 6, wherein the query parsing module is further configured to select a set of models (Brienza, see nearest image below, “SKM Agent” and “GKM Agent” are being interpreted as non-limiting examples of “a set of models” that were selected) for the scene understanding module to use to generate the frame document (Brienza, pg 3, column 2, first full paragraph: PNG media_image2.png 300 762 media_image2.png Greyscale . “scene description” and “relationships between objects” are being interpreted as non-limiting examples involving “frame document”). Claims 2 is rejected using the same rationale and motivation as applied to claim 7 discussed above. Claims 3 and 12 is rejected using the same rationale and motivation as applied to claim 7 discussed above. Claims 4 and 13 is rejected using the same rationale and motivation as applied to claim 8 discussed above. Claim 5 and 14 is rejected using the same rationale and motivation as applied to claim 10 discussed above. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Wang et al ("Wonderful Team: Zero-Shot Physical Task Planning with Visual LLMs”, 2024) disclose multi-agent Visual LLMs that output an alert (plan). Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET). Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /J.B.D./Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Read full office action

Prosecution Timeline

Jan 28, 2025
Application Filed
Sep 30, 2026
Non-Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12762728
MULTIMODAL 3D OBJECT DETECTION USING TEMPORAL AND STRUCTURE CONSISTENCY IN VOXEL FEATURE SPACE
2y 10m to grant Granted Oct 13, 2026
Patent 12731360
METHOD FOR RECOGNIZING AND DIAGNOSING TRANSFORMER EQUIPMENT BASED ON IMAGE FUSION AND TARGET RECOGNITION
2y 5m to grant Granted Sep 08, 2026
Patent 12718443
MEDICAL IMAGE PROCESSING DEVICE, MEDICAL IMAGE PROCESSING METHOD, AND NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM
3y 9m to grant Granted Aug 25, 2026
Patent 12705893
ARTIFICIALLY INTELLIGENT SPORTS COMPANION DEVICE
2y 10m to grant Granted Aug 11, 2026
Patent 12700216
SALIENCY-GUIDED MIXUP WITH OPTIMAL RE-ARRANGEMENTS FOR EFFICIENT DATA AUGMENTATION
3y 2m to grant Granted Aug 04, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
64%
Grant Probability
97%
With Interview (+33.3%)
3y 3m (~1y 6m remaining)
Median Time to Grant
Low
PTA Risk
Based on 63 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month