Prosecution Insights
Last updated: April 19, 2026
Application No. 17/517,329

GROUNDED MULTIMODAL AGENT INTERACTIONS

Final Rejection §101§102§103
Filed
Nov 02, 2021
Examiner
GRANT, MICHAEL CHRISTOPHER
Art Unit
3715
Tech Center
3700 — Mechanical Engineering & Manufacturing
Assignee
Microsoft Technology Licensing, LLC
OA Round
6 (Final)
21%
Grant Probability
At Risk
7-8
OA Rounds
3y 8m
To Grant
28%
With Interview

Examiner Intelligence

Grants only 21% of cases
21%
Career Allow Rate
161 granted / 751 resolved
-48.6% vs TC avg
Moderate +7% lift
Without
With
+6.6%
Interview Lift
resolved cases with interview
Typical timeline
3y 8m
Avg Prosecution
74 currently pending
Career history
825
Total Applications
across all art units

Statute-Specific Performance

§101
30.3%
-9.7% vs TC avg
§103
33.2%
-6.8% vs TC avg
§102
12.1%
-27.9% vs TC avg
§112
19.6%
-20.4% vs TC avg
Black line = Tech Center average estimate • Based on career data from 751 resolved cases

Office Action

§101 §102 §103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Applicant’s amendments dated 2/3/26 are hereby entered. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-2, 4-8, 15-18, and 21-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claims 1-2, 4-8, 15-18, and 21-22 are directed to an abstract idea without significantly more. The claims recite a mental process that can be performed by human being and/or recite a method of organizing human activity and/or the rules of a game and/or training/employing a machine learning model in a particular technological environment. In regard to Claims 1 and 15, the following limitations can be performed as a mental process by a human being in terms of claiming collecting data, analyzing that data, and providing outputs based on that analysis which has been held by the CAFC to be an abstract idea in decisions such as, e.g., Electric Power Group, University of Florida Research Foundation, and Yousician v Ubisoft (non-precedential); and/or recite a method of organizing human activity in terms of claiming the teaching/training/evaluation of a human subject’s which has been identified by MPEP 2106.04(a)(2)(II) as being a method of organizing human activity; and/or claim mathematical concepts as outlined at MPEP 2106.04(a)(2)(I), in terms of the Applicant claiming; and/or claim the rules of a game which has been held by the CAFC to be patent ineligible in decisions such as, e.g., Savvy Dog Systems (non-precedential); and/or training/employing a machine learning model in a particular technological environment, which has been held by the CAFC to be abstract in, e.g., Recentive Analytics, in terms of claiming: [a] method for controlling a […] game […] using model output of a multimodal […] model, the method comprising: receiving […] a user input having a first content type; generating a prompt for priming the multimodal […] model based on the user input, wherein: the multimodal generative […] model is configured to receive a prompt and generate a response thereon: the prompt corresponds to a plurality of behaviors each associated with a different scenario of the […] game […]; and the prompt includes natural language and a programmatic definition of at least one of the behaviors; determining, based on the received user input and the prompt associated with the […] game […], a model output associated with the multimodal […] model, wherein the multimodal […]model is associated with the first content type and a second content type, wherein the model output comprises generated natural language model output and generated [script] output; executing one or more instructions of the generated [script] output to control functionality of the […] game […]; and displaying the natural language model output on a display in association with a display element. In regard to Claims 1 and 15, training/employing a machine learning model in a particular technological environment has been held by the CAFC to be abstract in, e.g., Recentive Analytics. In regard to the dependent claims, they also claim an abstract idea to the extent that they merely claim further limitations that likewise could be performed as a mental process by a human being and/or recite a method of organizing human activity and/or claim the rules of a game and/or claim training/employing a machine learning model. Furthermore, this judicial exception is not integrated into a practical application because to the extent that additional elements are claimed either alone or in combination such as, e.g., a processor, a display, memory storing instructions, computer instructions embodying Applicant’s claimed abstract idea(s), a video game application, and/or training employing a multimodal generative machine learning model, these are merely claimed to add insignificant extra-solution activity to the judicial exception (e.g., data gathering), to embody the abstract idea on a general purpose computer, and/or do no more than generally link the use of a judicial exception to a particular technological environment or field of use. In this regard, see MPEP 2106.04(d)(I) in regard to “courts have also identified limitations that did not integrate a judicial exception into a practical application…” Furthermore, the claims do not include additional elements that taken individually, and also taken as an ordered combination, are sufficient to amount to significantly more than the judicial exception because to the extent that, e.g., a processor, a display, memory storing instructions, computer instructions embodying Applicant’s claimed abstract idea(s), a video game application, and/or training employing a multimodal generative machine learning model, these are generic, well-known, and conventional elements and are claimed for the generic, well-known, and conventional functions of collecting and processing data and/or providing an analysis/outputs based on that processing. To the extent that an apparatus is claimed as an additional element said apparatus fails to qualify as a “particular machine” to the extent that it is claimed generally, merely implements the steps of Applicant’s claimed method, and is claimed merely for purposes of extra-solution activity or field of use. See MPEP 2106.05(b). As evidence that these additional elements are generic, well-known, and conventional, Applicant’s specification discloses the support for these elements in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. § 112(a). See, e.g., F1 in Applicant’s PGPUB and text regarding same; and, e.g., p61 in regard to training/employing a generative machine learning model. Claim Rejections - 35 USC § 102 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claims 1-2, 4-8, 15-16, 18, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by PGPUB US 20190197402 A1 by Kovacs et al (“Kovacs”). In regard to Claim 1, Kovacs teaches a system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising: (see, e.g., p33); receiving, at a video game application, a user input comprising natural language; (see, e.g., F19, 1905; and F10, 1051 and p106); generating a prompt for priming the multimodal machine learning model based on the user input, wherein: the multimodal generative machine learning model is configured to receive a prompt and generate a response thereon: the prompt corresponds to a plurality of behaviors each associated with a different scenario of the video game application; and the prompt includes natural language and a programmatic definition of at least one of the behaviors; (see, e.g., F19, 1905, 1909 and 1911 regarding the prompt to the model being built by receiving natural language user input and then employing that user input to update the NPC’s belief system, the belief system which is then used as the prompt to the AI model (“the prompt includes natural language and a programmatic definition of at least one of the behaviors”); as well as, e.g., p104 in regard to the prompt being used to generate any of a plurality of behaviors based on the game situation (“the prompt corresponds to a plurality of behaviors each associated with a different scenario of the video game application”)); determining…output; and (see, e.g., F10, 1017 and p169 in regard to “multimodal generative machine learning model”; otherwise see, e.g., F10, 1021-1023 and F19, 1911 and p35 in regard to “determining…a model output”; see, e.g., p104 in regard to “wherein the model output…programmatic model output); executing…application; and (see, e.g., F10, 1027; and F19, 1913 and p104); displaying the natural language model output…element (see, e.g., p158); In regard to Claim 2, Kovacs teaches these limitations. See, e.g., F19, 1909. In regard to Claim 4, Kovacs teaches these limitations. See, e.g., p165 in regard to storing the natural language input; and, e.g., p103 in regard to storing a solution plan. In regard to Claims 5-6, Kovacs teaches these limitations. See, e.g., F10, 1017 and p169 regarding same. In regard to Claims 7-8, Kovacs teaches these limitations. See, e.g., p104. In regard to Claim 15, see rejection of Claim 1. In regard to Claim 16, Kovacs teaches these limitations. See, e.g., p165 in regard to storing the natural language input in the memory of the agent; and, e.g., p103 in regard to storing a solution plan. See, e.g., F19, 1909 and p255 and p103 in regard to using the memory/belief-state and natural language input in order to drive the next output of the model. In regard to Claim 16, Kovacs teaches these limitations. See, e.g., p165 in regard to storing the natural language input in the memory of the agent; and, e.g., p103 in regard to storing a solution plan. See, e.g., F19, 1909 and p255 and p103 in regard to using the memory/belief-state and natural language input in order to drive the next output of the model. In regard to Claim 18, Kovacs teaches these limitations. See, e.g., p164 in regard to the belief being updated/identified based on both the natural language input and the in-game environment. In regard to Claim 21, Kovacs teaches these limitations. See, e.g., p35-37, 217, 227, 234, 246, 248, 256, and 262. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Kovacs, in view of admitted prior art. In regard to Claim 17, while Kovacs teaches employing a machine learning model whereby the past natural language inputs and solution plans are used to generate new model outputs it may not teach whereby when the same input is received at a second future time the model necessarily provides a different output, however, the Examiner takes OFFICIAL NOTICE that employing a machine learning model that changes outputs in regard to the same inputs later in time was old and well-known at the time of Applicant’s invention. Such functionality allows for the machine learning model to effectively “learn” from past experience and more closely mimic human behavior. As such it would have been obvious to one of ordinary skill in the art at the time of invention to implement the claimed functionality within the invention of the cited prior art so as to allow for the machine learning model to effectively “learn” from past experience and more closely mimic human behavior. Furthermore, the Applicant failed to adequately traverse the Examiner’s taking of official notice in the prior Office action and, therefore, these claimed limitations are now admitted prior art. See MPEP 2144.03. Response to Arguments Applicant’s arguments in its Remarks in regard to the rejections made under 35 USC 102 are largely addressed by the updated statement of rejection made supra, which was necessitated by Applicant’s amendments. Applicant further argues in this regard on page 10 of its Remarks: PNG media_image1.png 320 662 media_image1.png Greyscale Applicant’s argument is unpersuasive because Kovacs at, e.g., p104 discloses the AI model (“multimodal generative machine learning model”) generating both natural language output as well as in-game actions (“source code model output”): PNG media_image2.png 224 306 media_image2.png Greyscale Applicant argues on page 12 of its Remarks in regard to the rejections made under 35 USC 101: PNG media_image3.png 466 686 media_image3.png Greyscale Applicant’s arguments are not persuasive. Applicant claims using the output of its claimed machine learning model to then generate some visual display: PNG media_image4.png 142 638 media_image4.png Greyscale This is indistinguishable from Recentive in terms of the output of the machine learning model was, likewise, in that instance used to update the visual output of the event schedule: PNG media_image5.png 162 514 media_image5.png Greyscale Id., slip. op., page 4. What is more, the fact that Applicant claims steps in its abstract idea in addition to employing the output of its machine learning model (“recites the steps of the workflow by which the one or more instructions are generated”) makes those steps no less abstract. Applicant is basically claiming the rules of a game in which the way in which the game is played is partially determined based on a model/script used to determine, e.g., actions taken and words spoken by NPC’s in the game. This is no different than the actions that might be taken by, e.g., a human dungeon master in a game of Dungeons and Dragons by employing a script/algorithm to determine the actions and/or words spoken by NPC’s in a particular game campaign. In other words, Applicant’s claimed invention makes no improvement to the functioning of its claimed machine learning model but, instead, merely employs the outputs of that model to generate some visual display, which is the same thing held to be patent ineligible in Recentive. Applicant further argues that is claimed invention is analogous to that of the Office’s 101 Example 47, claim 3. It is unclear, however, how to apply the 101 Examples provided by the Office given that the Mayo test is a legal test and the Office cannot make law; because the Examples themselves do not cite to legal authority; as well as because the Examples are not provided with specifications that would allow the BRI of the limitations in question to be determined. Be that as it may, Applicant does not claim an improvement to computer networking technology. And to the extent that Applicant claims a set of rules for providing a visual display for a video game. And the CAFC has held in, e.g., Savvy Dog Systems (non-predential) and Yousician (non-precedential) that claims directed to such displays were not patent eligible. Applicant further argues that because it claims its abstract idea, in part, embodied as computer code (“source code”) it has claimed patent eligible subject matter. Merely embodying an otherwise abstract idea as computer code, however, does not claim “significantly more”. See, e.g., Alice. Conclusion THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Mike Grant whose telephone number is 571-270-1545. The Examiner can normally be reached on Monday through Friday between 8:00 a.m. and 5:00 p.m., except on the first Friday of each bi-week. If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's Supervisory Primary Examiner, Peter Vasat can be reached at 571-270-7625. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MICHAEL C GRANT/Primary Examiner, Art Unit 3715
Read full office action

Prosecution Timeline

Nov 02, 2021
Application Filed
Oct 03, 2023
Non-Final Rejection — §101, §102, §103
Jan 05, 2024
Interview Requested
Jan 10, 2024
Response Filed
Jan 28, 2024
Final Rejection — §101, §102, §103
May 01, 2024
Request for Continued Examination
May 03, 2024
Response after Non-Final Action
May 05, 2024
Non-Final Rejection — §101, §102, §103
Oct 08, 2024
Response Filed
Oct 20, 2024
Final Rejection — §101, §102, §103
Feb 24, 2025
Response after Non-Final Action
Feb 24, 2025
Notice of Allowance
Apr 02, 2025
Response after Non-Final Action
Jul 10, 2025
Request for Continued Examination
Jul 14, 2025
Response after Non-Final Action
Oct 01, 2025
Non-Final Rejection — §101, §102, §103
Jan 28, 2026
Examiner Interview Summary
Jan 28, 2026
Applicant Interview (Telephonic)
Feb 03, 2026
Response Filed
Feb 16, 2026
Final Rejection — §101, §102, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12485332
PROJECTILE RAMP-LAUNCHING GAME AND METHOD OF PLAY
2y 5m to grant Granted Dec 02, 2025
Patent 12478863
SENSING DEVICE, BALL SHAFT FOR SMART MAGIC CUBE, AND SMART MAGIC CUBE
2y 5m to grant Granted Nov 25, 2025
Patent 12460901
HAND-OPERATED SELF DEFENSE DEVICE
2y 5m to grant Granted Nov 04, 2025
Patent 12434128
SYSTEM AND METHODS FOR GAME PLAY
2y 5m to grant Granted Oct 07, 2025
Patent 12345501
EXPANDABLE BATON
2y 5m to grant Granted Jul 01, 2025
Study what changed to get past this examiner. Based on 5 most recent grants.

AI Strategy Recommendation

Get an AI-powered prosecution strategy using examiner precedents, rejection analysis, and claim mapping.
Powered by AI — typically takes 5-10 seconds

Prosecution Projections

7-8
Expected OA Rounds
21%
Grant Probability
28%
With Interview (+6.6%)
3y 8m
Median Time to Grant
High
PTA Risk
Based on 751 resolved cases by this examiner. Grant probability derived from career allow rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month