DETAILED ACTION
Acknowledgements
This office action is in response to the claims filed 07/30/2026.
Claims 1-7 are canceled.
Claims 8, 11, 13, 17 and 19 are amended.
Claims 8-20 are pending.
Claims 8-20 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 07/30/2026 have been fully considered but they are not persuasive.
101
The claim does currently recite an additional element but the additional element does not integrate the judicial exception into a practical application.
The additional element is “training, via the processor, the Al model with the reward model….” According to the disclosure (¶ 3, 16, 18, 34, 68-72, 84), “At block 445, the reward model may be used to train (e.g., fine-tune) one or more AI models. The one or more AI models (e.g., retrained or fine-tuned AI models) can be used to generate outputs 410. … The reward model is provided to the one or more AI models at block 615. At block 620, the one or more AI models are trained based on or using the reward model…. The workflow 400 uses one or more AI models 405 to generate outputs 410. The one or more AI models may be pre-trained AI models. In one embodiment, the outputs 410 are generated by one or more LLMs based on a prompt. A user interface 415 is provided to a user device, where the user interface displays the prompt and the outputs 410 as output options for which a user provides feedback. … Reinforcement learning from human feedback (RLHF) currently works by asking human labelers to select the single best completion to a given document, or the single best response to a model prompt to produce a product, such as artwork or freeform text… Reinforcement learning from human feedback is a means of applying additional policy on top of the intrinsic loss function used when training a machine learning model from a curated dataset without human feedback… Providing feedback for a system's current behavior using human feedback, also known as “Reinforcement Learning From Human Feedback” (RLHF) is an increasingly popular means of creating a reward structure to help shape desired behaviours for the AI models to learn. However, this approach is expensive in terms of both financial costs and time-often requiring hundreds or even thousands of person hours… The reward model provides the reinforcement learning aspect of RLHF. ” First, the disclosure does not provide what steps are included in “training…” the AI model, just that an AI model is trained by the data collected and formatted to “train” the AI model to provide “outputs”. Secondly, the AI models appear to be acting as search engines, as they provide “outputs” based on prompts. This additional element does not integrate the judicial exception into a practical application, as the every other element of the claim are directed towards the abstract idea of commercial behavior of receiving and using user feedback and reviews, the disclosure regurgitates the conclusive language of “training” an AI model without context, algorithms, or explanation of how this occurs and providing an output for users to give feedback about is directed towards the abstract idea. The rejection is maintained.
103
The combination of Aggarwal and Heine teach the amended claims.
Aggarwal discloses generating, via the processor, a training dataset comprising a first output option of the plurality of output options and a first allocation of credits associated with the first output option by utilizing the first allocation of credits as a confidence-weighted label to weight the first output option: (¶ 45, 59-64, 74-80, 99, 110-125);
Aggarwal- The search module 130 also fetches and clusters assets having tags similar to most relevant assets retrieved, e.g., to formulate related categorical asset options. Relevance scores of retrieved assets are calculated on the basis of the degree of match between asset metadata and the search query. These scores are dispatched with fetched assets and categorical options to the NLP engine 126…a stochastic environment is developed, where the modeled virtual human user responds to agent actions of the agent 229. The virtual human user is modeled using some initial search query sessions data (e.g., which may be performed by real users 101), the corresponding user actions and other interactions with the agent 229. This information is used to generate a virtual user, which simulates human behavior while searching and conversing with the agent 229 during search episodes. Individual records in the query log is mapped to one of the user actions as depicted in Table 5 below… the agent 229 is trained to learn the optimal action policy, which requires actual conversational search data with the user 101. When such large set of actual conversational search data is available, the agent 229 may be trained using such actual conversational search data. ( ¶ 45, 75, 76)
generating, via the processor, a reward model that is trained on the training dataset; and training, via the processor, the Al model with the reward model ( ¶ 22, 49, 65-80, 141-170);
Aggarwal- the AI module 128 (e.g., the agent 229) is trained using A3C algorithm, which uses RL. For example, RL is used to select an action of the agent 229, in response to input received from the user 101…Thus, the agent prompts the user to complete one or more auxiliary tasks, during interaction or conversation of the agent with the user. Rewards awarded to the agent, based on the user performing one or more auxiliary tasks, is generally referred to herein as an auxiliary reward... wherein the artificial intelligence module is to: train the RL model using rewards, wherein rewards awarded during a search episode include a first reward for successful completion of the search episode, a second reward, based on user response to an action selected by the RL model, and a third reward for completion of an auxiliary task identified by the RL model. ( ¶ 22, 49, 170);
Heine teaches and an amount of credits to be apportioned to the plurality of output options; and wherein each allocation of credits represents an assessment of accuracy of each corresponding output option (Figure 5; ¶ 8, 22-50, 52-60, 64-66);
Heine- the queries received from the requesting users include reward values assigned by the requesting user, wherein the reward values are offered to suggesting users in exchange for description suggestions…Ultimately, with enough entries (e.g., requests and suggestions), the crowd assisted query system may train an image search AI that would then be able to identify items based on a visual search alone… At operation 325, the categorization module 210 associates the description suggestion provided by the user 109 with the data object(s) of the query request within a database (e.g., the database 120). For example, the database 120 may include an AI database useable to train the AI module 220 to accurately identify items based on text or audio inputs (e.g., natural language speech), or based on image data alone… By ranking the description suggestions based on a reliability score calculated by the ranking module 225, the crowd assisted query system 110 may surface and more readily identify accurate description suggestions (¶ 22, 28, 44, 49);
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 8-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Subject Matter Eligibility Standard
When considering subject matter eligibility under 35 U.S.C. § 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (101 Analysis: Step 1). Even if the claim does fall within one of the statutory categories, it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea) (101 Analysis: Step 2a(Prong 1), and if so, Identify whether there are any additional elements recited in the claim beyond the judicial exception(s), and evaluate those additional elements to determine whether they integrate the exception into a practical application of the exception. (101 Analysis: Step 2a (Prong 2). If additional elements does not integrate the exception into a practical application of the exception, claim still requires an evaluation of whether the claim recites additional elements that amount to an inventive concept (aka “significantly more”) than the recited judicial exception. If the claim as a whole amounts to significantly more than the exception itself (there is an inventive concept in the claim), the claim is eligible. If the claim as a whole does not amount to significantly more (there is no inventive concept in the claim), the claim is ineligible. (101 Analysis: Step 2b).
The 2019 PEG explains that the abstract idea exception includes the following groupings of subject matter: a) Mathematical concepts b) Certain methods of organizing human activity and c) Mental processes
Analysis
In the instant case, claim 8 is directed to a method, and claim 17 is directed to an article of manufacture.
Step 2a.1– Identifying an Abstract Idea
The claims recite the steps of “receiving… options… providing, to a user device, a user interface that displays … receiving, … an allocation of credits …generating… a training dataset…generating, … a reward model …and training, … the Al model ….” The recited limitations fall within the certain methods of organizing human activity grouping of abstract ideas, specifically, commercial behavior of receiving and using user feedback and reviews. Accordingly, the claims recites an abstract idea.
See MPEP 2106.
Step 2a.2 – Identifying a Practical Application
The claim does currently recite an additional element but the additional element does not integrate the judicial exception into a practical application.
The additional element is “training, via the processor, the Al model with the reward model” According to the disclosure (¶ 3, 16, 18, 34, 68-72, 84), “At block 445, the reward model may be used to train (e.g., fine-tune) one or more AI models. The one or more AI models (e.g., retrained or fine-tuned AI models) can be used to generate outputs 410. … The reward model is provided to the one or more AI models at block 615. At block 620, the one or more AI models are trained based on or using the reward model…. The workflow 400 uses one or more AI models 405 to generate outputs 410. The one or more AI models may be pre-trained AI models. In one embodiment, the outputs 410 are generated by one or more LLMs based on a prompt. A user interface 415 is provided to a user device, where the user interface displays the prompt and the outputs 410 as output options for which a user provides feedback. … Reinforcement learning from human feedback (RLHF) currently works by asking human labelers to select the single best completion to a given document, or the single best response to a model prompt to produce a product, such as artwork or freeform text… Reinforcement learning from human feedback is a means of applying additional policy on top of the intrinsic loss function used when training a machine learning model from a curated dataset without human feedback… Providing feedback for a system's current behavior using human feedback, also known as “Reinforcement Learning From Human Feedback” (RLHF) is an increasingly popular means of creating a reward structure to help shape desired behaviours for the AI models to learn. However, this approach is expensive in terms of both financial costs and time-often requiring hundreds or even thousands of person hours… The reward model provides the reinforcement learning aspect of RLHF. ” First, the disclosure does not provide what steps are included in “training…” the AI model, just that an AI model is trained by the data collected and formatted to “train” the AI model to provide “outputs”. Secondly, the AI models appear to be acting as search engines, as they provide “outputs” based on prompts. This additional element does not integrate the judicial exception into a practical application, as the every other element of the claim are directed towards the abstract idea of commercial behavior of receiving and using user feedback and reviews, the disclosure regurgitates the conclusive language of “training” an AI model without context, algorithms, or explanation of how this occurs and providing an output for users to give feedback about is directed towards the abstract idea.
Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Mere instructions to apply the exception using generic computer components and limitations to a particular field of use or technological environment do not amount to practical applications. The claim in directed to an abstract idea.
Step 2b
The claim limitations recite “receiving… options… providing, to a user device, a user interface that displays … receiving, … an allocation of credits …generating… a training dataset…generating, … a reward model …and training, … the Al model ….” are not additional elements and they amount to no more than mere instructions to apply the exception using a generic computer component. For the same reason these elements are not sufficient to provide an inventive concept. This is also determined to be well-understood, routine and conventional activity in the field. The Symantec, TLI, and OIP Techs, court decision cited in MPEP 2106.05(d)(II) indicates that mere receipt or transmission of data over a network is a well-understood, routine and conventional function when it is claimed in a merely generic manner, as it is here. Therefore, when considering the additional elements alone, and in combination, there is no inventive concept in the claim and thus the claim is not eligible.
Viewed as a whole, instructions/method claims recite the concept of a commercial behavior as performed by a generic computer. The claims do not currently recite any additional elements or combination of additional elements that amount to significantly more than the judicial exception. The elements used to perform the claimed judicial exception amount to no more than mere instructions to implement the abstract idea in a network, and/or merely uses a network as a tool to perform an abstract idea and/or generally linking the use of the judicial exception to a particular environment.
Dependent Claims 9, 10, 12, 14, 16, and 18 provide descriptive language surrounding the abstract idea. As such, these elements do not provide the significantly more to the underlying abstract idea necessary to render the invention patentable.
Dependent claims 11, 13, 15, 19 and 20 discuss functions in more descriptive detail of the steps geared toward the abstract idea. As such, these elements do not provide the significantly more to the underlying abstract idea necessary to render the invention patentable.
The claims do not, for example, purport to improve the functioning of the computer itself. Nor do they effect an improvement in any other technology or technical field. Therefore, based on case law precedent, the claims are claiming subject matter similar to concepts already identified by the courts as dealing with abstract ideas. See Alice Corp. Pty. Ltd., 573 U.S. 208 (citing Bilski v. Kappos, 561, U.S. 593, 611 (2010)).
The claims at issue amount to nothing significantly more than an instruction to apply the abstract idea using some unspecified, generic computer. See Alice Corp. Pty. Ltd., 573 U.S. 208. Mere instructions to apply the exception using a generic computer component and limitations to a particular field of use or technological environment cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. The use of a computer or processor to merely automate and/or implement the abstract idea cannot provide significantly more than the abstract idea itself (MPEP 2106.05(I)(A)(f) & (h)). Therefore, the claim is not patent eligible.
Conclusion
The claim as a whole, does not amount to significantly more than the abstract idea itself. This is because the claim does not affect an improvement to another technology or technical filed; the claim does not amount to an improvement to the functioning of a computer system itself; and the claim does not move beyond a general link of the use of an abstract idea to a particular technological environment.
Accordingly, the Examiner concludes that there are no meaningful limitations in the claim that transform the judicial exception into a patent eligible application such that the claim amounts to significantly more than the judicial exception itself.
Dependent claims do not resolve the deficiency of independent claims and accordingly stand rejected under 35 USC 101 based on the same rationale.
Dependent claims 9-16 and 18-20 are also rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 8-20 are rejected under 35 U.S.C. 103 as being unpatentable over Aggarwal et al. (US 20200341976) (“Aggarwal”), and further in view of Heine et al. (US 20180129724) (“Heine”).
Regarding claims 8 and 17, Aggarwal discloses receiving, via a processor, from an artificial intelligence (AI) model, a plurality of output options (¶ 16, 44-50, 107-110);
Aggarwal- In an example, the AI module 128 is trained using Asynchronous Advantage Actor-Critic (A3C) algorithm, which uses RL. The AI module 128 transmits information associated with the agent action to the NLP engine 126, which translates it to a format that is presentable in the UI 300 b. The NLP engine 126 then transmits a formatted response to the module 156, where the formatted response may be based on the selected action. In some cases, the formatted response may also include the requested search results. ( ¶ 46)
providing, via the processor, to a user device, a user interface that displays at least the plurality of output options associated with a prompt (¶ 16, 65-95, 110-121);
Aggarwal- This interaction takes place over multiple interaction cycles with the user, where in a given cycle the RL-based agent prompts, and the user responds (or the user provides an input, and the RL-based agent responds). (¶ 16);
receiving, via the processor, from the user device via the user interface, user feedback associated with the plurality of output options, the user feedback comprising an allocation of credits to each output option in the plurality of output options, wherein each allocation of credits ranges between a first amount of credits and a second amount of credits and the allocations of credits, ( ¶ 65-82, 113-121, 141-170);
Aggarwal- User actions may be categorized into two or more feedback categories, such as good, average, bad, etc. (or may be scaled in a scale or 1 to 5, with 5 being best or as intended by the agent 229, and 1 being worst). For example, if the agent 229 prompts the user 101 to refine a search query and the user does follow the prompt, then the agent 229 receives a relatively high extrinsic reward rextrinsic, e.g., because the user 101 played along with the agent 229. On the other hand, if the user 101 refuses to refine the search query, a relatively low (or zero, or even negative) extrinsic reward rextrinsic is awarded to the agent 229. ( ¶ 71)
generating, via the processor, a training dataset comprising a first output option of the plurality of output options and a first allocation of credits associated with the first output option by utilizing the first allocation of credits as a confidence-weighted label to weight the first output option: ( ¶ 45, 59-64, 74-80, 99, 110-125);
Aggarwal- The search module 130 also fetches and clusters assets having tags similar to most relevant assets retrieved, e.g., to formulate related categorical asset options. Relevance scores of retrieved assets are calculated on the basis of the degree of match between asset metadata and the search query. These scores are dispatched with fetched assets and categorical options to the NLP engine 126…a stochastic environment is developed, where the modeled virtual human user responds to agent actions of the agent 229. The virtual human user is modeled using some initial search query sessions data (e.g., which may be performed by real users 101), the corresponding user actions and other interactions with the agent 229. This information is used to generate a virtual user, which simulates human behavior while searching and conversing with the agent 229 during search episodes. Individual records in the query log is mapped to one of the user actions as depicted in Table 5 below… the agent 229 is trained to learn the optimal action policy, which requires actual conversational search data with the user 101. When such large set of actual conversational search data is available, the agent 229 may be trained using such actual conversational search data. ( ¶ 45, 75, 76)
generating, via the processor, a reward model that is trained on the training dataset; and training, via the processor, the Al model with the reward model ( ¶ 22, 49, 65-80, 141-170);
Aggarwal- the AI module 128 (e.g., the agent 229) is trained using A3C algorithm, which uses RL. For example, RL is used to select an action of the agent 229, in response to input received from the user 101…Thus, the agent prompts the user to complete one or more auxiliary tasks, during interaction or conversation of the agent with the user. Rewards awarded to the agent, based on the user performing one or more auxiliary tasks, is generally referred to herein as an auxiliary reward... wherein the artificial intelligence module is to: train the RL model using rewards, wherein rewards awarded during a search episode include a first reward for successful completion of the search episode, a second reward, based on user response to an action selected by the RL model, and a third reward for completion of an auxiliary task identified by the RL model. ( ¶ 22, 49, 170)
Aggarwal does not disclose and an amount of credits to be apportioned to the plurality of output options; and wherein each allocation of credits represents an assessment of accuracy of each corresponding output option;
Heine teaches and an amount of credits to be apportioned to the plurality of output options; and wherein each allocation of credits represents an assessment of accuracy of each corresponding output option (Figure 5; ¶ 8, 22-50, 52-60, 64-66);
Heine- the queries received from the requesting users include reward values assigned by the requesting user, wherein the reward values are offered to suggesting users in exchange for description suggestions…Ultimately, with enough entries (e.g., requests and suggestions), the crowd assisted query system may train an image search AI that would then be able to identify items based on a visual search alone… At operation 325, the categorization module 210 associates the description suggestion provided by the user 109 with the data object(s) of the query request within a database (e.g., the database 120). For example, the database 120 may include an AI database useable to train the AI module 220 to accurately identify items based on text or audio inputs (e.g., natural language speech), or based on image data alone… By ranking the description suggestions based on a reliability score calculated by the ranking module 225, the crowd assisted query system 110 may surface and more readily identify accurate description suggestions (¶ 22, 28, 44, 49)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Aggarwal and Heine in order to assist and train AI in search queries (Heine; ¶ 1, 28).
Regarding claim 9, Aggarwal discloses wherein the first amount of credits is zero and the second amount of credits is a maximum amount of credits that can be allocated to a respective output option ( ¶ 65-75).
Regarding claims 10 and 18, Aggarwal discloses wherein the user interface further comprises at least one of: a credits deployed indicator indicating a total number of credits deployed; a time to completion indicator indicating an estimated time to completion; a points indicator indicating a total number of earned points; a conversion component configured to convert earned points into available credits; or an ablation indicator, the ablation indicator configured to indicate a portion of at least one output option for ablation ( ¶ 16, 65-95, 110-121).
Regarding claim 11, Aggarwal discloses awarding a user account associated with the user feedback one or more points based on a determination that the user feedback reduces uncertainty of the Al model ( ¶ 19, 73, 101, 162).
Regarding claim 12, Aggarwal discloses wherein: the method further comprises recording at least one of: a time-to-answer for the allocation of credits; or a speed of a movement of an input device; and the time-to-answer or the speed of the movement are considered in the determination that the user feedback reduces uncertainty of the Al model ( ¶ 66-68, 81-85, 95).
Regarding claim 13, Aggarwal discloses subtracting one or more points from a user account associated with the user feedback based on a determination that the user feedback does not reduce uncertainty of the Al model ( ¶ 68-71).
Regarding claim 14, Aggarwal discloses wherein the Al model comprises a large language model ( ¶ 30, 34, 45, 46, 136; claim 14).
Regarding claim 15, Aggarwal discloses receiving, via the user interface, audio input or text input as additional feedback about one or more output options ( ¶ 14, 16, 45, 46, 64, 120).
Regarding claim 16, Aggarwal discloses wherein each output option comprises one or more of: natural language; an image; or a video ( ¶ 14, 16, 45, 46, 64, 120).
Regarding claim 19, Aggarwal discloses wherein the memory stores further instructions for: awarding a user account associated with the user feedback one or more points based on the determination that the user feedback reduces the uncertainty of the Al model; and subtracting one or more points from a user account associated with the user feedback based on a determination that the user feedback does not reduce the uncertainty of the Al model ( ¶ 65-71).
Regarding claim 20, Aggarwal discloses wherein the memory stores further instructions for comparing the user feedback to confidence-weighted user feedback associated with a group of users prior to awarding the one or more points ( ¶ 90-106).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Edwards et al., (US 20240354646) teaches user feedback in training AI.
Bechtel et al. (US 8239228) teaches awarding users points for collaborating on results.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ILSE I IMMANUEL whose telephone number is (469)295-9094. The examiner can normally be reached Monday-Friday 9:00 am to 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, NEHA H PATEL can be reached on (571) 270-1492. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ILSE I IMMANUEL/Primary Examiner, Art Unit 3699
