DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
2. The information disclosure statement (IDS) submitted on 11/12/2025 has been received, entered into the record, and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment
3. Receipt of Applicant’s Amendment filed on 11/12/2025 is acknowledged. The amendment includes the amending of claims 1-2, 7, 9, 11-12, 17-18, and 20.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims (1-10 & 19), (11-18), and (20) are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Under the 2019 PEG, when considering subject matter eligibility under 35 U.S.C. § 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (step 1). If the claim does fall within one of the statutory categories, it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea) (step 2A prong 1), and if so, it must additionally be determined whether the claim is integrated into a practical application (step 2A prong 2). If an abstract idea is present in the claim without integration into a practical application, any element or combination of elements in the claim must be sufficient to ensure that the claim amounts to significantly more than the abstract idea itself (step 2B).
In the instant case, claims (1-10 & 19), (11-18), and (20) are directed to a method, system, and non-transitory computer-readable medium respectively. Thus, each of the claims falls within one of the four statutory categories. However, the claims also fall within the judicial exception of an abstract idea.
Under Step 2A Prong 1, the test is to identify whether the claims are “directed to” a judicial exception. The examiner notes that the claimed invention is directed to an abstract idea in that the instant application is directed to mental processes, specifically updating a constraint function.
The examiner further notes that claims (1-10 & 19), (11-18), and (20) are directed to a method, system, and non-transitory computer-readable medium for updating a constraint function which is similar to themes defined above of method of mental processes such as performing the updating a constraint function, and is similar to the abstract idea identified in the 2019 PEG in grouping “c” in that the claims recite certain methods of mental processes such as performing the updating of a constraint function. The limitations, substantially comprising the body of the claim, recite a process of updating a constraint function. The examiner notes that the claimed invention updates a constraint function. Because the limitations above closely follow the steps in updating a constraint function, and the steps of the claims involve mental processes, the claim recites an abstract idea consistent with the “mental processes” grouping set forth in the 2019 PEG.
Claim 1:
A method for learning a constraint function consistent with a demonstration, comprising: obtaining: demonstration data representative of the demonstration, the demonstration data comprising a sequence of actions, each action being taken in the context of a respective state of a demonstration environment;
an initial policy operable to determine an action for an agent based on a current state of an agent environment, such that a current policy is set to the initial policy; and
an initial constraint function, such that a current constraint function is set to the initial constraint function;
performing a policy optimization procedure to adjust the current policy, thereby generating an adjusted policy;
adding the adjusted policy to a set of policies;
performing a constraint function optimization procedure to: obtain a mixture policy, based on the set of policies, that defines a second utility comprising the current constraint function applied to the mixture policy; and
adjust the current constraint function to maximize the second utility using a neural network based on the demonstration data and agent data representing one or more actions taken by the agent, such that a third utility is within a constraint threshold, the third utility being the current constraint function applied to the demonstration data; and
providing the current constraint function as the constraint function.
These limitations, as drafted, is an apparatus that, under its broadest reasonable interpretation, covers the performance of mental processes specifically updating a constraint function. Updating a constraint function has long before the modern computer was invented, and continues to be predominantly a product of human endeavor. The instant application is directed to updating a constraint function. Additionally, the claimed determining of an action for an agent can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed setting a constraint function to an initial constraint function can be performed by a human via their mind and/or pen & paper. Moreover, the claimed performing of an optimization procedure can be performed by a human via their mind and/or pen & paper. Additionally, the claimed adding of an adjusted policy to a set of policies can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed adjusting of a current constrain function can be performed by a human via their mind and/or pen & paper. Additionally, the claimed providing of the current constraint function as the constraint function can be performed by a human via their mind and/or pen & paper. Because the limitations above closely follow the steps of updating a constraint function, and the steps involved human judgments, observations and evaluations that can be practically or reasonably performed in the human mind and/or pen & paper, the claim recites an abstract idea consistent with the “mental process” grouping set forth in the 2019 PEG.
If the claims are directed toward the judicial exception of an abstract idea, it must then be determined under Step 2A Prong 2 whether the judicial exception is integrated into a practical application. The Examiner notes that considerations under Step 2A Prong 2 comprise most the consideration previously evaluated in the context of Step 2B. The Examiner submits that the considerations discussed previously determined that the claim does not recite “significantly more” at Step 2B would be evaluated the same under Step 2A Prong 1 and result in the determination that the claim does not integrate the abstract idea into a practical application.
The instant application fails to integrate the judicial exception into a practical application because the instant application merely recites words “apply it” (or an equivalent) with the judicial exception or merely includes instructions to implement an abstract idea. Specifically, the claim recites an additional element (using a neural network) that is a mere instruction to apply an exception. A recitation of the words “apply it” (or an equivalent) are mere instructions to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The instant application is directed to an apparatus instructing the reader to implement the identified apparatus of mental processes of updating a constraint function. The elements of the claim do not themselves amount to an improvement to the computer, to a technology or another technical field. Moreover, the obtaining of defined demonstration data is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application. Furthermore, the obtaining of a mixture policy is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application.
Here, the claim elements entirely comprise the abstract idea, leaving little if any aspects of the claim for further consideration under Step 2A Prong 2. In short, the claims have failed to integrate a practical application (see at least 84 Fed. Reg. (4) at 55). Under the 2019 PEG, this supports the conclusion that the claim is directed to an abstract idea, and the analysis proceeds to Step 2B.
While many considerations in Step 2A need not be reevaluated in Step 2B because the outcome will be the same. Here, on the basis of the additional elements other than the abstract idea, considered individually and in combination as discussed above, the Examiner respectfully submits that the claim 1 does not contain any additional elements that individually or as an ordered combination amount to an inventive concept and the claims are ineligible.
With respect to the dependent claims do not recite anything that is found to render the abstract idea as being transformed into a patent eligible invention. The dependent claims are merely reciting further embellishments of the abstract idea and do not claim anything that amounts to significantly more than the abstract idea itself.
With respect to the dependent claims, they have been considered and are not found to be reciting anything that amounts to being significantly more than the abstract idea. Claims 2-10 & 19 are directed to further embellishments of the central theme of the abstract idea in that the claims are directed to further embellishments of the updating a constraint function of the steps of claim 1 and do not amount to significantly more.
Specifically, claim 2 is directed towards the repetitive performing of optimization which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Furthermore, claim 3 is directed towards the adjusting of a policy which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Additionally, claim 4 is directed towards the adjusting of a policy and subsequent use of reinforcement learning which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Moreover, claim 5 is directed towards the use of a vanilla gradient descent in reinforcement learning which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Furthermore, claim 6 is directed towards the use of a vanilla gradient descent in optimization which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Additionally, claim 7 is directed towards the training of a neural network which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Moreover, claim 8 is directed towards the computation of a weighting mixture which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Furthermore, claim 9 is directed towards the computation of utilities which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Additionally, claim 10 is directed towards the operation of planning of motion which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Moreover, claim 19 is directed towards the operation of planning of motion which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Claim 11:
A system, comprising: a processing device;
a memory storing thereon machine-executable instructions that, when executed by the processing device, cause the system to learn a constraint function consistent with a demonstration by: obtaining: demonstration data representative of the demonstration, the demonstration data comprising a sequence of actions, each action being taken in the context of a respective state of a demonstration environment;
an initial policy operable to determine an action for an agent based on a current state of an agent environment, such that a current policy is set to the initial policy; and
an initial constraint function, such that a current constraint function is set to the initial constraint function;
performing a policy optimization procedure to adjust the current policy, thereby obtain an adjusted policy;
adding the adjusted policy to a set of policies;
performing a constraint function optimization procedure to: generate a mixture policy, based on the set of policies, that defines a second utility comprising the current constraint function applied to the mixture policy; and
adjust the current constraint function to maximize the second utility using a neural network based on the demonstration data and agent data representing one or more actions taken by the agent, such that a third utility is within a constraint threshold, the third utility being the current constraint function applied to the demonstration data; and
providing the current constraint function as the constraint function.
These limitations, as drafted, is an apparatus that, under its broadest reasonable interpretation, covers the performance of mental processes specifically updating a constraint function. Updating a constraint function has long before the modern computer was invented, and continues to be predominantly a product of human endeavor. The instant application is directed to updating a constraint function. Additionally, the claimed determining of an action for an agent can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed setting a constraint function to an initial constraint function can be performed by a human via their mind and/or pen & paper. Moreover, the claimed performing of an optimization procedure can be performed by a human via their mind and/or pen & paper. Additionally, the claimed adding of an adjusted policy to a set of policies can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed providing of the current constraint function as the constraint function can be performed by a human via their mind and/or pen & paper. Because the limitations above closely follow the steps of updating a constraint function, and the steps involved human judgments, observations and evaluations that can be practically or reasonably performed in the human mind and/or pen & paper, the claim recites an abstract idea consistent with the “mental process” grouping set forth in the 2019 PEG.
The mere nominal recitation of generic computing components such as a processing device and memory does not take the claim out of certain methods of mental processes grouping. Therefore, the limitation is directed to an abstract idea.
If the claims are directed toward the judicial exception of an abstract idea, it must then be determined under Step 2A Prong 2 whether the judicial exception is integrated into a practical application. The Examiner notes that considerations under Step 2A Prong 2 comprise most the consideration previously evaluated in the context of Step 2B. The Examiner submits that the considerations discussed previously determined that the claim does not recite “significantly more” at Step 2B would be evaluated the same under Step 2A Prong 1 and result in the determination that the claim does not integrate the abstract idea into a practical application.
The instant application fails to integrate the judicial exception into a practical application because the instant application merely recites words “apply it” (or an equivalent) with the judicial exception or merely includes instructions to implement an abstract idea. Specifically, the claim recites an additional element (using a neural network) that is a mere instruction to apply an exception. A recitation of the words “apply it” (or an equivalent) are mere instructions to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The instant application is directed to an apparatus instructing the reader to implement the identified apparatus of mental processes of updating a constraint function. The elements of the claim do not themselves amount to an improvement to the computer, to a technology or another technical field. Moreover, the obtaining of defined demonstration data is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application. Furthermore, the obtaining of a mixture policy is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application.
Here, the claim elements entirely comprise the abstract idea, leaving little if any aspects of the claim for further consideration under Step 2A Prong 2. In short, the claims have failed to integrate a practical application (see at least 84 Fed. Reg. (4) at 55). Under the 2019 PEG, this supports the conclusion that the claim is directed to an abstract idea, and the analysis proceeds to Step 2B.
While many considerations in Step 2A need not be reevaluated in Step 2B because the outcome will be the same. Here, on the basis of the additional elements other than the abstract idea, considered individually and in combination as discussed above, the Examiner respectfully submits that the claim 11 does not contain any additional elements that individually or as an ordered combination amount to an inventive concept and the claims are ineligible.
With respect to the dependent claims do not recite anything that is found to render the abstract idea as being transformed into a patent eligible invention. The dependent claims are merely reciting further embellishments of the abstract idea and do not claim anything that amounts to significantly more than the abstract idea itself.
With respect to the dependent claims, they have been considered and are not found to be reciting anything that amounts to being significantly more than the abstract idea. Claims 12-18 are directed to further embellishments of the central theme of the abstract idea in that the claims are directed to further embellishments of the updating a constraint function of the steps of claim 11 and do not amount to significantly more.
Specifically, claim 12 is directed towards the repetitive performing of optimization which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Furthermore, claim 13 is directed towards the adjusting of a policy which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Additionally, claim 14 is directed towards the adjusting of a policy and subsequent use of reinforcement learning which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Moreover, claim 15 is directed towards the use of a vanilla gradient descent in reinforcement learning which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Furthermore, claim 16 is directed towards the use of a vanilla gradient descent in optimization which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Additionally, claim 17 is directed towards the training of a neural network which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Moreover, claim 18 is directed towards the computation of utilities which can be performed by the human mind and/or pen & paper and does not amount to significantly more.
Claim 20:
A non-transitory computer-readable medium having instructions tangibly stored thereon that, when executed by a processing device of a computing system, cause the computing system to learn a constraint function consistent with a demonstration, by: obtaining: demonstration data representative of the demonstration, the demonstration data comprising a sequence of actions, each action being taken in the context of a respective state of a demonstration environment;
an initial policy operable to determine an action for an agent based on a current state of an agent environment, such that a current policy is set to the initial policy; and
an initial constraint function, such that a current constraint function is set to the initial constraint function;
performing a policy optimization procedure to adjust the current policy, thereby generating an adjusted policy;
adding the adjusted policy to a set of policies;
performing a constraint function optimization procedure to: obtain a mixture policy, based on the set of policies, that defines a second utility comprising the current constraint function applied to the mixture policy; and
adjust the current constraint function to maximize the second utility using a neural network based on the demonstration data and agent data representing one or more actions taken by the agent, such that a third utility is within a constraint threshold, the third utility being the current constraint function applied to the demonstration data; and
providing the current constraint function as the constraint function.
These limitations, as drafted, is an apparatus that, under its broadest reasonable interpretation, covers the performance of mental processes specifically updating a constraint function. Updating a constraint function has long before the modern computer was invented, and continues to be predominantly a product of human endeavor. The instant application is directed to updating a constraint function. Additionally, the claimed determining of an action for an agent can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed setting a constraint function to an initial constraint function can be performed by a human via their mind and/or pen & paper. Moreover, the claimed performing of an optimization procedure can be performed by a human via their mind and/or pen & paper. Additionally, the claimed adding of an adjusted policy to a set of policies can be performed by a human via their mind and/or pen & paper. Furthermore, the claimed adjusting current constrain function can be performed by a human via their mind and/or pen & paper. Additionally, the claimed providing of the current constraint function as the constraint function can be performed by a human via their mind and/or pen & paper. Because the limitations above closely follow the steps of updating a constraint function, and the steps involved human judgments, observations and evaluations that can be practically or reasonably performed in the human mind and/or pen & paper, the claim recites an abstract idea consistent with the “mental process” grouping set forth in the 2019 PEG.
The mere nominal recitation of generic computing components such as a non-transitory computer-readable medium, processing device, and computing device does not take the claim out of certain methods of mental processes grouping. Therefore, the limitation is directed to an abstract idea.
If the claims are directed toward the judicial exception of an abstract idea, it must then be determined under Step 2A Prong 2 whether the judicial exception is integrated into a practical application. The Examiner notes that considerations under Step 2A Prong 2 comprise most the consideration previously evaluated in the context of Step 2B. The Examiner submits that the considerations discussed previously determined that the claim does not recite “significantly more” at Step 2B would be evaluated the same under Step 2A Prong 1 and result in the determination that the claim does not integrate the abstract idea into a practical application.
The instant application fails to integrate the judicial exception into a practical application because the instant application merely recites words “apply it” (or an equivalent) with the judicial exception or merely includes instructions to implement an abstract idea. Specifically, the claim recites an additional element (using a neural network) that is a mere instruction to apply an exception. A recitation of the words “apply it” (or an equivalent) are mere instructions to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The instant application is directed to an apparatus instructing the reader to implement the identified apparatus of mental processes of updating a constraint function. The elements of the claim do not themselves amount to an improvement to the computer, to a technology or another technical field. Moreover, the obtaining of defined demonstration data is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application. Furthermore, the obtaining of a mixture policy is a data gathering operation that is an insignificant data gathering operation that does not integrate the abstract idea into a practical application.
Here, the claim elements entirely comprise the abstract idea, leaving little if any aspects of the claim for further consideration under Step 2A Prong 2. In short, the claims have failed to integrate a practical application (see at least 84 Fed. Reg. (4) at 55). Under the 2019 PEG, this supports the conclusion that the claim is directed to an abstract idea, and the analysis proceeds to Step 2B.
While many considerations in Step 2A need not be reevaluated in Step 2B because the outcome will be the same. Here, on the basis of the additional elements other than the abstract idea, considered individually and in combination as discussed above, the Examiner respectfully submits that the claim 20 does not contain any additional elements that individually or as an ordered combination amount to an inventive concept and the claims are ineligible.
Allowable Subject Matter
6. Claims 1, 11, and 20 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action.
Specifically, although the prior art (See Glazier articles) clearly teaches the obtaining of demonstration data as a basis to perform reinforced learning on constraints and mixed policies, the detailed claim language directed towards the use of the defined second and third utilities for adjusting the current constraint function that is subsequently provided as the current constraint function is not found in the prior art, in conjunction with the rest of the limitations of the independent claims.
Response to Arguments
7. Applicant's arguments filed 11/12/2025 have been fully considered but they are not persuasive.
Applicants argue on Page 10 that “It is submitted that claim 1 similarly recites a combination of elements which reflect a technical improvement to one or more problems in existing techniques for learning constraints from demonstrations. As set forth in paragraphs [0007]-[0014] of the specification, existing techniques for learning constraints from demonstrations exhibit a number of limitations. For example, one recent approach that uses the maximum entropy formulation of the problem assumes a deterministic Markov decision processes (MDP) to formulate the probability of the dataset following a set of constraints, which is not a realistic assumption. Furthermore, this approach can only work with hard constraints, which is not optimal. Moreover, this approach takes a long time to converge (e.g., a few days) and requires a significant amount of hyperparameter tuning, which restricts its practical application”. However, analysis under Step 2A, Prong Two, requires evaluation of any additional claimed elements, individually, and in combination to determine whether they integrate the judicial exception into a practical application. In this case, the claimed additional elements in the independent claims of the obtaining of the defined demonstration data and the obtaining of the mixture policy are not reflective of improvements to a technology. Rather, they are mere data gathering operations that are both insignificant data gathering operations that does not integrate the abstract idea into a practical application. Furthermore, the use of a neural network is a mere equivalent of an “apply it" instruction to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Applicants argue on Pages 10-12 that “The present application illustrates embodiments having features that can solve one or more technical problems in existing approaches for learning constraints from demonstrations…The features illustrated in the above-noted paragraphs of the present application are clearly reflected in claim 1 as now pending. For example, claim 1 as now pending recites, among other features, the features of "performing a constraint function optimization procedure to: obtain a mixture policy, based on the set of policies, that defines a second utility comprising the current constraint function applied to the mixture policy; and adjust the current constraint function to maximize the second utility using a neural network based on the demonstration data and agent data representing one or more actions taken by the agent, such that a third utility is within a constraint threshold, the third utility being the current constraint function applied to the demonstration data"”. However, as explained above, analysis under Step 2A, Prong Two, requires evaluation of any additional claimed elements, individually, and in combination to determine whether they integrate the judicial exception into a practical application. In this case, the claimed additional elements in the independent claims of the obtaining of the defined demonstration data and the obtaining of the mixture policy are not reflective of improvements to a technology. Rather, they are mere data gathering operations that are both insignificant data gathering operations that does not integrate the abstract idea into a practical application. Furthermore, the use of a neural network is a mere equivalent of an “apply it" instruction to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The rest of the limitations of the independent claims (determining of an action for an agent, setting a constraint function to an initial constraint function, performing of an optimization procedure, adding of an adjusted policy to a set of policies, adjusting of a current constrain function, and providing of the current constraint function as the constraint function) can be performed by a human via their mind and/or pen & paper.
Applicants argue on Page 12 that “analogous to Example 48, it is submitted that claim 1 as now pending recites an ordered combination of steps for that provides a solution to one or more technical problems in existing approaches for learning constraints from demonstrations, as described in the specification. Therefore, it is submitted that claim 1 as now pending discloses an ordered combination of steps that reflects a technical improvement to a technical field”. However, the claims are ineligible as similarly described in Claim 1 of Example 48 where the use of a generic neural network is a mere equivalent of an “apply it" instruction to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)).
Applicants argue on Page 13 that “At least for the same reasons as discussed above in Step 2A, Prong Two, it is submitted that the additional elements of claim 1 as now pending or additional elements of claim 1 in combination with the judicial exception satisfy at least the above-identified example limitation ii), and provide improvements to existing approaches for learning constraints from expert demonstrations. In this regard, please consider the above-identified features of "performing a constraint function optimization procedure to: obtain a mixture policy, based on the set of policies, that defines a second utility comprising the current constraint function applied to the mixture policy; and adjust the current constraint function to maximize the second utility using a neural network based on the demonstration data and agent data representing one or more actions taken by the agent, such that a third utility is within a constraint threshold, the third utility being the current constraint function applied to the demonstration data" recited in claim 1 as now pending”. However, as explained above, analysis under Step 2A, Prong Two, requires evaluation of any additional claimed elements, individually, and in combination to determine whether they integrate the judicial exception into a practical application. In this case, the claimed additional elements in the independent claims of the obtaining of the defined demonstration data and the obtaining of the mixture policy are not reflective of improvements to a technology. Rather, they are mere data gathering operations that are both insignificant data gathering operations that does not integrate the abstract idea into a practical application. Furthermore, the use of a neural network is a mere equivalent of an “apply it" instruction to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The rest of the limitations of the independent claims (determining of an action for an agent, setting a constraint function to an initial constraint function, performing of an optimization procedure, adding of an adjusted policy to a set of policies, adjusting of a current constrain function, and providing of the current constraint function as the constraint function) can be performed by a human via their mind and/or pen & paper.
Applicants argue on Page 13 that “the above-identified example limitation v) is met as well, because the above-identified features of claim 1 as now pending include specific limitations other than what is well-understood, routine, conventional activity that are found in the prior art of record, as acknowledged in the Office Action on page 13 under the "Allowable Subject Matter" heading”. However, the examiner did not use Berkheimer rationale for asserting that the additional elements were well-understood, routine, and conventional. Rather, the claimed additional elements in the independent claims of the obtaining of the defined demonstration data and the obtaining of the mixture policy are not reflective of improvements to a technology. Rather, they are mere data gathering operations that are both insignificant data gathering operations that does not integrate the abstract idea into a practical application. Furthermore, the use of a neural network is a mere equivalent of an “apply it" instruction to implement an abstract idea or other exception on a computer (See MPEP 2106.05(f)). Accordingly, these additional elements to not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The rest of the limitations of the independent claims (determining of an action for an agent, setting a constraint function to an initial constraint function, performing of an optimization procedure, adding of an adjusted policy to a set of policies, adjusting of a current constrain function, and providing of the current constraint function as the constraint function) can be performed by a human via their mind and/or pen & paper.
Conclusion
8. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Article entitled “Learning Behavioral Soft Constraints from Demonstrations”, by Glazier et al., dated 21 February 2022. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform reinforcement learning on constraints associated with demonstration data).
Article entitled “Making Human-Like Trade-offs in Constrained Environments by Learning from Demonstrations”, by Glazier et al., dated 22 September 2021. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform reinforcement learning on constraints associated with demonstration data).
U.S. PGPUB 2024/0202504 issued to Kubota et al. on 20 June 2024. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform reinforcement learning on constraints associated with demonstration data).
U.S. PGPUB 2020/0034706 issued to Pham et al. on 30 January 2020. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform reinforcement learning on constraints associated with demonstration data).
9. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
10. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272-2731. The examiner can normally be reached on Monday to Friday 8:20 am – 4:40 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached (571) 272-4085. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Mahesh Dwivedi
Primary Examiner
Art Unit 2168
January 26, 2026
/MAHESH H DWIVEDI/Primary Examiner, Art Unit 2168
Prosecution Insights