DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. This action is responsive to the application filed 7 / 2 0 /202 2 . Claims 1 - 2 2 are presented for examination. Information Disclosure Statement The information disclosure statement s (IDS) submitted 3 / 7 /202 3, 1/5/2024, 4/1/2025, and 10/15/2025 ha ve been considered by the examiner. Claim Objections Claim 5 is objected to because of the following informalities: Claim 5, line 1 in the claim limitations, recites in part “combining the first state the hierarchical state” should be --combining the first state of the hierarchical state-- to correct for apparent typographical error. Appropriate correction is required. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1- 2 2 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”). Claim 1 Step 1 : The claim recites “ A method, comprising : ”; therefore, it is directed to the statutory category of a process . Step 2A Prong 1 : The claim recites, inter alia: identifying a state sequence including a first state, a second state, and a third state from the plurality of states such that the agent can perform a first action to implement a transition from the first state to the second state and a second action to implement a transition from the second state to the third state in a consecutive manner : These limitations recite mentally performable process with aid of pen and paper of using judgement to identify a state sequence including a first state, a second state, and a third state from the plurality of states designed for the agent to be able to perform a first action to implement a transition from the first state to the second state and a second action to implement a transition from the second state to the third state in a consecutive manner. generating a hierarchical state configured to be associated with (i) a third action implementing a transition from the first state to the hierarchical state, and (ii) a fourth action implementing a transition from the hierarchical state to the third state, the first action and the second action forming an option : These limitations recite a mentally performable process with aid of pen and paper using judgement to generate a hierarchical state configured to be associated with (i) a third action implementing a transition from the first state to the hierarchical state, and (ii) a fourth action implementing a transition from the hierarchical state to the third state, the first action and the second action forming an option. and setting a value associated with the transition from the first state to the hierarchical state to be equal to a value combination that is a function of a value associated with the first action and a value associated with the second action : These limitations recite a mathematical relationship of setting a value associated with the transition from the first state to the hierarchical state to be equal to a value combination that is a function of a value associated with the first action and a value associated with the second action. and setting a value associated with the transition from the hierarchical state to the third state to be equal to a maximum value associated with the third state : These limitations recite a mathematical relationship of setting a value associated with the transition from the hierarchical state to the third state to be equal to a maximum value associated with the third state. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: receiving inputs associated with interactions of an agent with an environment, the interactions including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states : These additional elements recite receiving at a high level of generality and are considered to be insignificant extra-solution activity of data gathering and selecting a particular data source or type of data to be manipulated, e.g. interactions with an agent including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states. See MPEP 2106.05(g). receiving an indication of a target state to be achieved by the agent in the environment : These additional elements recite receiving at a high level of generality and are considered to be insignificant extra-solution activity of data gathering and selecting a particular data source or type of data to be manipulated, e.g. an indication of a target state to be achieved by the agent in the environment. See MPEP 2106.05(g). Step 2B : The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception , insignificant extra-solution activity recited by “ receiving inputs associated with interactions of an agent with an environment, the interactions including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states ” and “ receiving an indication of a target state to be achieved by the agent in the environment ” which are considered well-understood, routine, and conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 2 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 1. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the third action and the fourth action are hierarchical actions : These additional elements merely indicate a field of use or technological environment wherein the third action and the fourth action are hierarchical actions. See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 3 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: further comprising: computing the value combination : These limitations recite a mathematical calculation of computing the value combination. verifying that the value combination has a non-zero value; and verifying that the state sequence is non-cyclical, the generating the hierarchical state being based on the value combination having a non-zero value and the state sequence being non- cyclical : These limitations recite mathematical relationships of describing relationships of numbers and variables for the mentally performable process of generating of the hierarchical state of claim 1. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 4 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: wherein the hierarchical state is associated with a value combination of values from non-hierarchical states : These limitations recite a mathematical relationship describing relationship between the hierarchical state’s associated value combination and values from non-hierarchical states. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 5 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: further comprising: combining the first state of the hierarchical state and the third state to generate a sequence of states associated with the third action and the fourth action, the sequence of states forming an option sequence : These limitations recite a mentally performable process with aid of pen and paper of using judgement to combine the mentally identified first state of the hierarchical state and the identified third state to generate a sequence of states associated with the third action and the fourth action, the sequence of states determined to form an option sequence. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 6 Step 1 : a process , as in claim 3 . Step 2A Prong 1 : The claim recites, inter alia: further comprising: comparing the value combination with a threshold value, the generating the hierarchical state being based on the value combination being greater than the threshold value : These limitations furthers the mentally performable process of claims 3 and 1 of generating the hierarchical state by further using judgement to compare the value combination with a threshold value and determining whether the value combination is greater than the threshold value before generating the hierarchical state. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 7 Step 1 : a process , as in claim 6 . Step 2A Prong 1 : The claim recites, inter alia: further comprising: receiving data associated with a performance of the agent in the environment; and modifying the threshold value based on the data associated with the performance of the agent in the environment : These limitations recite a mentally performable process of observing data gathered associated with a performance of the agent in the environment and using judgement to modify the threshold value based on the observed data gathered. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 8 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: further comprising: determining that the state sequence including the first state, the second state, and the third state to be generalizable : These limitations recite a mentally performable process of using judgement to determine that the state sequence including the state, the second state, and the third state to be generalizable for designing of agent of claim 1. and discovering the hierarchical state configured to form the option, the option being configured to implement a transition from the first action to the second action in a reusable sequence : These limitations recite a mentally performable process of using judgement to discover the hierarchical state configured to form the option, wherein the option being designed to implement a transition from the first action to the second action in a reusable sequence. Step 2A Prong 2 & Step 2B : The are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such the claim is patent ineligible. Claim 9 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: the method further comprising: discovering the hierarchical state and the option; determining that the option does not exist in an options dictionary; and forming the option in response to the determining that the option does not exist in the options dictionary : These limitations recite a mentally performable process with aid of pen and paper of using judgement to discovering the hierarchical state and the option and determine that the option does not exist in an options dictionary; and forming the option in response to the determining that the option does not exist in the options dictionary. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the second state is associated with a first number of potential decisions to implement a transition from the first state to the third state, and the hierarchical state is associated with a second number of potential decisions to implement the transition from the first state to the third state : These additional elements merely indicate a field of use or technological environment wherein the second state is associated with a first number of potential decisions to implement a transition from the first state to the third state, and the hierarchical state is associated with a second number of potential decisions to implement the transition from the first state to the third state. See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment . Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 10 Step 1 : a process , as in claim 1 . Step 2A Prong 1 : The claim recites, inter alia: the method further comprising: generating a second hierarchical state associated with a plurality of actions and at least two of the first state, the second state, the third state, the first hierarchical state, or a fourth state different than the first state, the second state, the third state, and the first hierarchical state : These limitations recite a mentally performable process with aid of pen and paper of furthering using judgement to generate a second hierarchical state associated with a plurality of actions and at least two of the first state, the second state, the third state, the first hierarchical state, or a fourth state different than the first state, the second state, the third state, and the first hierarchical state. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the hierarchical state is a first hierarchical state : These additional elements merely indicate a field of use or technological environment wherein the hierarchical state is a first hierarchical state. See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment . Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 11 Step 1 : The claim recites “ A n apparatus, comprising : ”; therefore, these claims are directed to the statutory category of a machine . Step 2A Prong 1 : The claim recites, inter alia: identify a state sequence including a first state, a second state, and a third state from the plurality of states such that the agent can perform a first action to implement a transition from the first state to the second state and a second action to implement a transition from the second state to the third state in a consecutive manner, at least one of the first state, the second state, or the third state being a hierarchical state and associated with a primitive action : These limitations recite a mentally performable process with aid of pen and paper of using judgement to identify a state sequence including a first state, a second state, and a third state from the plurality of states such that the agent can be designed to perform a first action to implement a transition from the first state to the second state and a second action to implement a transition from the second state to the third state in a consecutive manner, at least one of the first state, the second state, or the third state being a hierarchical state and associated with a primitive action. determine an identifier associated with the hierarchical state; search a dictionary associated with the machine learning model to determine whether the identifier associated with the hierarchical state is included in the dictionary; add, based on the determination that the identifier associated with the hierarchical state is not included in the dictionary, the identifier associated with the hierarchical state to the dictionary to generate an updated dictionary; and store the updated dictionary : These limitations recite mentally performable processes with aid of pen and paper of using judgement to determine an identifier associated with the identified hierarchical state, search an observed dictionary associated with the machine learning model to determine whether the identifier associated with the hierarchical state is included in the dictionary; add, based on the determination that the identifier associated with the hierarchical state is not included in the dictionary, the identifier associated with the hierarchical state to the dictionary to generate an updated dictionary, and store the updated dictionary by annotation of pen and paper. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: An apparatus, comprising: a memory; and a hardware processor operatively coupled to the memory, the hardware processor configured to : These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity, i.e. apparatus comprising: a memory; and a hardware processor operatively coupled to the memory, the hardware processor configured, to implement the underlying judicial exception. See MPEP 2106.05(f). receive inputs associated with interactions of an agent with an environment, the interactions including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states : These additional elements recite receive at a high level of generality and are considered to be insignificant extra-solution activity of data gathering and selecting a particular data source or type of data to be manipulated, e.g. interactions with an agent including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states. See MPEP 2106.05(g). receive an indication of a target state to be achieved by the agent in the environment by implementing a machine learning model : These additional elements are mere instructions to implement the judicial exception because the additional elements only recite the idea of a solution of implementing a machine learning model to receive an indication of a target state to be achieved by the agent in the environment but failing to recite details of how the machine learning model is implemented, e.g. no details as to whether the machine learning model implemented by the agent or somewhere else, whether the machine learning model implementation involves training a machine learning model or using a pre-trained model, etc. Thus, these additional elements cannot be considered to integrate a judicial exception into a practical application. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception, insignificant extra-solution activity recited by “ receive inputs associated with interactions of an agent with an environment, the interactions including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states ” which are considered well-understood, routine, and conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II). Additionally, the additional elements further include adding the words “apply it” (or an equivalent) with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 1 2 Step 1 : a machine , as in claim 1 1 . Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 11. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the state that is a hierarchical state is associated with fewer actions than a state that is not a hierarchical state : These additional elements are considered to be insignificant extra-solution activity of arranging a hierarchy of groups , sorting information , eliminating information and determining information . See MPEP 2106.05(d)(II) and MPEP 2106.05(g). Step 2B : The additional elements from Step 2A Prong 2 include insignificant extra-solution activity recited by “ wherein the state that is a hierarchical state is associated with fewer actions than a state that is not a hierarchical state ” which are considered well-understood, routine, and conventional activity similar to arranging a hierarchy of groups, sorting information, eliminating less restrictive pricing information and determining the price described in MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 1 3 Step 1 : a machine , as in claim 11 . Step 2A Prong 1 : The claim recites, inter alia: the first action is associated with a value combination that is a function of a value associated with the third action and a value associated with the fourth action, the second action is associated with a maximum value associated with the third state : These limitations recite mathematical relationships defining the first action is associated with a value combination that is a function of a value associated with the third action and a value associated with the fourth action, the second action is associated with a maximum value associated with the third state. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the second state is the hierarchical state, the hierarchical state is an abstraction of a fourth state, the hierarchical state is associated with fewer actions than the fourth state, a third action implements a transition from the first state to the fourth state, a fourth action implements a transition from the fourth state to the third state : These additional elements are considered to be insignificant extra-solution activity of arranging a hierarchy of groups, sorting information, eliminating information and determining information . See MPEP 2106.05(d)(II) and MPEP 2106.05(g). Step 2B : The additional elements from Step 2A Prong 2 include insignificant extra-solution activity recited by “ wherein the state that is a hierarchical state is associated with fewer actions than a state that is not a hierarchical state ” which are considered well-understood, routine, and conventional activity similar to arranging a hierarchy of groups, sorting information, eliminating less restrictive pricing information and determining the price described in MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 1 4 Step 1 : a machine , as in claim 11 . Step 2A Prong 1 : The claim recites, inter alia: the processor being further configured to add the identifier associated with the hierarchical state to the dictionary to generate the updated dictionary further based on a determination that a value combination that is a function of the first value and the second value is greater than a threshold : These limitations recite a mathematical relationship used to organize and manipulate the identifier information associate with the hierarchical state to the dictionary to generate the updated dictionary via mathematical correlation of a value combination that is a function of the first value and the second value being greater than a threshold similar to organizing information and manipulating information through mathematical correlations per MPEP 2106.04(a)(2)(A)(iv). Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the machine learning model is configured to implement a plurality of interactions between the agent and the environment, the hardware processor further configured to implement, at a first time, a first set of interactions from the plurality of interactions between the agent and the environment to transition from the first state to third state via the second state, the first set of interactions being associated with a first value : These additional elements are mere instructions to implement the judicial exception of identifying the plurality of states and associated transitions and actions because the additional elements only recite the idea of a solution of implementing with the machine learning model a plurality of interactions between the agent and the environment at a first time via a first set of interactions associated with a first value to transition from the first state to third state via the second state but failing to recite details of how the machine learning model implements the first set of interactions, no details as to what type of interactions other than that it is generally associated with a first value, no details as to the architecture of the machine learning model and if it is pre-trained or if this occurs during training/etc. Thus, these additional elements cannot be considered to integrate a judicial exception into a practical application. See MPEP 2106.05(f). implement, at a second time, a second set of interactions from the plurality of interactions between the agent and the environment to transition from the first state to third state via the hierarchical state, the second set of interactions being associated with a second value : These additional elements are mere instructions to implement the judicial exception of identifying the plurality of states and associated transitions and actions because the additional elements only recite the idea of a solution of implementing with the machine learning model a second set of interactions associated with a second value between the agent and the environment at a second time to transition from the first state to third state via the hierarchical state but failing to recite details of how the machine learning model implements the second set of interactions, no details as to what type of interactions other than that it is generally associated with a second value, no details as to how the hierarchical state is used to facilitate the transition from the first state to third state, etc. Thus, these additional elements cannot be considered to integrate a judicial exception into a practical application. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include adding the words “apply it” (or an equivalent) with the judicial exception . Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 1 5 Step 1 : a machine , as in claim 11 . Step 2A Prong 1 : The claim recites, inter alia: implement, a set of interactions from the plurality of interactions between the agent and the environment to transition from the first state to third state via the hierarchical state, the set of interactions being associated with receiving a reward signal; and automatically adjust at least one hyperparameter from the set of hyperparameters in response to receiving the reward signal : These limitations recite a mentally performable process with aid of pen and paper of using judgement to implement design of a set of interactions from the plurality of interactions between the agent and the environment to transition from the first state to third state via the hierarchical state, the set of interactions being associated with receiving a reward signal and automatically adjust at least one hyperparameter from the set of hyperparameters in response to observing the reward signal. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the machine learning model is associated with a set of hyperparameters used to implement a plurality of interactions between the agent and the environment : These additional elements merely indicate a field of use or technological environment wherein the machine learning model is associated with a set of hyperparameters used to implement a plurality of interactions between the agent and the environment. See MPEP 2106.05(h). the hardware processor further configured to: These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity, i.e. apparatus comprising : a memory; and a hardware processor operatively coupled to the memory, the hardware processor configured, to implement the underlying judicial exception. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception and invoking computers or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 16 Step 1 : a machine , as in claim 11. Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 11. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the hierarchical state is generated by merging two or more states : These additional elements merely indicate a field of use or technological environment wherein the hierarchical state is generated by merging two or more states. See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 17 Step 1 : The claim recites “ A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the instructions comprising the code to cause the processor to : ”; therefore, these claims are directed to the statutory category of a n article of manufacture . Step 2A Prong 1 : The claim recites, inter alia: identify, based on the data associated with the interactions between the first agent and the first environment, a set of actions configured to be performed by the second agent while the second agent interacts with the second environment : These limitations recite a mentally performable process with aid of pen and paper of using judgement to identify based on observation of the data associated with the interactions between the first agent and the first environment, a set of actions configured to be performed by observing the second agent while the second agent interacts with the second environment. implement the second agent to perform an action from the set of actions, the action being configured to increase a likelihood of achieving the goal : These limitations recite a mentally performable process with aid of pen and paper of implementing a design of the second agent to perform an action from observing the set of actions, the action evaluated to increase a likelihood of achieving the observed goal. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the instructions comprising code to cause the processor to : These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity, i.e. a non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the instructions comprising code to cause the processor, to implement the underlying judicial exception. See MPEP 2106.05(f). receive data associated with interactions between a first agent and a first environment associated with a domain : These additional elements recite receive at a high level of generality and are considered to be insignificant extra-solution activity of data gathering and selecting a particular data source or type of data to be manipulated, e.g. data associated with interactions between a first agent and a first environment associated with a domain. See MPEP 2106.05(g). receive information about a second environment associated with the domain, the information including a goal that is desired to be achieved in the second environment : These additional elements recite receive at a high level of generality and are considered to be insignificant extra-solution activity of data gathering and selecting a particular data source or type of data to be manipulated, e.g. information about a second environment associated with the domain, the information including a goal that is desired to be achieved in the second environment. See MPEP 2106.05(g). implement, using a machine learning model, a second agent configured to interact with the second environment : These additional elements are mere instructions to implement the judicial exception because the additional elements only recite the idea of a solution of implementing using a machine learning model a second agent configured to interact with the second environment but fails to recite details of how the machine learning model is used to implement a second agent, no details as to how interactions with the second environment is accomplished by using a machine learning model, no details as to the architecture or design of the machine learning model, e.g. if it is pre-trained or operating in a training mode, etc. Thus, these additional elements cannot be considered to integrate a judicial exception into a practical application. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception, insignificant extra-solution activity recited by “ receive data associated with interactions between a first agent and a first environment associated with a domain ” and “ receive information about a second environment associated with the domain, the information including a goal that is desired to be achieved in the second environment ” which are considered well-understood, routine, and conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II). Additionally, the additional elements further include adding the words “apply it” (or an equivalent) with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 18 Step 1 : a n article of manufacture , as in claim 17. Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 17. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the first agent is the same as the second agent : These additional elements merely indicate a field of use or technological environment wherein the first agent is the same as the second agent . See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 19 Step 1 : a n article of manufacture , as in claim 17. Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 17. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the goal is a second goal, and the first environment is implemented to perform a first task, to achieve a first goal, in the domain and the second environment is implemented to perform a second task, to achieve the second goal, in the domain : These additional elements merely indicate a field of use or technological environment wherein the goal is a second goal, and the first environment is implemented to perform a first task, to achieve a first goal, in the domain and the second environment is implemented to perform a second task, to achieve the second goal, in the domain. See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 20 Step 1 : a n article of manufacture , as in claim 19. Step 2A Prong 1 : The claim recites the same abstract ideas as in claim 19. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the domain is at least one of financial trading, agricultural technology, or natural language processing (NLP) : These additional elements merely indicate a field of use or technological environment wherein the domain is at least one of financial trading, agricultural technology, or natural language processing (NLP). See MPEP 2106.05(h). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 21 Step 1 : a n article of manufacture , as in claim 17. Step 2A Prong 1 : The claim recites, inter alia: implement the second agent to perform the first action : These limitations recite a mentally performable process with aid of pen and paper using judgement to implement the design of the second agent to perform the first action. implement the second agent to perceive the second environment to transition from a first state to a second state, the second state being a hierarchical state configured to increase the likelihood of achieving the goal and increasing an efficiency in computation associated with achieving the goal : These limitations recite a mentally performable process with aid of pen and paper using judgement to implement the design of the second agent to perceive the second environment to transition from a first state to a second state, the second state being a hierarchical state configured to increase the likelihood of achieving the goal and increasing an efficiency in computation associated with achieving the goal. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the action is a first action: These additional elements merely indicate a field of use or technological environment wherein the action is a first action. See MPEP 2106.05(h). the instructions comprising code to cause the processor to … include code to cause the processor to: These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity, i.e. the instructions comprising code to cause the processor, to implement the underlying judicial exception. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment for the underlying judicial exception and invoking computers or other machinery to apply the underlying judicial exception . Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim 22 Step 1 : a n article of manufacture , as in claim 17. Step 2A Prong 1 : The claim recites, inter alia: implement the second agent to perform the action : These limitations recite a mentally performable process with aid of pen and paper using judgement to implement the design of the second agent to perform the action. implement the second agent to perceive the second environment to transition from a first state to a second state, the second state being a hierarchical state configured to increase the likelihood of achieving the goal and increasing an efficiency in computation associated with achieving the goal : These limitations recite a mentally performable process with aid of pen and paper using judgement to implement the design of the second agent to perceive the second environment to transition from a first state to a second state, the second state being a hierarchical state configured to increase the likelihood of achieving the goal and increasing an efficiency in computation associated with achieving the goal. Step 2A Prong 2 : This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows: wherein the instructions comprising code to cause the processor to … include code to cause the processor to: These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity, i.e. the instructions comprising code to cause the processor, to implement the underlying judicial exception. See MPEP 2106.05(f). Step 2B : The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself . See MPEP 2106.05. Claim Rejections - 35 USC § 102 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis ( i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale , or otherwise available to the public before the effective filing date of the claimed invention. Claims 1-2 and 4-5 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sutton et al., “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning” (2019) (hereinafter Sutton). Regarding independent claim 1, Sutton discloses a method, comprising: receiving inputs associated with interactions of an agent with an environment, the interactions including a plurality of states associated with the environment and a plurality of actions associated with each state from the plurality of states (page 184 standard framework of discrete-time, finite Markov decision processes (MDPs) comprising an RL agent that interacts with an environment at discrete time steps, the agent perceives the state ( st ) and chooses a primitive action ( at ), the environment produces a next state ( st +1) and a numerical reward ( rt +1) in response to the action); receiving an indication of a target state to be achieved by the agent in the environment (pages 184-185 The overall objective of the agent in the framework described is to maximize the expected discounted future reward. Page 185 discloses The concept of a termination point or goal is fundamental to the MDP framework wherein tasks can be episodic, involving repeated trials ending with a special terminal state. Pages 193-194 examples provided, such as the rooms environment, explicitly define goal states ( G 1 or G 2) that the agent navigates to. Formally, a goal state is a state from which actions lead to the terminal state with a positive reward); identifying a state sequence including a first state, a second state, and a third state from the plurality of states such that the agent can perform a first action to implement a transition from the first state to the second state and a second action to implement a transition from the second state to the third state in a consecutive manner (page 182 MDPs are based on sequential, discrete-time steps, where a course of action persists over a period of time. Page 184 Identifying a state sequence including a first state, a second state, and a third state such that the agent performs a first action (S1 → S2) and a second action (S2 → S3) in a consecutive manner. This element describes a basic sequence of primitive actions and transitions within a standard MDP. The state trajectory of an MDP is made up of small, discrete-time transitions. This fundamental framework inherently includes sequences of consecutive states and actions) ; generating a hierarchical state configured to be associated with (i) a third action implementing a transition from the first state to the hierarchical state, and (ii) a fourth action implementing a transition from the hierarchical state to the third state, the first action and the second action forming an option (pages 181, 186-187 options as temporally extended courses of action that encapsulate closed-loop policies for taking action over a period of time as part of central concept of temporal abstraction. The composition of primitive actions (the first action and the second action) yields a temporally extended course of action, which is referred to as an option wherein options allow temporally abstract knowledge and action to be included in the reinforcement learning framework in a natural and general way. Page 183, 188 explicitly establishes that a set of options defines a Semi-Markov Decision Process (SMDP) embedded within the original MDP. The essence of this concept is that the option (the abstract action replacing the sequence of primitive actions) allows the system to analyze the MDP trajectory at a higher, temporally abstract level. The actions defined in the claim (third action and fourth action) that transition to/from the "hierarchical state" functionally represent the initiation and termination phases of a single, abstract option, derived from the sequence of primitive actions) ; and setting a value associated with the transition from the first state to the hierarchical state to be equal to a value combination that is a function of a value associated with the first action and a value associated with the second action (pages 189, 202 provide the theoretical basis for calculating the value of these temporally extended options, which necessarily combines the values/rewards of the constituent primitive actions, defines the cumulative discounted reward for an option o , denoted rso , as the expected cumulative discounted reward received along the way, until the option terminates. This reward rso is derived by summing or integrating (as a discounted expectation) the immediate rewards ( rt +1, rt +2, etc.) obtained from the primitive actions that make up the option. The value of the abstract