Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claims 1-20 are presented in the case. Priority Application claims benefit of priority to Provisional Application No. 63/145,926 filed Feb . 04 , 20 2 1 is acknowledged . Foreign copies have been received for PCT/EP2022/052751 Information Disclosure Statement The information disclosure statement submitted on 10 / 28 /20 24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Objections Claims 9 and 17-18 are objected to because of the following informalities: Claim 9 , line 4 recites the phrase “ and a or the first value estimate ” which should be “ and / or the first value estimate ” Claim 1 7 , line 1 2 recites the phrase “ the method further comprising ” which should be “ the operations further comprising ” Claim 18, line 13 recites the phrase “ the method further comprising ” which should be “ the operations further comprising ” For the informalities above and wherever else they may occur appropriate correction is required. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claim 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”) Claims 1 , 1 7 and 1 8 have the following abstract idea analysis. Step 1 : The claim is directed to “a method, system and crm ”. The claims are directed to the statutory categories accordingly. Step 2A Prong 1 : claims recite the abstract idea limitations of " determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; ” and “ scaling the temporal difference error by the scale factor to determine a scaled temporal difference error; ". These limitations include mathematical concepts see MPEP § 2106.04(a)(2)) where it cites "the phrase “calculating the force of the object by multiplying its mass by its acceleration” is using a textual replacement for the particular equation " and "a conversion between binary coded decimal and pure binary". The specification also provides example calculation of scale factor and scaling (See USPGPUB ¶34 and ¶37). See USPTO 2024 example 47 where an ANN was deemed eligible. That claim did not recite a mathematical concept whereas here the claim itself recites a mathematical concept. Thus, these steps are an abstract idea in the “mathematical concept”. Other sections of the claims such as " obtaining observations ", “ processing the observations to select actions ", " training the reinforcement learning neural network system " and “ updating parameters " are advanced processes, too generic or high level to be listed as a judicial exception given the available descriptions and MPEP comparisons. Step 2A Prong 2: The judicial exceptions recited in these claims are not integrated into a practical application. Merely invoking "a reinforcement learning neural network ", " training ", " computers ", or " storage media " does not yield eligibility. Claims are still in line with mathematical concepts such as claim 1 , 1 7 and 1 8 are not specific to a practical application. The additional elements as such are processors and instructions which do not include specialized hardware. See MPEP § 2106.05(f). Claim 1 , 1 7 and 1 8 do not include a particular field but even doing so may not be sufficient to overcome the abstract idea rejection. Merely applying an model to a field or data without an advancement in the new field or new hardware is ineligible. MPEP § 2106.05(h). Step 2B : The claims do not contain significantly more than their judicial exceptions. Processors, memory and other hardware are in their standard forms in the field. These additional elements are well-understood, routine, and conventional activity, see MPEP 2106.05(d)(II). Claims lacks any particular "how" or algorithm for a solution in a field in a novel way. Claims require more specificity on processes that would be incapable of simple mathematics, mental processes or use more substantial structure than conventional devices such as non-textbook implementations. Regarding claims 2- 16 and 1 9 -20 merely narrow the previously recited abstract idea limitations with more abstract concepts and/or routine fundamental processes. For the reasons described above with respect to claim 1, 17 and 18 this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. Abstract idea steps 1, 2A prong 1 and 2 remain the same as independent analysis above. See specification for more practical application concepts as none are seen in claims 2-16 and 19-20 . A certain type of data used may only be applying concepts to a field of use. With respect to step 2B The claims disclose similar limitations described for the independent claims above and do not provide anything significantly more than mathematical or mental concepts. Claims 2-16 and 19-20 recite the additional elements of "wherein the temporal difference error is further dependent upon a time discounted value estimate generated by the reinforcement learning neural network system, and wherein the square of the scale factor includes a second term, the method further comprising determining a value for the second term by: determining an estimate of a variance of a time discount factor of the time discounted value estimate; determining an estimate of an expectation value of returns-squared, wherein a return comprises a time discounted sum of one or more rewards received after a reinforcement learning time step; and forming a product of the estimate of the variance of a time discount factor and the estimate of the expectation value of the returns-squared. for each of a plurality of action selection time steps: obtaining the observation for a current time step characterizing a current state of the environment; processing the observation for the current time step using the value function neural network and in accordance with current values of value function neural network parameters, to generate a current value estimate relating to the current state of the environment; selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network; and causing the agent to perform the selected action and, in response, receiving a reward for the current time step characterizing progress made in the environment as a result of the agent performing the selected action, the environment transitioning to a next state of the environment; and wherein the method further comprises, for each of a plurality of training time steps: determining a temporal difference error between a first value estimate for a first one of the training time steps generated by processing the observation for the first time step using the value function neural network, and a sum of the reward at the first time step and a time discounted value estimate for a subsequent state of the environment at a subsequent one of the time steps; determining the value of the scale factor; scaling the temporal difference error by the scale factor to determine the scaled temporal difference error; and updating the values of the value function neural network parameters using the scaled temporal difference error. wherein determining the value of the scale factor comprises determining an estimate of the variance of the rewards from the rewards received at the time steps, and using the estimate of the variance of the rewards to determine the first term. determining an estimate of a variance of a time discount factor, wherein the time discount factor is a multiplier of the time discounted value estimate; determining an estimate of an expectation value of returns-squared, wherein a return comprises a time discounted sum of one or more rewards received after a time step; forming a product of the estimate of the variance of a time discount factor and the estimate of the expectation value of the returns-squared. maintaining a target value function neural network with target value function neural network parameters and the same structure as the value function neural network; processing an observation of the next state of the environment using the target value function neural network to determine a value estimate for the subsequent state of the environment; and applying a time discount factor to the value estimate for the subsequent state of the environment to determine the time discounted value estimate for the subsequent state of the environment. determining an estimate of an expectation value of a squared difference between the first value estimate and a value estimate for the state of the environment at the first time step determined by processing the observation for the first time step using the target value function neural network. wherein the value function neural network has multiple heads each to generate a respective first value estimate, determining a respective value of the scale factor for each head; and scaling a respective temporal difference error for each head by the scale factor to determine a respective scaled temporal difference error for updating the values of the value function neural network parameters. wherein the temporal difference error is an n-step temporal difference error, the method comprising determining the n-step temporal difference error between a sum of i ) the reward ii) n−1 subsequent rewards and iii) a time discounted value estimate for the nth subsequent state of the environment, and a or the first value estimate. maintaining an experience replay memory that stores experience tuples generated as a result of the agent interacting with the environment, wherein the experience tuples identify, for each of a plurality of the time steps, at least: the observation, the action selected, the reward received, and a next observation; and sampling the experience tuples in the experience replay memory for performing the plurality of training time steps. wherein sampling the experience tuples prioritizes the sampling using a priority dependent on a magnitude of the scaled temporal difference error. initializing the value of the scale factor to a non-zero value. wherein the method is performed online, the plurality of training time steps corresponds to the plurality of action selection time steps, and the first value estimate for the first time step is the current value estimate for the current time step. an action-value function neural network for determining an action value for each of a plurality of possible actions, wherein the current value estimate is used to determine an action value for each of the possible actions, wherein selecting the action to be performed by the agent in response to the observation comprises selecting the action based on the action value for each of the possible actions, and wherein the time discounted value estimate for a subsequent state of the environment at a subsequent one of the time steps comprises a time discounted action value for the subsequent state of the environment. processing the observation for the time step using the action selection neural network and in accordance with current values of action selection neural network parameters, to generate an action selection output, and selecting, using the action selection output, an action to be performed by the agent in response to the observation; and further comprising updating the action selection neural network parameters using the first value estimate. wherein the agent is a mechanical agent, the environment is a real-world environment, and the actions are actions taken by the mechanical agent in the real-world environment to perform the task." These elements are more abstract concepts, generic applications to a field of use or well-understood, routine, conventional activity (see MPEP § 2106.05(d) and can't be simply appended to qualify as significantly more or being a practical application. What type of application, or structure of components beyond generic machine learning is still unknown for these claims. Therefore claims 2-16 and 19-20 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than the judicial exception, and are rejected under U.S.C. 101 Regarding claim 17 , the claim limitation recites “One or more computer-readable storage media storing instructions”. However, the usage of the phrase “computer-readable storage media” is broad enough to include both “non-transitory” and “transitory” media. The specification further explicitly does not limit the utilization of a non-transitory computer-readable medium (See PGPUB specification, ¶ [ 1 00] where “storage medium" transitory and non-transitory mediums are discussed, however, readable medium is not defined). When the specification is silent, the BRI of a CRM and a computer readable storage media (CRSM) in view of the state of the art covers a signal per se. See Ex parte Mewherter , 2012-007962 (PTAB, 2013). Therefore, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten , 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1 - 2 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Dasgupta et al. (US 11080586 B2 hereinafter Dasgupta) in view of VADORI et al. (US 20210232970 A1 hereinafter VADORI) and BURHANI et al. (US 20190370649 A1 hereinafter BURHANI) As to independent claim 1 , Dasgupta teaches a method of training a computer-implemented reinforcement learning neural network system used to control an agent interacting with an environment to perform a task, the method comprising: [performs reinforcement learning in a neural network Col. 3 ln. 44-58] obtaining observations of states of the environment; [sensors receive observations from an environment like software or a game Col. 4 ln. 9-19 "The observations may be observed through sensors"] processing the observations to select actions to be performed by the agent in response to the observations, wherein the agent receives rewards in response to the actions; [selects and causes actions to be performed and gets rewards Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] training the reinforcement learning neural network system using temporal difference errors, wherein each temporal difference error is dependent upon at least a difference between one of the rewards and a value estimate generated by the reinforcement learning neural network system; the method further comprising: [RL learning (training) using TD-learning based on rewards and values Col. 11-12 ln. 33-54 " a general class of on-policy TD-learning methods for RL. SARSA stands for State-Action-Reward-State-Action,"… "TD-error modulated reinforcement learning"] Dasgupta does not specifically teach determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error . However, VADORI teaches determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; [scale factor (correction factor) (formula with square ¶80-81), and uses rewards ¶4 "calculating a correction factor based on the reward at time t+1, the average reward over time,"] scaling the temporal difference error by the scale factor to determine a scaled temporal difference error; and [scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by VADORI because both techniques address the same field of reinforcement learning and by incorporating VADORI into Dasgupta helps models maximize future gains and provide more optimal actions for agents [ VADORI ¶ 3 ] Dasgupta and VADORI do not specifically teach updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. However, BURHANI teaches updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. [train using a normalized by a scaling factor (updates with applied normalizations ¶101-102 ) in reinforcement learning ¶126 "training engine 118 can train the reinforcement learning network 110 using the normalized order count. The total volume of the order can be normalized by dividing the total volume by a scaling factor"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta and VADORI by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by BURHANI because all techniques address the same field of reinforcement learning and by incorporating BURHANI into Dasgupta and VADORI speeds up reinforcement learning for more optimal and metric based results [ BURHANI ¶ 42] As to dependent claim 2 , the rejection of claim 1 is incorporated Dasgupta , VADORI and BURHANI further teach wherein the temporal difference error is further dependent upon a time discounted value estimate generated by the reinforcement learning neural network system, and wherein the square of the scale factor includes a second term, the method further comprising determining a value for the second term by: determining an estimate of a variance of a time discount factor of the time discounted value estimate; [Dasgupta time based discount factor Col. 11 ln. 33-45 "taking A.sub.t, γ is the discount factor for future reward, and η is the learning rate"] determining an estimate of an expectation value of returns-squared, wherein a return comprises a time discounted sum of one or more rewards received after a reinforcement learning time step; and [ BURHANI mean and standard deviation of rewards include a sum ¶13-14] forming a product of the estimate of the variance of a time discount factor and the estimate of the expectation value of the returns-squared. [BURHANI formula with deviation time and estimate performance ¶80-81] As to dependent claim 16 , the rejection of claim 1 is incorporated Dasgupta , VADORI and BURHANI further teach wherein the agent is a mechanical agent, the environment is a real- world environment, and the actions are actions taken by the mechanical agent in the real-world environment to perform the task. [Dasgupta agent model for robotics or machines (real-work environment and task) Col. 15 ln. 29-47 "A neural network in accordance with the present invention can be used for a myriad of applications"…"robotics"] As to independent claim 17 , Dasgupta teaches one or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to [computer, storage media and programs Col. 14 ln. 26-44] perform operations for training a computer-implemented reinforcement learning neural network system used to control an agent interacting with an environment to perform a task, the operations comprising: [performs reinforcement learning in a neural network Col. 3 ln. 44-58] obtaining observations of states of the environment; [sensors receive observations from an environment like software or a game Col. 4 ln. 9-19 "The observations may be observed through sensors"] processing the observations to select actions to be performed by the agent in response to the observations, wherein the agent receives rewards in response to the actions; [selects and causes actions to be performed and gets rewards Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] training the reinforcement learning neural network system using temporal difference errors, wherein each temporal difference error is dependent upon at least a difference between one of the rewards and a value estimate generated by the reinforcement learning neural network system; the method further comprising: [RL learning (training) using TD-learning based on rewards and values Col. 11-12 ln. 33-54 " a general class of on-policy TD-learning methods for RL. SARSA stands for State-Action-Reward-State-Action,"… "TD-error modulated reinforcement learning"] Dasgupta does not specifically teach determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error . However, VADORI teaches determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; [scale factor (correction factor) (formula with square ¶80-81), and uses rewards ¶4 "calculating a correction factor based on the reward at time t+1, the average reward over time,"] scaling the temporal difference error by the scale factor to determine a scaled temporal difference error; and [scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by VADORI because both techniques address the same field of reinforcement learning and by incorporating VADORI into Dasgupta helps models maximize future gains and provide more optimal actions for agents [ VADORI ¶ 3] Dasgupta and VADORI do not specifically teach updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. However, BURHANI teaches updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. [train using a normalized by a scaling factor (updates with applied normalizations ¶101-102 ) in reinforcement learning ¶126 "training engine 118 can train the reinforcement learning network 110 using the normalized order count. The total volume of the order can be normalized by dividing the total volume by a scaling factor"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta and VADORI by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by BURHANI because all techniques address the same field of reinforcement learning and by incorporating BURHANI into Dasgupta and VADORI speeds up reinforcement learning for more optimal and metric based results [ BURHANI ¶ 42] As to independent claim 18 , Dasgupta teaches a system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to [computer, storage media and programs Col. 14 ln. 26-44] perform operations for training a computer-implemented reinforcement learning neural network system used to control an agent interacting with an environment to perform a task, the operations comprising: [performs reinforcement learning in a neural network Col. 3 ln. 44-58] obtaining observations of states of the environment; [sensors receive observations from an environment like software or a game Col. 4 ln. 9-19 "The observations may be observed through sensors"] processing the observations to select actions to be performed by the agent in response to the observations, wherein the agent receives rewards in response to the actions; [selects and causes actions to be performed and gets rewards Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] training the reinforcement learning neural network system using temporal difference errors, wherein each temporal difference error is dependent upon at least a difference between one of the rewards and a value estimate generated by the reinforcement learning neural network system; the method further comprising: [RL learning (training) using TD-learning based on rewards and values Col. 11-12 ln. 33-54 " a general class of on-policy TD-learning methods for RL. SARSA stands for State-Action-Reward-State-Action,"… "TD-error modulated reinforcement learning"] Dasgupta does not specifically teach determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error . However, VADORI teaches determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; [scale factor (correction factor) (formula with square ¶80-81), and uses rewards ¶4 "calculating a correction factor based on the reward at time t+1, the average reward over time,"] scaling the temporal difference error by the scale factor to determine a scaled temporal difference error; and [scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by VADORI because both techniques address the same field of reinforcement learning and by incorporating VADORI into Dasgupta helps models maximize future gains and provide more optimal actions for agents [ VADORI ¶ 3] Dasgupta and VADORI do not specifically teach updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. However, BURHANI teaches updating parameters of the reinforcement learning neural network system using the scaled temporal difference error. [train using a normalized by a scaling factor (updates with applied normalizations ¶101-102 ) in reinforcement learning ¶126 "training engine 118 can train the reinforcement learning network 110 using the normalized order count. The total volume of the order can be normalized by dividing the total volume by a scaling factor"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta and VADORI by incorporating the determining a value of a scale factor, wherein a square of the scale factor has a first term dependent upon a variance of the rewards; and scaling the temporal difference error by the scale factor to determine a scaled temporal difference error disclosed by BURHANI because all techniques address the same field of reinforcement learning and by incorporating BURHANI into Dasgupta and VADORI speeds up reinforcement learning for more optimal and metric based results [ BURHANI ¶ 42] As to dependent claim 19 , the rejection of claim 1 8 is incorporated Dasgupta , VADORI and BURHANI further teach wherein the temporal difference error is further dependent upon a time discounted value estimate generated by the reinforcement learning neural network system, and wherein the square of the scale factor includes a second term, the method further comprising determining a value for the second term by: determining an estimate of a variance of a time discount factor of the time discounted value estimate; [Dasgupta time based discount factor Col. 11 ln. 33-45 "taking A.sub.t, γ is the discount factor for future reward, and η is the learning rate"] determining an estimate of an expectation value of returns-squared, wherein a return comprises a time discounted sum of one or more rewards received after a reinforcement learning time step; and [BURHANI mean and standard deviation of rewards include a sum ¶13-14] forming a product of the estimate of the variance of a time discount factor and the estimate of the expectation value of the returns-squared. [BURHANI formula with deviation time and estimate performance ¶80-81] Claims 3 -7 , 13-15 and 20 a re rejected under 35 U.S.C. 103 as being unpatentable over Dasgupta in view of VADORI and BURHANI as applied to the rejection of claim 1 -2 and 1 8 above, and further in view of MACGLASHAN (US 20200302323 A1 MACGLASHAN) As to dependent claim 3 , the combination of Dasgupta, VADORI and BURHANI t each all the limitations of claim 1 that is incorporated. Dasgupta, VADORI and BURHANI further teach obtaining the observation for a current time step characterizing a current state of the environment; [Dasgupta sensors receive observations from an environment like software or a game Col. 4 ln. 9-19 "The observations may be observed through sensors"] processing the observation for the current time step using the value function neural network and in accordance with current values of value function neural network parameters, to generate a current value estimate relating to the current state of the environment; [Dasgupta Fig. 1 107 Col. 4- ln. 66-11 "determine a current action-value from an evaluation of action-value function 112"] causing the agent to perform the selected action and, in response, receiving a reward for the current time step characterizing progress made in the environment as a result of the agent performing the selected action, the environment transitioning to a next state of the environment; and [Dasgupta selects and causes actions to be performed and gets rewards Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] wherein the method further comprises, for each of a plurality of training time steps: determining a temporal difference error between a first value estimate for a first one of the training time steps generated by processing the observation for the first time step using the value function neural network, and [ Dasgupta TD-learning based on rewards and values Col. 11-12 ln. 33-54 " a general class of on-policy TD-learning methods for RL. SARSA stands for State-Action-Reward-State-Action,"… "TD-error modulated reinforcement learning"] a sum of the reward at the first time step and a time discounted value estimate for a subsequent state of the environment at a subsequent one of the time steps; [ BURHANI mean and standard deviation of rewards include a sum ¶13-14] determining the value of the scale factor; [VADORI scale factor (correction factor) (formula with square ¶80-81), and uses rewards ¶4 "calculating a correction factor based on the reward at time t+1, the average reward over time,"] scaling the temporal difference error by the scale factor to determine the scaled temporal difference error; and [VADORI scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] updating the values of the value function neural network parameters using the scaled temporal difference error. [train using a normalized by a scaling factor (updates with applied normalizations ¶101-102) in reinforcement learning ¶126 "training engine 118 can train the reinforcement learning network 110 using the normalized order count. The total volume of the order can be normalized by dividing the total volume by a scaling factor"] Dasgupta, VADORI and BURHANI do not specifically teach selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network . However, MACGLASHAN teaches selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network; [critic model called an action-value model for selecting actions ¶46, ¶11 " action-value model estimating, within one or more processors of the agent, an expected future discounted reward that would be received if a hypothetical action was selected under a current observation of the agent and the agent's behavior was followed thereafter"]. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta, VADORI and BURHANI by incorporating the selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network disclosed by MACGLASHAN because all techniques address the same field of reinforcement learning and by incorporating MACGLASHAN into Dasgupta, VADORI and BURHANI reduces overfitting in models for stable results [ MACGLASHAN ¶ 4-5] As to dependent claim 4 , the rejection of claim 3 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach wherein determining the value of the scale factor comprises determining an estimate of the variance of the rewards from the rewards received at the time steps, and using the estimate of the variance of the rewards to determine the first term. [BURHANI mean and standard deviation of rewards are an estimate of variance of rewards ¶13-14] As to dependent claim 5 , the rejection of claim 3 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach wherein the square of the scale factor includes a second term, the method further comprising determining a value for the second term by: determining an estimate of a variance of a time discount factor, wherein the time discount factor is a multiplier of the time discounted value estimate; [Dasgupta time-based discount factor Col. 11 ln. 33-45 "taking A.sub.t, γ is the discount factor for future reward, and η is the learning rate"] determining an estimate of an expectation value of returns-squared, wherein a return comprises a time discounted sum of one or more rewards received after a time step; [ BURHANI mean and standard deviation of rewards include a sum ¶13-14] forming a product of the estimate of the variance of a time discount factor and the estimate of the expectation value of the returns-squared. [BURHANI formula with deviation time and estimate performance ¶80-81] As to dependent claim 6 , the combination of Dasgupta, VADORI and BURHANI teach all the limitations of claim 2 that is incorporated. Dasgupta, VADORI and BURHANI further teach processing an observation of the next state of the environment using the target value function neural network to determine a value estimate for the subsequent state of the environment; and [Dasgupta Fig. 1 107 Col. 4- ln. 66-11 "determine a current action-value from an evaluation of action-value function 112"] applying a time discount factor to the value estimate for the subsequent state of the environment to determine the time discounted value estimate for the subsequent state of the environment. [Dasgupta time based discount factor Col. 11 ln. 33-45 "taking A.sub.t, γ is the discount factor for future reward, and η is the learning rate"] Dasgupta, VADORI and BURHANI do not specifically teach determining an estimate of an expectation value of a squared difference between the first value estimate and a value estimate for the state of the environment at the first time step determined by processing the observation for the first time step using the target value function neural network. However, MACGLASHAN teaches determining an estimate of an expectation value of a squared difference between the first value estimate and a value estimate for the state of the environment at the first time step determined by processing the observation for the first time step using the target value function neural network. [MACGLASHAN difference between estimate from critic and target model ¶73 "The loss function may then be represented using any norm of the difference between the critic and target. "] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta, VADORI and BURHANI by incorporating the determining an estimate of an expectation value of a squared difference between the first value estimate and a value estimate for the state of the environment at the first time step determined by processing the observation for the first time step using the target value function neural network disclosed by MACGLASHAN because all techniques address the same field of reinforcement learning and by incorporating MACGLASHAN into Dasgupta, VADORI and BURHANI reduces overfitting in models for stable results [ MACGLASHAN ¶ 4-5] As to dependent claim 7 , the rejection of claim 6 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach wherein the square of the scale factor includes a third term, the method further comprising determining a value for the third term by: determining an estimate of an expectation value of a squared difference between the first value estimate and a value estimate for the state of the environment at the first time step determined by processing the observation for the first time step using the target value function neural network. [MACGLASHAN difference between estimate from critic and target model ¶73 "The loss function may then be represented using any norm of the difference between the critic and target. " ]. As to dependent claim 13 , the rejection of claim 3 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach wherein the method is performed online, the plurality of training time steps corresponds to the plurality of action selection time steps, and the first value estimate for the first time step is the current value estimate for the current time step. [MACGLASHAN online variant for training ¶10 "an online variant, in which data is collected as the algorithm trains the policy model"; timesteps ¶47] As to dependent claim 14 , the rejection of claim 3 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach wherein the value function neural network comprises an action-value function neural network for determining an action value for each of a plurality of possible actions, wherein the current value estimate is used to determine an action value for each of the possible actions, wherein selecting the action to be performed by the agent in response to the observation comprises selecting the action based on the action value for each of the possible actions, and wherein the time discounted value estimate for a subsequent state of the environment at a subsequent one of the time steps comprises a time discounted action value for the subsequent state of the environment. [Dasgupta selects and causes actions from possible actions using possible rewards from each Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] As to dependent claim 15 , the rejection of claim 3 is incorporated Dasgupta , VADORI BURHANI and MACGLASHAN further teach processing the observation for the time step using the action selection neural network and in accordance with current values of action selection neural network parameters, to generate an action selection output, and selecting, using the action selection output, an action to be performed by the agent in response to the observation; and further comprising updating the action selection neural network parameters using the first value estimate. [MACGLASHAN action model (neural network) with updates ¶8 "action-value model and the policy model, wherein the stale copy is initialized identically to the fresh copy and is slowly moved to match the fresh copy as learning updates are performed on the fresh copy, wherein the algorithm has both an offline variant, in which the algorithm is trained using previously collected data, and an online variant, in which data is collected as the algorithm trains the policy model."] As to dependent claim 20 , the combination of Dasgupta, VADORI and BURHANI teach all the limitations of claim 18 that is incorporated. Dasgupta, VADORI and BURHANI further teach obtaining the observation for a current time step characterizing a current state of the environment; [Dasgupta sensors receive observations from an environment like software or a game Col. 4 ln. 9-19 "The observations may be observed through sensors"] processing the observation for the current time step using the value function neural network and in accordance with current values of value function neural network parameters, to generate a current value estimate relating to the current state of the environment; [Dasgupta Fig. 1 107 Col. 4- ln. 66-11 "determine a current action-value from an evaluation of action-value function 112"] causing the agent to perform the selected action and, in response, receiving a reward for the current time step characterizing progress made in the environment as a result of the agent performing the selected action, the environment transitioning to a next state of the environment; and [Dasgupta selects and causes actions to be performed and gets rewards Col. 4 ln. 41-54 "select the possible action that yields the largest reward probability from the probability function."] wherein the method further comprises, for each of a plurality of training time steps: determining a temporal difference error between a first value estimate for a first one of the training time steps generated by processing the observation for the first time step using the value function neural network, and [ Dasgupta TD-learning based on rewards and values Col. 11-12 ln. 33-54 " a general class of on-policy TD-learning methods for RL. SARSA stands for State-Action-Reward-State-Action,"… "TD-error modulated reinforcement learning"] a sum of the reward at the first time step and a time discounted value estimate for a subsequent state of the environment at a subsequent one of the time steps; [ BURHANI mean and standard deviation of rewards include a sum ¶13-14] determining the value of the scale factor; [VADORI scale factor (correction factor) (formula with square ¶80-81), and uses rewards ¶4 "calculating a correction factor based on the reward at time t+1, the average reward over time,"] scaling the temporal difference error by the scale factor to determine the scaled temporal difference error; and [VADORI scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] updating the values of the value function neural network parameters using the scaled temporal difference error. [train using a normalized by a scaling factor (updates with applied normalizations ¶101-102) in reinforcement learning ¶126 "training engine 118 can train the reinforcement learning network 110 using the normalized order count. The total volume of the order can be normalized by dividing the total volume by a scaling factor"] Dasgupta, VADORI and BURHANI do not specifically teach selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network . However, MACGLASHAN teaches selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network; [critic model called an action-value model for selecting actions ¶46, ¶11 " action-value model estimating, within one or more processors of the agent, an expected future discounted reward that would be received if a hypothetical action was selected under a current observation of the agent and the agent's behavior was followed thereafter"]. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta, VADORI and BURHANI by incorporating the selecting an action to be performed by the agent in response to the observation, using the current value estimate or using an action selection neural network updated using value estimates generated by the value function neural network disclosed by MACGLASHAN because all techniques address the same field of reinforcement learning and by incorporating MACGLASHAN into Dasgupta, VADORI and BURHANI reduces overfitting in models for stable results [ MACGLASHAN ¶ 4-5] Claim 8 is r ejected under 35 U.S.C. 103 as being unpatentable over Dasgupta in view of VADORI and BURHANI as applied to the rejection of claim 1 above, and further in view of DA SILVA et al. (US 20210073912 A1 herein Da Silva) As to dependent claim 8 , the combination of Dasgupta, VADORI and BURHANI teach all the limitations of claim 1 that is incorporated. Dasgupta, VADORI and BURHANI further teach scaling a respective temporal difference error for each head by the scale factor to determine a respective scaled temporal difference error for updating the values of the value function neural network parameters. [VADORI scales (applies factor to data) for corrected results ¶70-73 "apply the risk-sensitive policy with the correction factor Q.sup .β(s.sub.t, a.sub.t) to the real-time data"] Dasgupta, VADORI and BURHANI do not specifically teach wherein the value function neural network has multiple heads each to generate a respective first value estimate, the method comprising: determining a respective value of the scale factor for each head . However, Da Silva teaches wherein the value function neural network has multiple heads each to generate a respective first value estimate, the method comprising: determining a respective value of the scale factor for each head; [multiple heads with value estimates ¶56 "Each head estimates a value for each action. Due to the aleatoric nature of the exploration and network initialization, each head will output a different estimate of the action values"] Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the reinforcement network by Dasgupta, VADORI and BURHANI by incorporating the wherein the value function neural network has