Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to the claims filed on 11/12/2025.
Claims 1, 3, 5, 7, 8, 10, 12, 14, 15, 17, 19 are presented for examination, the claims 2, 4, 6, 9, 11, 13, 16, 18, 20 were canceled.
Response to Argument
In reference to applicant’s argument regrading rejections under 35 U.S.C. § 101:
Applicant’s Argument:
The applicant’s argument regarding the 101 rejection based on the claim amendment filed on 11/12/2025.
Examiner’s Response:
The applicant’s argument is persuasive; therefore, the 101 rejection is withdrawn in view of the claim amendment filed on 11/12/2025.
In reference to applicant’s argument regrading rejections under 35 U.S.C. § 102:
Applicant’s Argument:
The applicant’s argument regarding the 102 rejection based on the claim amendment filed on 11/12/2025
Examiner’s Response:
This argument includes the newly amended limitations. It has been fully considered but is moot in view of the new grounds of rejection presented below necessitated by the amendment.
In reference to applicant’s argument regrading rejections under 35 U.S.C. § 103:
Applicant’s Argument:
The applicant’s argument regarding the 103 rejection based on the claim amendment filed on 11/12/2025
Examiner’s Response:
This argument includes the newly amended limitations. It has been fully considered but is moot in view of the new grounds of rejection presented below necessitated by the amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5, 8, 12, 15, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (NPL: Safe Reinforcement Learning With Stability Guarantee for Motion Planning of Autonomous Vehicles-hereinafter, Zhang. ) further in view of Ismailsheriff et al. (Patent. No. US 10873533-hereinafter, Ismailsheriff). .
Regarding claim 1, Zhang teaches a computer-implemented method comprising: determining, by the reinforcement learning program, whether a stability of the next reinforcement learning state of a reinforcement learning problem is above a predetermined threshold (Zhang, [Abstract], ““This article is focused on safe motion planning with the stability guarantee for autonomous vehicles with limited size and power. To this end, the risk-identification method and the Lyapunov function are integrated with the well-known soft actor–critic (SAC) algorithm. By borrowing the concept of Lyapunov functions in the control theory, the learned policy can theoretically guarantee that the state trajectory always stays in a safe area.”. and Sec.II Preliminaries], “
PNG
media_image1.png
488
587
media_image1.png
Greyscale
”
Examiner’s note, determine whether the stability at the next state is above threshold d0 or not for example, whether the vehicle is entering into the safety state or not.) ;
wherein the reinforcement learning program is trained to determine stability of reinforcement learning states based on Lyapunov Stability (Zhang, [Introduction], “Motivated by the above observations, in this article, a novel learning-based algorithm called Lyapunov-based SAC with collision probability prediction (LSAC-CPP) algorithm is proposed for the safe motion planning of autonomous vehicles. Based on the SAC that is proven to improve the capability of exploration, the proposed algorithm defines the risk according to the probability of collision for learning the safe policy [32]. Furthermore, this article generalizes the definition of stability in control theory for safe RL and borrows the concept of the Lyapunov function to guarantee the stability of the system. Specifically, a risk-sensitivity parameter (RSP) is introduced to strike a balance between maximizing the return and minimizing the cumulative risk, and a data-driven method based on the generalized Lyapunov function is introduced to analyze
the stability of the system. The main contributions of this article are summarized as
follows: 1) an approach is proposed to predict the probability of collision, which utilizes a sequence of states and actions; 2) a practical learning-based algorithm is designed to train the safe policy for motion planning of autonomous vehicles; and 3) a novel learning-based method is proposed to construct Lyapunov functions in a model-free framework to further ensure the safety of the system” Examiner’s note, the this article generalizes the definition of stability in control theory for safe RL and borrows the concept of the Lyapunov function to guarantee the stability of the system);
and responsive to determining that the stability of the next reinforcement state is below the predetermined threshold (zhang, [Section II, C) :
PNG
media_image2.png
438
522
media_image2.png
Greyscale
”
Examiner’s note. Determining that the stability of the next learning state (vehicle entering the next state) below the threshold. ):
determining, by the reinforcement learning program, a stability of an alternate next reinforcement learning state of the reinforcement learning problem (Zhang, [Sec. II Preliminaries], “
PNG
media_image1.png
488
587
media_image1.png
Greyscale
”
PNG
media_image3.png
528
602
media_image3.png
Greyscale
Examiner’s note, determine whether the stability at the next state is entering into the safety state or not.);
;
and responsive to determining that the stability of the alternate next reinforcement state is above the predetermined threshold (Zhang, Sec. II
PNG
media_image1.png
488
587
media_image1.png
Greyscale
”:
)
transitioning from the current reinforcement learning state to the alternate next reinforcement learning state based (Zhang, [Introduction], ““Furthermore, this article generalizes the definition of stability in control theory for safe RL and borrows the concept of the Lyapunov function to guarantee the stability of the system. Specifically, a risk-sensitivity parameter (RSP) is introduced to strike a balance between maximizing the return and minimizing the cumulative risk, and a data-driven method based on the generalized Lyapunov function is introduced to analyze the stability of the system. The main contributions of this article are summarized as follows: 1) an approach is proposed to predict the probability of collision, which utilizes a sequence of states and actions; 2) a practical learning-based algorithm is designed to train the safe policy for motion planning of autonomous vehicles; and 3) a novel learning-based method is proposed to construct Lyapunov functions in a model-free framework to further ensure the safety of the system.” And [Sec.2 Preliminaries], “
PNG
media_image1.png
488
587
media_image1.png
Greyscale
”
PNG
media_image3.png
528
602
media_image3.png
Greyscale
Examiner’s note, transition to the next state based on the determining if the vehicle is entering into the next safe place, when the min dt above threshold.).
However, Zhang does not teach generating, by a reinforcement learning program of a server, a Q-Table, Q(s,a), wherein (s,a) are state action pairs, determining, by the reinforcement learning program, a current reinforcement learning state of the Q-Table and a next reinforcement learning state of the Q-Table,selecting an action with a highest Q-value for the current reinforcement learning state within the Q-Table, and based on the action with the highest Q-value within the Q-Table; computing a temporal difference between the current reinforcement learning state and the alternate next reinforcement learning state and updating a set of Q-values of the Q-Table, Q(s,a), for the current reinforcement learning state based on an observed reward for transitioning to the alternate next reinforcement learning state and a maximum possible reward for transitioning to the alternate next reinforcement learning state.
On the other hand, Ismailsheriff teaches generating, by a reinforcement learning program of a server, a Q-Table, Q(s,a), wherein (s,a) are state action pairs (Ismailsheriff , [Col.40, table 5, lines 36-55], “In some embodiments, the network devices may rely on one or more STACKing reinforcement learning agents 430 to determine whether to perform STACKing on traffic data flowing through the network devices. For a first time step, each STACKing reinforcement learning agent 430 can randomly select an action for the network object the agent represents (e.g., a traffic class, a network device, a virtual network segment, a flow, etc.) that transitions from the initial state to the next state and receive a reward as a result of the state transition. Each STACKing reinforcement learning agent can update Q-values for the state-action pair with the reward and the expected return: Q.sub.t+1 (s.sub.t, a.sub.t)=r.sub.t+1+γ max.sub.a Q(s.sub.t+1, a) until transitioning to an end state. The process can be repeated iteratively for a first time period until the STACKing reinforcement learning agents 430 determine the optimal state-action function Q*. The optimal policy for each agent can be learned by applying a greedy policy to Q*. Table 5 sets forth an example of pseudo-code for Q-learning.”.
PNG
media_image4.png
200
400
media_image4.png
Greyscale
”
Table 5 shows the Q-value with the state-action pair. ;
determining, by the reinforcement learning program, a current reinforcement learning state of the Q-Table and a next reinforcement learning state of the Q-Table (Ismailsheriff , [Col.40, table 5, lines 36-55], “In some embodiments, the network devices may rely on one or more STACKing reinforcement learning agents 430 to determine whether to perform STACKing on traffic data flowing through the network devices. For a first time step, each STACKing reinforcement learning agent 430 can randomly select an action for the network object the agent represents (e.g., a traffic class, a network device, a virtual network segment, a flow, etc.) that transitions from the initial state to the next state and receive a reward as a result of the state transition. Each STACKing reinforcement learning agent can update Q-values for the state-action pair with the reward and the expected return: Q.sub.t+1 (s.sub.t, a.sub.t)=r.sub.t+1+γ max.sub.a Q(s.sub.t+1, a) until transitioning to an end state. The process can be repeated iteratively for a first time period until the STACKing reinforcement learning agents 430 determine the optimal state-action function Q*. The optimal policy for each agent can be learned by applying a greedy policy to Q*. Table 5 sets forth an example of pseudo-code for Q-learning.”.
PNG
media_image4.png
200
400
media_image4.png
Greyscale
”);
selecting an action with a highest Q-value for the current reinforcement learning state within the Q-Table (Ismailsheriff, [Col16, table 5, lines 49-59], “For example, given a flow, a STACKing reinforcement learning agent can determine whether initiating, continuing, or suspending STACKing for the flow maximizes a total reward (e.g., maximizes transmission throughput) or minimizes a total cost (e.g., minimizes network latency), and select the course of action that maximizes the total reward or minimizes the total cost. Other embodiments may include different numbers and/or types of machine learning models but one of ordinary skill the art will appreciate that such variations do not depart from the scope of the present disclosure.” And [Col.27, lines 58-67], “This reward evaluates the immediate effect of action a.sub.t (i.e., the transition from the state s.sub.t to the state s.sub.t+1) The behavior of the agent can be described by its policy π, which is typically a stochastic function, π: S×A.fwdarw.[0,1]. The goal of the MDP (and other reinforcement learning systems) can be to maximize, at each time step t, the expected discounted return G.sub.t=Σ.sub.k=0.sup.∞γ.sup.kR.sub.t+k+1, where γ∈[0,1) is a discount factor (e.g., a floating point number between 0 and 1 in which future rewards are given greater weight the closer γ is to 1 and less weight the closer γ is to 0). This can involve determining the optimal value function or the value function that returns a higher value than other functions for all states: V*(s)=max.sub.πV.sub.π(s) for all s∈S, or the optimal action-value function: Q*(s, a)=max.sub.πQ.sub.π(s, a) for all s∈S and for all a∈A. That is, the optimal value of taking an action a for a state s is the expected immediate reward plus the expected return (e.g., the expected discounted optimal value attainable from the next state). A greedy policy is deterministic and can pick the action with the highest Q-value for every state: π*(s)=arg max.sub.a Q*(s, a).” Examiner’s note, pick the action with the highest Q-value for every state based on the highest expect return (the optimal value that return value higher than for all states or higher than the threshold).);
and based on the action with the highest Q-value within the Q-Table; computing a temporal difference between the current reinforcement learning state and the alternate next reinforcement learning state (Ismailsheriff, [Col.27, lines 58-67], “This reward evaluates the immediate effect of action a.sub.t (i.e., the transition from the state s.sub.t to the state s.sub.t+1) The behavior of the agent can be described by its policy π, which is typically a stochastic function, π: S×A.fwdarw.[0,1]. The goal of the MDP (and other reinforcement learning systems) can be to maximize, at each time step t, the expected discounted return G.sub.t=Σ.sub.k=0.sup.∞γ.sup.kR.sub.t+k+1, where γ∈[0,1) is a discount factor (e.g., a floating point number between 0 and 1 in which future rewards are given greater weight the closer γ is to 1 and less weight the closer γ is to 0). This can involve determining the optimal value function or the value function that returns a higher value than other functions for all states: V*(s)=max.sub.πV.sub.π(s) for all s∈S, or the optimal action-value function: Q*(s, a)=max.sub.πQ.sub.π(s, a) for all s∈S and for all a∈A. That is, the optimal value of taking an action a for a state s is the expected immediate reward plus the expected return (e.g., the expected discounted optimal value attainable from the next state). A greedy policy is deterministic and can pick the action with the highest Q-value for every state: π*(s)=arg max.sub.a Q*(s, a).” Examiner’s note, pick the action with the highest Q-value for every state based on the highest expect return (the optimal value that return value higher than for all states or higher than the threshold).;
and updating a set of Q-values of the Q-Table, Q(s,a), for the current reinforcement learning state based on an observed reward for transitioning to the alternate next reinforcement learning state and a maximum possible reward for transitioning to the alternate next reinforcement learning state (Ismailsheriff , [Col.40, table 5, lines 36-55], “In some embodiments, the network devices may rely on one or more STACKing reinforcement learning agents 430 to determine whether to perform STACKing on traffic data flowing through the network devices. For a first time step, each STACKing reinforcement learning agent 430 can randomly select an action for the network object the agent represents (e.g., a traffic class, a network device, a virtual network segment, a flow, etc.) that transitions from the initial state to the next state and receive a reward as a result of the state transition. Each STACKing reinforcement learning agent can update Q-values for the state-action pair with the reward and the expected return: Q.sub.t+1 (s.sub.t, a.sub.t)=r.sub.t+1+γ max.sub.a Q(s.sub.t+1, a) until transitioning to an end state. The process can be repeated iteratively for a first time period until the STACKing reinforcement learning agents 430 determine the optimal state-action function Q*. The optimal policy for each agent can be learned by applying a greedy policy to Q*. Table 5 sets forth an example of pseudo-code for Q-learning.”.
Zhang and Ismailsheriff are analogous in arts because they have the same field of endeavor of generating the plurality of reinforcement learning state.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to modify the rand responsive to determining that the stability of the next reinforcement state is below the predetermined threshold, determining, by the reinforcement learning program, a stability of an alternate next reinforcement learning state of the reinforcement learning problem and responsive to determining that the stability of the alternate next reinforcement state is above the predetermined threshold, transitioning from the current reinforcement learning state to the alternate next reinforcement learning state based, on determining that the stability of the next reinforcement learning state is above the predetermined threshold, as taught by Zhang, to include the generating, by a reinforcement learning program of a server, a Q-Table, Q(s,a), wherein (s,a) are state action pairs, determining, by the reinforcement learning program, a current reinforcement learning state of the Q-Table and a next reinforcement learning state of the Q-Table,selecting an action with a highest Q-value for the current reinforcement learning state within the Q-Table, and based on the action with the highest Q-value within the Q-Table; computing a temporal difference between the current reinforcement learning state and the alternate next reinforcement learning state and updating a set of Q-values of the Q-Table, Q(s,a), for the current reinforcement learning state based on an observed reward for transitioning to the alternate next reinforcement learning state and a maximum possible reward for transitioning to the alternate next reinforcement learning state., as taught by Ismailsheriff. The modification would have been obvious because one of the ordinary skills in art would be motivated to minimize the cost and maximize the total reward (Ismailsheriff, [Col.52, lines 39-49], “As discussed above with respect to FIG. 4 and elsewhere in the present disclosure, the STACKing reinforcement learning agents 430 can comprise one or more reinforcement learning agents or other machine learning models that can receive input traffic data and output a decision whether to initiate, continue, or suspend STACKing for the traffic data to maximize a total reward (e.g., maximize transmission throughput) or minimize a total cost (e.g., minimize network latency).)” ,
Regarding claim 5, Zhang as modified in view of Ismailsheriff teaches the computer-implemented method of claim 4, wherein respective Q-value within the Q-table computed using a greedy search algorithm (Ismailsheriff , [Col. 27, lines 24-43], “Various approaches can be used to learn or approximate the optimal policy, and a common approach can involve determining or approximating optimal value functions, including the optimal state-value function (e.g., a function that assigns to each state the largest expected return or total amount of reward accumulated over the future, starting from that state) and the optimal action-value function or optimal Q-value function (e.g., a function that assigns to each state-action pair the largest expected return, or total amount of reward accumulated over the future, for a given state and a given action). An optimal policy can be derived using a greedy policy that selects actions having the highest Q-value for each state. An example algorithm for determining the optimal Q-value function is Q-learning, which can involve iteratively updating Q-values for each state-action pair (e.g., taking an action, receiving a reward and state transition, updating Q-values with the reward and largest expected return, and repeating until transitioning to an end state) for each time step over a time period until convergence with the optimal Q-value function.” And
PNG
media_image4.png
200
400
media_image4.png
Greyscale
).
Zhang and Ismailsheriff are analogous in arts because they have the same field of endeavor of generating the plurality of reinforcement learning state.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to modify the reinforcement learning state, as taught by Zhang, to include the wherein a Q-value of the action is computed using a greedy search algorithm, as taught by Ismailsheriff. The modification would have been obvious because one of the ordinary skills in art would be motivated to improve the network performance (Ismailsheriff, [Col.2, lines 39-49], “Systems and methods provide for generating traffic class-specific congestion signatures and other machine learning models for improving network performance. In some embodiments, a network controller can receive historical traffic data captured by a plurality of network devices within a first period of time that the plurality of network devices apply one or more traffic shaping policies for one or more traffic classes (including at least one predetermined traffic class) and one or more congestion states (including at least one predetermined congestion state).)” ,
The claim 8 is rejected for the same reason as the claim 1, since these claims recite the same limitations.
The claim 12 is rejected for the same reason as the claim 5, since these claims recite the same limitations.
The claim 15 is rejected for the same reason as the claim 1, since these claims recite the same limitations.
The claim 19 is rejected for the same reason as the claim 5, since these claims recite the same limitations.
Claims 3, 7, 10, 14, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (NPL: Safe Reinforcement Learning With Stability Guarantee for Motion Planning of Autonomous Vehicles-hereinafter, Zhang. ) further in view of Ismailsheriff et al. (Patent. No. US 10873533-hereinafter, Ismailsheriff) and further in view of Han et al. (PUB. No. US 20210166158 -hereinafter, Han).
Regarding claim 3, Zhang teaches the computer-implemented method of claim 1, However, Zhang teach the metric of Lyapunov Stability (Zhang, [Introduction], “Motivated by the above observations, in this article, a novel learning-based algorithm called Lyapunov-based SAC with collision probability prediction (LSAC-CPP) algorithm is proposed for the safe motion planning of autonomous vehicles. Based on the SAC that is proven to improve the capability of exploration, the proposed algorithm defines the risk according to the probability of collision for learning the safe policy [32]. Furthermore, this article generalizes the definition of stability in control theory for safe RL and borrows the concept of the Lyapunov function to guarantee the stability of the system. Specifically, a risk-sensitivity parameter (RSP) is introduced to strike a balance between maximizing the return and minimizing the cumulative risk, and a data-driven method based on the generalized Lyapunov function is introduced to analyze the stability of the system. The main contributions of this article are summarized as follows: 1) an approach is proposed to predict the probability of collision, which utilizes a sequence of states and actions; 2) a practical learning-based algorithm is designed to train the safe policy for motion planning of autonomous vehicles; and 3) a novel learning-based method is proposed to construct Lyapunov functions in a model-free framework to further ensure the safety of the system.”),
However, Zhang does not teach wherein the reinforcement learning program is further trained to determine that respective reinforcement learning states are stable if all eigenvalues of a metric are within a unit circle and trained to determine that respective reinforcement learning states_are unstable if one or more eigenvalues of the metric are outside of the unit circle
on the other hand, Han teaches wherein the reinforcement learning program is further trained to determine that respective reinforcement learning states are stable if all eigenvalues of a metric are within a unit circle and trained to determine that respective reinforcement learning states_are unstable if one or more eigenvalues of the metric are outside of the unit circle (Han, [Par.0049], “In addition, the reward is that in case that the next state information, which is the result of control according to the control information, is generated according to a preset threshold range, and is within the threshold range, it is determined that the device 300 has properly operated, and a compensation value (e.g., +1) set in advance is applied to the next state information, and if it is out of the threshold range, it is determined that the device 300 has not properly operated and a preset compensation value (e.g. −1) is applied to the next state information.” And [Par.0131-0132], “ Wherein, the device state information monitoring unit 260 performs monitoring the device state information, and decides whether the transition is occurred or not according to the preset threshold range when transitioning from the current state information of the device 300 to the next state information according to the control information, so that whether the reinforcement learning is re-performed or not should be determined…[0132] That is, the device state information monitoring unit 260 makes the federated reinforcement learning re-performed when the next state information exceeds the threshold range, and adapts to aging or changes in surrounding environment according to the use of the device 300, and thus the device 300 can be continuously and precisely controlled.” Examiner’s note, the state of the reinforcement learning has properly operated when the state information is whining the threshold, that is corresponding to the reinforcement learning state is stable, and when the state of the reinforcement learning has not properly operated when the state information is exceeding the threshold, that is corresponding to the reinforcement learning state is unstable .),
Zhang, Ismailsheriff and Han are analogous in arts because they have the same field of endeavor of generating the plurality of reinforcement learning state.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to modify the reinforcement learning state, as taught by Zhang, to include wherein the reinforcement learning state is stable if all eigenvalues of A are within a unit circle and the reinforcement learning state is unstable if one or more eigenvalues of A are outside of the unit circle, as taught by Han. The modification would have been obvious because one of the ordinary skills in art would be motivated to enable the efficient and precise control of the device, (Han, [Par.0017], “The present disclosure was devised so as to resolve the problems of the state of the art mentioned above, an object of the present disclosure is to provide a system for controlling multiple devices and a method for controlling the device through a federated reinforcement learning that enables efficient and precise control of the device by generating each learning model capable of automatically controlling each of the devices through reinforcement learning in each of device controllers for a plurality of devices having similar or same characteristics and purposes.”).
Regarding claim 7, Zhang as modified in view of Han teaches the computer-implemented method of claim 6, wherein the observed reward for transitioning to the next reinforcement learning state is calculated based on an immediate reward for transitioning to the alternate next reinforcement learning state (Han, [par.0097-0101], “.n addition, the reward generation unit 130 performs a function of generating a reward for the next state information of the state of the device 300 that is transferred based on the control information.[0098] The reward is generated by calculating the reward for the next state information based on a preset threshold range based on the received next state information on the device 300.[0099] In addition, the reward is generated as a positive compensation value previously set in case that the next state information of the received device 300 is operated within the threshold range, and the reward is differentially generated as a negative compensation value set in advance according to the proximity of the threshold range in case that the reward is operated outside the threshold range. [0101] …, and the reward created according to the next state information.”).
Zhang, Ismailsheriff and Han are analogous in arts because they have the same field of endeavor of generating the plurality of reinforcement learning state.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to modify the determining whether a stability of a next reinforcement learning state of a reinforcement learning, as taught by Zhang, to include the observed reward for transitioning to the next reinforcement learning state is calculated based, at least in part, on an immediate reward for transitioning to the next reinforcement learning state and a value associated with the next reinforcement learning state, as taught by Han. The modification would have been obvious because one of the ordinary skills in art would be motivated to enable the efficient and precise control of the device, (Han, [Par.0017], “The present disclosure was devised so as to resolve the problems of the state of the art mentioned above, an object of the present disclosure is to provide a system for controlling multiple devices and a method for controlling the device through a federated reinforcement learning that enables efficient and precise control of the device by generating each learning model capable of automatically controlling each of the devices through reinforcement learning in each of device controllers for a plurality of devices having similar or same characteristics and purposes.”).
The claim 10 is rejected for the same reason as the claim 3, since these claims recite the same limitations.
The claim 14 is rejected for the same reason as the claim 7, since these claims recite the same limitations.
The claim 17 is rejected for the same reason as the claim 3, since these claims recite the same limitations.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure is provide below.
CAMILO GAMBOA HIGUERA et al. (Pub. No.:us 20210097386-hereinafter, CAMILO GAMBOA HIGUERA) teaches transferring learning of the reinforcement learning of plurality state.
Manek et al. (Pub. No.:us 20210042457-hereinafter, Manek) teaches reinforcement learning based on the A lyapunov.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EM N TRIEU whose telephone number is (571)272-5747. The examiner can normally be reached on Mon-Fri from 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached on (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.T./ Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/ Supervisory Patent Examiner, Art Unit 2128