DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 02/11/2026 has been entered. Claims 1, 5, 8-11, 14-15, 17-18 and 20 have been amended. Claims 21-25 have been added. Claims 4, 6, 7, 16 and 19 have been canceled. Claims 1-3, 5, 8-15, 17-18, and 20-25 remain pending in the application.
Response to Arguments
Claim Rejections - 35 USC § 103
Regarding the newly amended independent claim 1, Applicant submits that the cited references fail to disclose at least the newly-added features of the independent claims.
In response, Examiner relies on a new combination of references.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 8-10, 14-15, 17-18, 20-22 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over CHANDAVARKAR (US 2025/0165365 Al) in view of (Siyadatzadeh , “ReLIEF: A Reinforcement-Learning-Based Real-Time Task Assignment Strategy in Emerging Fault-Tolerant Fog Computing”, hereinafter “Siyadatzadeh”) in view of Liu (US 11,507,471 B2) in view of TORIGOE (US 2022/0391661 Al) in further view of Nara (US 2024/0103987 Al)
Regarding claim 1, CHANDAVARKAR discloses: An apparatus comprising: at least one processing device comprising a processor coupled to a memory; the at least one processing device being configured: to identify a plurality of backup operations to be performed in a backup infrastructure environment, (CHANDAVARKAR [0053] different priorities can also be assigned to the various replication and snapshot operations (corresponding to “a plurality of backup operations”) to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed. For example, the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation; )
the backup infrastructure environment comprising two or more backup servers;
(CHANDAVARKAR, [0137] Each node can be associated with one server, multiple servers, or portions of a server. The nodes can be associated (e.g., logically and/or physically) with the clusters; [0018] Disaster recovery services constitute a critical component of modem information technology infrastructure,… A disaster recovery service is a dynamic amalgamation of methodologies, ranging from sophisticated backup routines to real-time data replication mechanisms and automated failover procedures; [0020] The disaster recovery preparation service… includes creating backups of critical data and systems, which are stored in the storage system.)
to generate a first data structure characterizing a prioritization of at least a subset of the plurality of backup operations; (CHANDAVARKAR, [0053] different priorities can also be assigned to the various replication and snapshot operations to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed. For example, the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation; [0120] Some embodiments of a module include instructions that are stored in a memory for execution so as to facilitate operational and/or performance characteristics pertaining to management of block stores. Various implementations of the data repository comprise storage media organized to hold a series of records and/or data structures.)
to generate a second data structure characterizing status of the two or more backup servers in the backup infrastructure environment;
(CHANDAVARKAR, [0152] replication status; …; hardware health and status; network connectivity status; error and event logs, snapshot status;…; alerts and notifications; configuration details; and historical statistics; [0075] CPU Usage: CPU usage indicates the percentage of a capacity of a processor that is currently in use. Monitoring CPU usage helps ensure that the system has enough processing power to handle the replication tasks without being overburdened; [0137] Each node can be associated with one server, multiple servers, or portions of a server. The nodes can be associated (e.g., logically and/or physically) with the clusters;)
to determine, utilizing at least one machine learning model that is implemented by the at least one processing device and that takes as input at least a portion of the first data structure and at least a portion of the second data structure, an execution schedule for the subset of the plurality of backup operations;
(CHANDAVARKAR, [0006], e.g. applying a machine learning model to determine a change to one or more of a plurality of adjustable parameters of the disaster recovery preparation system based on the state of the storage system and the recovery point objective that is not met;… a priority assigned to the disaster recovery preparation system; or a number of snapshots that can be concurrently performed and replicated by the disaster recovery preparation system; [0053] different priorities can also be assigned to the various replication and snapshot operations (corresponding to “a plurality of backup operations”) to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed…the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation)
and to execute the subset of the plurality of backup operations in the backup infrastructure environment in accordance with the determined execution schedule; (CHANDAVARKAR [0053] different priorities can also be assigned to the various replication and snapshot operations to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed. For example, the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation; [0008] parameters for a storage system can be tuned rapidly and efficiently by using an automated tuner that monitors the storage system's performance)
wherein the at least one machine learning model comprises a multi-agent reinforcement learning model comprising: (CHANDAVARKAR, Fig. 1, Machine Learning Agents 110A, 110 B,… 110 N; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions; [0030] an automated tuner 120 that includes one or more machine learning agents 110 can be used to tune the parameters; [0059], e.g. allocate each agent 110 exclusively to fine-tune a specific parameter from the discussed set.)
a first agent configured to take as input at least a portion of the first data structure (CHANDAVARKAR, [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions; [0030] an automated tuner 120 that includes one or more machine learning agents 110 can be used to tune the parameters; [0059], e.g. allocate each agent 110 exclusively to fine-tune a specific parameter from the discussed set; [0051] Modifying a Time of Replication: Adjusting the replication schedule affects when data is synchronized between primary and secondary systems. The time of replication can be set to real-time or at scheduled intervals.)
and a second agent configured to take as input at least a portion of the second data structure (CHANDAVARKAR, [0006], e.g. applying a machine learning model to determine a change to one or more of a plurality of adjustable parameters of the disaster recovery preparation system based on the state of the storage system and the recovery point objective that is not met;… a priority assigned to the disaster recovery preparation system; or a number of snapshots that can be concurrently performed and replicated by the disaster recovery preparation system; [0053] different priorities can also be assigned to the various replication and snapshot operations (corresponding to “a plurality of backup operations”) to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed…the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation)
However CHANDAVARKAR does not clearly disclose:
to generate a first data structure, to generate a second data structure; and to determine, for first time intervals, a first action associated with allocation of respective ones of the plurality of backup operations to at least one of the two or more backup servers that currently have an active status; and to determine, for second time intervals, a second action associated with setting the active status of respective ones of the two or more backup servers in the backup infrastructure environment, a length of at least one of the second time intervals being greater than a length of at least one of the first time intervals.
However Siyadatzadeh discloses:
to generate a first data structure, (Siyadatzadeh , “ReLIEF: A Reinforcement-Learning-Based Real-Time Task Assignment Strategy in Emerging Fault-Tolerant Fog Computing”, Page 4, right column, las paragraph- Each fog node has two queues for tasks to be processed. Primary tasks are stored in the primary queue of fog nodes, and backup tasks are stored in the backup queue (corresponding to “a data structure”) of fog nodes. Queued tasks are executed in accordance with the task’s deadline. Tasks in the backup queue have a higher priority)
and to determine, a first action associated with allocation of respective ones of the plurality of backup operations to at least one of the two or more backup servers (Siyadatzadeh , page 7, left column- 2) Action-Space: There are only a limited number of distinct actions available for the model to search in. Actions are denoted by A = {a1, a2, . . . , a|A|}, where each ai represents the id of two fog nodes. One of them is the id of a fog node that is responsible for the execution of the primary task, and the second one is the id of a fog node that is responsible for the execution of the backup task.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR with the teaching of Siyadatzade to offload tasks in a manner that improves reliability, (Siyadatzadeh, page 2, left column, last paragraph), and also to improve workload distribution and decrease the number of tasks in the queues. Therefore, the delay reduces, and more tasks could be completed before the deadline,( (Siyadatzadeh, page 3, right column)
However CHANDAVARKAR in view of Siyadatzadeh does not clearly disclose:
to generate a second data structure; to determine, for first time intervals, a first action associated with allocation of respective ones of the plurality of backup operations to at least one of the two or more backup servers that currently have an active status; and to determine, for second time intervals, a second action associated with setting the active status of respective ones of the two or more backup servers in the backup infrastructure environment, a length of at least one of the second time intervals being greater than a length of at least one of the first time intervals.
However Liu discloses:
to generate a second data structure
(Liu, column 5, line 13- the state 101 of the backup system 120 may indicate storage usage at each backup server 121 and idle time of each backup server 121 within a historical time period (for example, within the past 24 hours), average time and a success rate of one or more data backups performed by each backup client 122 within the historical time period (for example, within the past 24 hours)…column 5, line 60- the configuration information 103 may be represented by a 60 1 *N array, where N represents the number of backup clients and each element in the array indicates that a target backup server to which each backup client is allocated.)
to determine, for first time intervals, a first action associated with allocation of respective ones of the plurality of backup operations to at least one of the two or more backup servers that currently have an active status; (Liu, column 4, line 24, The plurality of backup servers 121… where each backup server 121 is configured to back up data ;column 5, line 6- The reward score 102, for example, may be a reward of the last allocation action (i.e., allocating the plurality of backup clients 122 to the plurality of backup servers 121) performed against the backup system 120, which may be derived from the state 101 of the backup 10 system 120 …line 13- the state 101 of the backup system 120 may indicate storage usage at each backup server 121 and idle time of each backup server 121 within a historical time period (for example, within the past 24 hours), average time and a success rate of one or more data backups performed by each backup client 122 within the historical time period (for example, within the past 24 hours)… the backup manager 110 may determine the reward score 102, based on the storage usage at each backup server 121, the idle time of each backup server 121 within the historical time period, the average time and the success rate of the one or more data backups performed by each backup client 122, for example, as shown by the formula (1) )
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade with the teaching of Liu to improve stability and convergence of the ActorCritic network, making it more suitable for handling the management of a backup system, (Liu, column 6, line 12) and also to ensure the efficiency of data backup and improve the utilization of the backup server, (Liu, column 3, line 22).
However CHANDAVARKAR in view of Siyadatzade in view of Liu does not clearly disclose:
and to determine, for second time intervals, a second action associated with setting the active status of respective ones of the two or more backup servers in the backup infrastructure environment, a length of at least one of the second time intervals being greater than a length of at least one of the first time intervals.
However TORIGOE discloses:
and to determine, for second time intervals, a length of at least one of the second time intervals being greater than a length of at least one of the first time intervals. (TORIGOE, [0089], The first agent simulator 200-X performs a simulation of a first agent (subject agent) 4X at a first processing time interval TX. On the other hand, the second agent simulator 200-Y performs a simulation of a second agent (subject agent) 4Y at a second processing time interval TY. The second processing time interval TY is longer than the first processing time interval TX.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu with the teaching of TORIGOE to suppress unnecessary increase in the traffic , (TORIGOE, [0089]) and also to reduce computation load, (TORIGOE, [0113]). Therefore, the unnecessary increase in the traffic is suppressed, and thus the unnecessary consumption of the communication resources is suppressed, (TORIGOE, [0123]).
However CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE does not clearly disclose:
a second action associated with setting the active status of respective ones of the two or more backup servers in the backup infrastructure environment
However Nara discloses:
and to determine, for second time intervals, a second action associated with setting the active status of respective ones of the two or more backup servers in the backup infrastructure environment (Nara [0318] changes its status from passive node to active node; [0268] When a failover event is detected, the selected standby passive node initiates a failover process; [0314] At block 706, the routine initiates a failover routine ( e.g., the failover application 444) at the standby passive node that has been selected to become the next active storage managing node; [0315] active node configuration data 306; [0307] monitoring accessibility of the standby nodes and upgrades to software can occur at different time intervals)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE with the teaching of Nara in order to greatly improve the speed with which system performs information management operations and can also improve the capacity of the system to handle large numbers of such operations, while reducing the computational load, (Nara, [0079]) and also to provide scalable systems capable of dynamic storage operations, load balancing, and failover, (Nara, [0132]).
Claims 15 and 18 correspond to claim 1, are rejected accordingly.
Regarding claim 2, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 1 as outlined above. Claim 2 further recites: wherein the backup infrastructure environment further comprises backup storage infrastructure, the two or more backup servers being configured to store data to be backed up on the backup storage infrastructure. (CHANDAVARKAR , [0020] This often includes creating backups of critical data and systems, which are stored in the storage system; [0137] virtualization system architecture SD00 includes a distributed virtualization system that includes multiple clusters (e.g., cluster 5831 , ... , cluster 583N) comprising multiple nodes that have multiple tiers of storage in a storage pool. Representative nodes (e.g., node 581u, ... , node 5811M) and storage pool 590 associated with cluster 5831 are shown. Each node can be associated with one server, multiple servers, or portions of a server. The nodes can be associated (e.g., logically and/or physically) with the clusters;)
Regarding claim 3, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 1 as outlined above. CHANDAVARKAR does not clearly disclose: wherein generating the first data structure comprises, for a given backup operation in the subset of the plurality of backup operations, determining a priority based at least in part on (i) a predicted execution time of the given backup operation and (ii) a waiting time of the given backup operation.
However Siyadatzade discloses:
wherein generating the first data structure comprises, for a given backup operation in the subset of the plurality of backup operations, determining a priority based at least in part on (i) a predicted execution time of the given backup operation and (ii) a waiting time of the given backup operation. (Siyadatzadeh ,Page 5, right column, first paragraph- 3) Executing Time: The time for executing the task τi on a specific fog node fj is expressed as
PNG
media_image1.png
200
400
media_image1.png
Greyscale
4) Queuing Delay:
PNG
media_image2.png
200
400
media_image2.png
Greyscale
indicates how long a task must wait in the queue before being processed by a fog node. This delay is calculated as the sum of the processing times of tasks in the primary and backup queues that have a deadline earlier than task τi. As a result, total delay D for task τi can be calculated as follows:
PNG
media_image3.png
200
400
media_image3.png
Greyscale
Page 7, right column, After determining the state of the system, the best policy must be chosen,…, the broker waits for the results of the task τi…if the result is not received prior to a certain time, a backup is sent to the specified node (lines 11–15). That specific moment can be estimated using the formula in line 11.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR with the teaching of Siyadatzade to offload tasks in a manner that improves reliability, (Siyadatzadeh, page 2, left column, last paragraph), and also to improve workload distribution and decrease the number of tasks in the queues. Therefore, the delay reduces, and more tasks could be completed before the deadline,( (Siyadatzadeh, page 3, right column)
Regarding claim 5, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 1 as outlined above. Claim 5 further recites: wherein at least one of the first agent and the second agent of the multi-agent reinforcement learning model implements an actor-critic deep reinforcement learning algorithm. (CHANDAVARKAR, [0027] Example reinforcement learning training algorithms include Monte Carlo, Q-learning, state-action-reward-state-action (SARSA), Q-learning lambda, SARSA lambda, deep Q network, deep deterministic policy gradient, asynchronous advantage actor-critic, Q-learning with normalized advantage functions, trust region policy optimization, proximal policy optimization, twin delayed deep deterministic policy gradient, soft actor-critic, and/or the like; Fig. 1, Machine Learning Agents 110A, 110 B,… 110 N; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence)
Regarding claim 8, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 1 as outlined above. CHANDAVARKAR in view of Siyadatzade in view of Liu does not clearly disclose: wherein a length of a given one of the second time intervals is a designated multiple of a length of a given one of the first time intervals.
However TORIGOE discloses:
wherein a length of a given one of the second time intervals is a designated multiple of a length of a given one of the first time intervals. (TORIGOE, [0089], The first agent simulator 200-X performs a simulation of a first agent (subject agent) 4X at a first processing time interval TX. On the other hand, the second agent simulator 200-Y performs a simulation of a second agent (subject agent) 4Y at a second processing time interval TY. The second processing time interval TY is longer than the first processing time interval TX.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu with the teaching of TORIGOE to suppress unnecessary increase in the traffic , (TORIGOE, [0089]) and also to reduce computation load, (TORIGOE, [0113]). Therefore, the unnecessary increase in the traffic is suppressed, and thus the unnecessary consumption of the communication resources is suppressed, (TORIGOE, [0123]).
Regarding claim 9, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 1 as outlined above. Claim 9 further recites: the first agent of the multi-agent reinforcement learning model is associated with a first action space, a first state space and a first reward function; (CHANDAVARKAR, FIG. 1, Machine Learning Agent 110A, Action space module 112, State space module 112, Reward function module 116; [0023] a system 100 for dynamically tuning the parameters associated with a storage system using reinforcement learning techniques, according to various embodiment …The automated tuner 120 includes, without limitation, one or more machine learning agents 110 (e.g., machine learning agent 110A, 110B ... 110N) and a weighting function module 122.…Each machine learning agent 110 includes, without limitation, an action space module 112, a state space module 114, a reward function module 116, and a transition probabilities module 118. )
and the second agent of the multi-agent reinforcement learning model is associated with a second action space, a second state space and a second reward function. (CHANDAVARKAR, FIG. 1, Machine Learning Agent 110B; [0023] a system 100 for dynamically tuning the parameters associated with a storage system using reinforcement learning techniques, according to various embodiment …The automated tuner 120 includes, without limitation, one or more machine learning agents 110 (e.g., machine learning agent 110A, 110B ... 110N) and a weighting function module 122.…Each machine learning agent 110 includes, without limitation, an action space module 112, a state space module 114, a reward function module 116, and a transition probabilities module 118.)
Claim 21 corresponds to claim 9, is rejected accordingly.
Regarding claim 10, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 9 as outlined above. Claim 10 further recites: wherein: the first action space associated with the first agent of the multi-agent reinforcement learning model (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions.)
and the second action space associated with the second agent of the multi-agent reinforcement learning model (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions.)
However CHANDAVARKAR does not clearly disclose:
characterizes whether respective ones of the plurality of backup operations are allocated to one of the two or more backup servers for execution; characterizes whether respective ones of the two or more backup servers in the backup infrastructure environment have the active status.
However Siyadatzadeh discloses:
characterizes whether respective ones of the plurality of backup operations are allocated to one of the two or more backup servers for execution; (Siyadatzadeh , page 7, left column- 2) Action-Space: There are only a limited number of distinct actions available for the model to search in. Actions are denoted by A = {a1, a2, . . . , a|A|}, where each ai represents the id of two fog nodes. One of them is the id of a fog node that is responsible for the execution of the primary task, and the second one is the id of a fog node that is responsible for the execution of the backup task.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR with the teaching of Siyadatzade to offload tasks in a manner that improves reliability, (Siyadatzadeh, page 2, left column, last paragraph), and also to improve workload distribution and decrease the number of tasks in the queues. Therefore, the delay reduces, and more tasks could be completed before the deadline,( (Siyadatzadeh, page 3, right column).
However CHANDAVARKAR in view of Siyadatzadeh does not clearly disclose: characterizes whether respective ones of the two or more backup servers in the backup infrastructure environment have the active status.
However Liu discloses:
characterizes whether respective ones of the two or more backup servers in the backup infrastructure environment have the active status. (Liu, column 5, line 13- the state 101 of the backup system 120 may indicate storage usage at each backup server 121 and idle time of each backup server 121 within a historical time period (for example, within the past 24 hours), average time and a success rate of one or more data backups performed by each backup client 122 within the historical time period (for example, within the past 24 hours)… the backup manager 110 may determine the reward score 102, based on the storage usage at each backup server 121, the idle time of each backup server 121 within the historical time period, the average time and the success rate of the one or more data backups performed by each backup client 122, for example, as shown by the formula (1) )
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade with the teaching of Liu to improve stability and convergence of the ActorCritic network, making it more suitable for handling the management of a backup system, (Liu, column 6, line 12) and also to ensure the efficiency of data backup and improve the utilization of the backup server, (Liu, column 3, line 22).
Claim 22 corresponds to claim 10, is rejected accordingly.
Regarding claim 14, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 9 as outlined above. Claim 14 further recites: the first reward function associated with the first agent of the multi-agent reinforcement learning model is based at least in part on a first weighted sum of average priority of the plurality of backup operations in a task scheduling time slot and a proportion of the two or more backup servers that are active in the task scheduling time slot; (CHANDAVARKAR [0029] the reinforcement learning training process uses a Q-learning approach, in which selecting an action within the current state pursues the target objective. In one example, the training process evaluates different actions that are predicted to maximize the reward function (or minimize the cost function) based on the current state (“exploitation”) and some combinations of the weights that could produce further maximization of the reward function (or further minimization of the cost function) from the current state (“exploration”); [0079] The reward function module 116 of each machine learning agent 110 combines the performance metrics 150 and the current state 160 to determine how much reward or penalty to assign to the machine learning agent 110. [0060] the influence of a particular agent 110 can be increased or decreased to align with the current priorities. In some embodiments, the weighting function module 122 adapt dynamically to changing conditions; [0060] A weighted function, for example, is a mathematical algorithm that assigns relative importance or influence to different inputs or factors in a decision-making process…This process creates a weighted sum that reflects the combined influence of all agents 110 [0150] a priority assigned to the disaster recovery preparation system; or a number of snapshots that can be concurrently performed and replicated by the disaster recovery preparation system; [0051], The time of replication can be set to real-time or at scheduled intervals; [0075] CPU Usage: CPU usage indicates the percentage of a capacity of a processor that is currently in use. Monitoring CPU usage helps ensure that the system has enough processing power to handle the replication tasks without being overburdened.)
and the second reward function associated with the second agent of the multi-agent reinforcement learning model is based at least in part on a second weighted sum of average priority of the plurality of backup operations in a resource optimization time slot and a proportion of the two or more backup servers that are active in the resource optimization time slot, the resource optimization time slot comprising two or more instances of the task scheduling time slot. (CHANDAVARKAR 0029] the reinforcement learning training process uses a Q-learning approach, in which selecting an action within the current state pursues the target objective. In one example, the training process evaluates different actions that are predicted to maximize the reward function (or minimize the cost function) based on the current state (“exploitation”) and some combinations of the weights that could produce further maximization of the reward function (or further minimization of the cost function) from the current state (“exploration”); [0079] The reward function module 116 of each machine learning agent 110 combines the performance metrics 150 and the current state 160 to determine how much reward or penalty to assign to the machine learning agent 110. [0060] the influence of a particular agent 110 can be increased or decreased to align with the current priorities. In some embodiments, the weighting function module 122 adapt dynamically to changing conditions; [0060] A weighted function, for example, is a mathematical algorithm that assigns relative importance or influence to different inputs or factors in a decision-making process…This process creates a weighted sum that reflects the combined influence of all agents 110; [0059] if one agent is tasked with optimizing replication frequency;)
Claim 25 corresponds to claim 14, is rejected accordingly.
Regarding claim 17, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 15 as outlined above. CHANDAVARKAR in view of Siyadatzade in view of Liu does not clearly disclose: wherein a length of a given one of the second time intervals is a designated multiple of a length of a given one of the first time intervals.
However TORIGOE discloses:
wherein a length of a given one of the second time intervals is a designated multiple of a length of a given one of the first time intervals.
(TORIGOE, [0089], The first agent simulator 200-X performs a simulation of a first agent (subject agent) 4X at a first processing time interval TX. On the other hand, the second agent simulator 200-Y performs a simulation of a second agent (subject agent) 4Y at a second processing time interval TY. The second processing time interval TY is longer than the first processing time interval TX.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu with the teaching of TORIGOE to suppress unnecessary increase in the traffic , (TORIGOE, [0089]) and also to reduce computation load, (TORIGOE, [0113]). Therefore, the unnecessary increase in the traffic is suppressed, and thus the unnecessary consumption of the communication resources is suppressed, (TORIGOE, [0123]).
Claim 20 corresponds to claim 17, is rejected accordingly.
Claims 11-13 and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over CHANDAVARKAR (US 2025/0165365 Al) in view of (Siyadatzadeh , “ReLIEF: A Reinforcement-Learning-Based Real-Time Task Assignment Strategy in Emerging Fault-Tolerant Fog Computing”, hereinafter “Siyadatzadeh”) in view of Liu (US 11,507,471 B2) in view of TORIGOE (US 2022/0391661 Al) in view of Nara (US 2024/0103987 Al) in view of (Rafiliu, Stability Conditions of On-line Resource Managers for Systems with Execution Time Variations, hereinafter “Rafiliu”)
Regarding claim 11, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara discloses all of the features with respect to claim 9 as outlined above. Claim 11 further recites: the first state space associated with the first agent of the multi-agent reinforcement learning model (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions.)
and the second state space associated with the second agent of the multi-agent reinforcement learning model (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0148] The techniques use a reinforcement learning model, where one or more machine learning agents adjust various parameters based on the state of the storage system and RPO adherence. This adaptability ensures that the system remains finely tuned to meet specific workload demands and changing conditions.)
However CHANDAVARKAR does not clearly disclose: characterizes execution times for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment; characterizes a sum of (i) the execution times for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment and (ii) execution times for ones of the plurality of backup operations that are not allocated to one of the two or more backup servers for execution in the backup infrastructure environment.
However Siyadatzadeh discloses:
characterizes execution times for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment; (Siyadatzadeh, page 5, right column, first paragraph- 3) Executing Time: The time for executing the task τi on a specific fog node fj is expressed as
PNG
media_image1.png
200
400
media_image1.png
Greyscale
4) Queuing Delay:
PNG
media_image2.png
200
400
media_image2.png
Greyscale
indicates how long a task must wait in the queue before being processed by a fog node. This delay is calculated as the sum of the processing times of tasks in the primary and backup queues that have a deadline earlier than task τi. As a result, total delay D for task τi can be calculated as follows:
PNG
media_image3.png
200
400
media_image3.png
Greyscale
Page 7, right column, After determining the state of the system, the best policy must be chosen,…, the broker waits for the results of the task τi…if the result is not received prior to a certain time, a backup is sent to the specified node (lines 11–15). That specific moment can be estimated using the formula in line 11.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR with the teaching of Siyadatzade to offload tasks in a manner that improves reliability, (Siyadatzadeh, page 2, left column, last paragraph), and also to improve workload distribution and decrease the number of tasks in the queues. Therefore, the delay reduces, and more tasks could be completed before the deadline,( (Siyadatzadeh, page 3, right column).
However CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara does not clearly disclose: characterizes a sum of (i) the execution times for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment and (ii) execution times for ones of the plurality of backup operations that are not allocated to one of the two or more backup servers for execution in the backup infrastructure environment.
However Rafiliu discloses:
characterizes a sum of (i) the execution times for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment and (ii) execution times for ones of the plurality of backup operations that are not allocated to one of the two or more backup servers for execution in the backup infrastructure environment. (Rafiliu, page 4, left clumn- B. Model We will start to model our systems from the definition of resource demand (Section II-C). By disregarding task offsets, the formula is:
PNG
media_image4.png
200
400
media_image4.png
Greyscale
The first sum represents the accumulation of execution times from previously released, but not executed (queued up) jobs. The second sum represents the accumulation of execution times from jobs that are released during [t[k−1], t[k]]. We note that ci[k] represents the average execution time of the jobs of τi that were executed during [t[k−1], t[k]]. The resource demand uD[ k] corresponds to an amount of execution time, that needs to be executed by the end of the period. If this amount is less than h, then it will be executed entirely, otherwise only h out of it will get executed; page 2, right column, The resource demand, in an interval of time h, is:
PNG
media_image5.png
200
400
media_image5.png
Greyscale
where Ccurrent is the sum of execution times of all jobs released during h and Cprevious is the sum of execution times of all queued up jobs, released before the beginning of the interval.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara with the teaching of Rafiliu to improve robustness and maintain stable operation when workload fluctuate,( Rafiliu, abstract) and also to control the processor’s utilization, ( Rafiliu, page 1, right column).
Claim 23 corresponds to claim 11, is rejected accordingly.
Regarding claim 12, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara in view of Rafiliu discloses all of the features with respect to claim 11 as outlined above. Claim 12 further recites: the first state space further characterizes priorities for ones of the plurality of backup operations that are allocated to one of the two or more backup servers for execution in the backup infrastructure environment; (CHANDAVARKAR, [0053] different priorities can also be assigned to the various replication and snapshot operations (corresponding to “a plurality of backup operations”) to be performed by the disaster recovery preparation service 170, which also provides the ability to vary the overall storage system load as needed. For example, the automated tuner 120 can increase, decrease or keep the priority the same for a particular batch of data or replication operation; [0049] the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… selects an action 140 to be performed. The action, for example, can be one or more tuning steps to be applied to the disaster recovery preparation service 170; [0048] the information associated with the current state 160 is retrieved, at least in part, from the storage system state parameters and metrics module 176, which monitors and stores state-related parameters and metrics associated with one of more storage systems 190 that are part of the storage cluster 180. In some embodiments, the current state 160 also includes information regarding performance metrics associated with the storage systems 190.)
However CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara does not clearly disclose:
and the second state space further characterizes a number of the plurality of backup operations arriving in a current time slot and a number of the plurality of backup operations not executed in a previous time slot.
However Rafiliu discloses:
and the second state space further characterizes a number of the plurality of backup operations arriving in a current time slot and a number of the plurality of backup operations not executed in a previous time slot. (Rafiliu, page 2, right column, The resource demand, in an interval of time h, is:
PNG
media_image5.png
200
400
media_image5.png
Greyscale
where Ccurrent is the sum of execution times of all jobs released during h and Cprevious is the sum of execution times of all queued up jobs, released before the beginning of the interval.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara with the teaching of Rafiliu to improve robustness and maintain stable operation when workload fluctuate,( Rafiliu, abstract) and also to control the processor’s utilization, ( Rafiliu, page 1, right column).
Regarding claim 13, CHANDAVARKAR in view of Siyadatzade in view of Liu in view of TORIGOE in view of Nara in view of Rafiliu discloses all of the features with respect to claim 11 as outlined above. Claim 13 further recites: the first state space further characterizes which of the two or more backup servers are active in a task scheduling time slot; (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0150], e.g. determining whether the disaster recovery preparation system has not met a recovery point objective… a time window when disaster recovery preparation operations are performed; [0051], The time of replication can be set to real-time or at scheduled intervals; [0075] CPU Usage: CPU usage indicates the percentage of a capacity of a processor that is currently in use. Monitoring CPU usage helps ensure that the system has enough processing power to handle the replication tasks without being overburdened.)
and the second state space further characterizes which of the two or more backup servers are active in a resource optimization time slot, the resource optimization time slot comprising two or more instances of the task scheduling time slot. (CHANDAVARKAR, [0049], e.g. the current state 160 is received by the state space module 114 included in each of the machine learning agents 110… the state space module 114 supplies information about the current system state of the disaster recovery preparation service 170; [0059] if one agent is tasked with optimizing replication frequency; [0019] This calibration necessitates a deeper understanding of data transaction patterns of the storage system, overall storage system behavior, and the complex interplay between replication processes, backup schedules, and recovery mechanisms. Furthermore, calibrating the operational parameters requires careful decision-making in the selection of recovery points, optimizing resource utilization while upholding data fidelity. [0064] The performance metrics can be transmitted as metrics 150 to the automated tuner 120. The performance metrics provides information regarding the overall health of the storage systems 190 to the automated tuner 120… The performance metrics stored by the storage system performance metrics module 174 can include; [0074] Storage Availability: This metric assesses the accessibility of the storage systems. It measures whether the storage systems are online and operational, ensuring that data can be written to and read from them during replication. The storage availability also Indicates the amount of free space on storage devices, ensuring sufficient capacity for replication; [0150], e.g. determining whether the disaster recovery preparation system has not met a recovery point objective… a time window when disaster recovery preparation operations are performed; [0051], The time of replication can be set to real-time or at scheduled intervals;)
Claim 24 corresponds to claim 13, is rejected accordingly.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Faezeh Forouharnejad whose telephone number is (571)270-7416. The examiner can normally be reached on generally Monday through Friday.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shah Sanjiv can be reached on (571)272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)
/F.F. /
Examiner, Art Unit 2166
/SANJIV SHAH/Supervisory Patent Examiner, Art Unit 2166
Prosecution Insights