Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 6-14, 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Singh [20240037193], in view of Chakraborti et al. [11,797,820]
As to claim 1,
Singh [20240037193] teaches a method for managing a distributed system, the method comprising:
deploying a trained agent to a data processing system of the distributed system, the trained agent being adapted to manage, at least, configuration of the data processing system responsive to changes in state of the data processing system, and the trained agent having been trained using, at least in part, an optimization function based on tacit statements of stakeholders with respect to the distributed system [abstract: “Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions. ” and Fig. 1, Singh teaches methods for managing distributed systems using RL agents [174]. Abstract: "The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run". See [0006]; FIG. 1: Singh expressly discloses deploying trained management-system agents to live target distributed computer systems. Abstract: "before being deployed to live, target distributed computer systems." See [0006]: "When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent." FIG. 1 (deployment pipeline); FIG. 3 (deployed agent in distributed system). Singh discloses agents that monitor state and dynamically modify configuration: ¶ [0141]: "Each deployed management-system agent... produces traces that are collected and stored." See [0107]: agents apply control decisions that modify the distributed system's configuration responsive to observed state changes. See FIG. 15 (agent state - action pipeline); See FIG. 44D (configuration modification workflow). Singh discloses simulation-based training of agents using a digital twin driven by historic system traces and conditions: ¶ [0151], [0175]; FIG. 43B-C) ];
updating, by the trained agent, the configuration of the data processing system over time to guide an actual state of the data processing system to a desired state (Singh explicitly discloses this step: [0006], [0141]: deployed management-system agents continuously update configuration of the distributed system. See ¶ [0006]: the agent uses its trained policy to select control actions that move the system toward the desired state and 00174-0175, and 0041: “the management-system agent iteratively receives state vectors from the environment and, in response, issues actions to the controlled environment. ” and 0095 and 0093: “information regarding the managed environment that allows the reinforcement-learning-based management-system agent to infer the current state of the environment which, in turn, allows the reinforcement-learning-based management-system agent to issue actions that push the managed environment towards states that, over time, produce the greatest cumulative reward feedbacks. ”); and
while the configuration is updated, providing, by the data processing system, desired computer implemented services (Singh expressly discloses the system continuing to provide services during agent management: Abstract: agents "operate in a controller mode" while the distributed system continues running. See [0080]-[0082]: the distributed system continues providing its services while the agent manages it that the agent does not require downtime). Singh teaches “When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent."
But Singh does NOT explicitly disclose executing function/action based on tacit statements of stakeholders with respect to the distributed system.
However Chakraborti et al. (US No. 11,797,820) teaches executing function/action using tacit statements of stakeholders (as shown in Fig. 1A, which is reproduced below for ease of reference and convenience, Chakraborti discloses reinforcement learning (RL) agent adapting its behavior based on system state changes, see col. 12:21-38: the RL agent receives state information and executes corresponding actions to manage the system. FIG. 1A (RL model 115 taking state-based actions). Chakraborti directly discloses using stakeholder-expressed information (tacit statements in knowledge stores) to drive/augment RL agent training. See ¶ [0028]-[0035]: the knowledge store contains user-generated, semi-structured, natural language information (tacit statements) from stakeholders. See col. 3:51 thru col. 4:33: this information is processed by the NLP engine and used to define reward functions that drive RL training that directly teaching that stakeholder tacit statements drive the training/simulation process).
It would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine teaching of Singh and Chakraborti because both are directed toward training agent by being able to execute corresponding action by being able to information stakeholder information such that efficiently perform function as the per need.
As to claim 2,
Singh 2. The method of claim 1, wherein the optimization function is an objective function, and the trained agent is obtained using, at least in part, a training modality that rewards activity of the agent that optimizes the objective function[[0006]: "When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent." FIG. 1 (deployment pipeline); FIG. 3 (deployed agent in distributed system). Singh discloses agents that monitor state and dynamically modify configuration]
As to claim 3,
Singh teaches the trained agent is trained, at least in part, using a simulation of at least the data processing system of the distributed system, the simulation being usable to predict operation of the distributed system in response to activity of the trained agent [0107]: agents apply control decisions that modify the distributed system's configuration responsive to observed state changes. See FIG. 15 (agent state - action pipeline); See FIG. 44D (configuration modification workflow). Singh discloses simulation-based training of agents using a digital twin driven by historic system traces and conditions: ¶ [0151], [0175]; FIG. 43B-C) ].
As to claim 4,
Chakraborti teaches the activity comprises:obtaining information regarding changes in state of the distributed system in the simulation; andchanging, based on the information, at least a portion of operation of the distributed system [col. 12:21-38: the RL agent receives state information and executes corresponding actions to manage the system. FIG. 1A (RL model 115 taking state-based actions). Chakraborti directly discloses using stakeholder-expressed information (tacit statements in knowledge stores) to drive/augment RL agent training. See ¶ [0028]-[0035]:
As to claim 6,
Singh teaches while the desired computer implemented services are provided: obtaining second changes in the state of the distributed system; andupdating operation of the trained agent based, at least in part, on the second changes [ [0006], [0151]: deployed management-system agents continuously update configuration of the distributed system. See ¶ [0006]: the agent uses its trained policy to select control actions that move the system toward the desired state)- desired state]
As to claim 7,
Chakraborti teaches the tacit statements comprises at least one derived tacit statement based on public information from the stakeholders, the tacit statements being private statements [see col. 12:21-38: the RL agent receives state information and executes corresponding actions to manage the system. FIG. 1A (RL model 115 taking state-based actions). Chakraborti teaches using stakeholder-expressed information (tacit statements in knowledge stores) to drive/augment RL agent training. See ¶ [0028]-[0035]: the knowledge store contains user-generated, semi-structured, natural language information (tacit statements) from stakeholders]
As to claim 8:
Singh teaches the tacit statements indicates at least one goal of the stakeholder with respect to the distributed system that is to be accomplished [0006]: "When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent.".]
As to claim 9,
Singh teaches goal is not defined in terms of operation of the distributed system [0006]; FIG. 1 (distributed system architecture). Singh expressly discloses deploying trained management-system agents to live target distributed computer systems. Abstract: "before being deployed to live, target distributed computer systems." [See ¶ [0006]: "When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent." – determining has two aspect of one with condition met and second when condtion is not met. Operation is not defined when the determination condition is not met ].
As to claim 10,
Chakraboty teaches the data processing system is a production system of the distributed system that contributed to, at least in part, primary computer implemented services provided by the distributed system [see col. 12:21-38: the RL agent receives state information and executes corresponding actions to manage the system. FIG. 1A (RL model 115 taking state-based actions). Chakraborti directly discloses using stakeholder-expressed information (tacit statements in knowledge stores) to drive/augment RL agent training. See ¶ [0028]-[0035]: the knowledge store contains user-generated, semi-structured, natural language information (tacit statements) from stakeholders .]
As to claim 11-14,
Singh and Chakraborthy teach this claim according to the reasoning set forth in claim 1 -4 supra.
As to claim 16-19,
Singh and Chakraborthy teach this claim according to the reasoning set forth in claim 1 -4 supra.
Allowable Subject Matter
Claim 5, 15, 20 objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KESHAB R PANDEY whose telephone number is (571)270-0176. The examiner can normally be reached Monday-Friday 9:00-5:00(ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jaweed Abbaszadeh can be reached at (571) 270-1640. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KESHAB R PANDEY/Primary Examiner, Art Unit 2176
