Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
The following is a Final Office action to Application Serial Number 17/411,666, filed on August 25, 2021. In response to Examiner’s Non-Final Office Action of July 30, 2025, Applicant, on December 1, 2025, amended 1, 2,6, 9, 10, 14, 17, 18 and 19; and cancelled claims 4, 5, 12, 13 and 20. Claims 1-3, 6-11 and 14-19 are pending in this application and have been rejected below. Information Disclosure Statement (IDS) filed 10/3/2025 is acknowledged.
Response to Amendment
Applicant’s amendments are acknowledged.
Regarding 35 U.S.C. § 101 rejection, the amended claims have been considered
and are insufficient to overcome the rejection. Please refer to the 35 U.S.C. § 101 rejection for further explanation and rationale.
The 35 U.S.C. § 103 rejections are hereby amended pursuant to applicants amendments. Updated 35 U.S.C. § 103 rejections have been applied to amended
claims. Please refer to the § 103 rejection for further explanation and rationale.
Response to Arguments
Applicant’s arguments filed December 1, 2025 have been fully considered but they are not persuasive and/or are moot in view of the revised rejections. Applicant’s arguments will be addressed herein below in the order in which they appear in the response filed December 1, 2025.
On Pg. 7-8 , regarding the 35 U.S.C. § 101 rejection, Applicant states claimed subject matter provides a technological solution to computer technology which overcomes the challenges with training reinforcement learning networks with state data in real time or near real time by storing time-sensitive state data in a static location in RAM, thereby allowing agents and processes to access the state data without significant overhead or delay. As noted at paragraph [0066], "Storing real time or near real time data in a local memory device 320 [RAMI allows separate processes by a number of system components (e.g., agents 180 or memory controller 330) to quickly access the state data, which can be time sensitive for a trading platform, without significant 10 overhead and without significant delay, thereby improving the speed, accuracy, performance, and efficiency of the learning process of the multiple automated agents 180". The Applicant submits that it is clear that the claimed subject matter is not directed simply to "resource management" using generic computer components, as alleged in the subject Office Action as noted in Desjardins. In response, the claims primarily recite the additional element of using computer components to perform each step. The “automated agents”, ”system”; “processor”; “software code”; “memory”; “memory controller”, “memory device”, “computing device”, and “computer-readable storage medium” is recited at a high-level of generality, such that it amounts no more than mere instructions to apply the exception using a computer component. See MPEP 2106.05(f). Examiner finds the present claims do not demonstrate any functional advancement to any technology or technological field, in order for the claim elements to be considered significantly more than the abstract idea itself. Applicant has not identified anything in the claimed invention that shows or even submits the technology is being improved or there was a problem in the technology that the claimed invention solves. Utilizing computer structure and technology to collect, analyze and display data are all, both individually and in combination, generic computer functions such as receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); electronic recordkeeping, Ultramercial, 772 F.3d at 716, 112 USPQ2d at 1755 (updating an activity log) and storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93 (See MPEP 2106.05(d)(II).
On page 8-9, regarding the 35 U.S.C. § 103 rejection, Applicant argues that prior art does not disclose (1) "allocating a particular memory location on random access memory (RAM) as a dedicated portion of said RAM to store data for the respective resource"; (2) "for each of the training cycles for the subset of the plurality of automated agents (180), store said formatted current state data for the respective resource in the particular memory location of said RAM allocated to the respective resource by appending the current state data to an existing array for the respective resource during each training cycle at the particular location of said RAM allocated to the respective resource" and (3) addressing… (improving the speed, accuracy, performance, and/or efficiency of the learning process of multiple reinforcement learning agents). In response, new ground(s) of rejection is made necessitated by amendment see MPEP 706.07a. Regarding the 35 U.S.C. § 103 rejection, Applicant’s arguments with respect to claims has been considered but are moot in view of the new grounds of rejection.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 6-11 and 14-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claims 1-3, 6-11 and 14-19 are directed to resource task management.
Claim 1 recites a system for resource task management, Claim 9 recites a method for resource task management and Claim 17 recites an article of manufacture for resource task management, which include generating resource task requests for a plurality of resources; for each resource of the plurality of resources, allocating, using a memory allocation signal, a particular memory location to store all state data for the respective resource; obtain current state data for the respective resource, said current state data being a feature vector including feature data for the respective resource; format current state data for said respective resource for storage in said particular memory location; receiving a request for state data for a particular resource from a subset of the plurality of automated agents; for each of the training cycles for the subset of the plurality of automated agents, storing formatted current state data for the particular resource in the particular memory location allocated to the particular resource by appending the formatted current state data to an existing state data array for the particular resource during each training cycle at particular memory location allocated to the particular resource, wherein the formatted current state data becomes the newest member of the state data array as the last data instance of the state data array; and transmitting a pointer to the said particular memory location for the particular resource to the subset of the automated agents, to facilitate asynchronous reading of the current state data for the particular resource during each training cycle with reduced overhead and delay. As drafted, this is, under its broadest reasonable interpretation, within the Abstract idea grouping of “Methods of Organizing Human Activity” – managing interactions. The recitation of “automated agents”, ”system”; “processor”; “software code”; “memory”; “memory controller”, “signal”; “random access memory (RAM)”; “market data manager”; “computing device”, and “computer-readable storage medium”, provide nothing in the claim elements to preclude the step from being “Methods of Organizing Human Activity”- managing interactions. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims primarily recite the additional element of using computer components to perform each step. The “automated agents”, ”system”; “processor”; “software code”; “memory”; “memory controller”, “signal”; “random access memory (RAM)”; “market data manager”; “computing device”, and “computer-readable storage medium” recited at a high-level of generality, such that it amounts no more than mere instructions to apply the exception using a computer component. See MPEP 2106.05(f).
Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims also fail to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, and/or an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See 84 Fed. Reg. 55. In particular, there is a lack of improvement to a computer or technical field in resource management.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “automated agents”, ”system”; “processor”; “software code”; “memory”; “memory controller”, “signal”; “random access memory (RAM)”; “market data manager”; “computing device”, and “computer-readable storage medium “is insufficient to amount to significantly more. (See MPEP 2106.05(f) – Mere Instructions to Apply an Exception – “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. With regards to receiving data and step 2B, it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) and Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Examiner concludes that the additional elements in combination fail to amount to significantly more than the abstract idea based on findings that each element merely performs the same function(s) in combination as each element performs separately. The claim is not patent eligible. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
Dependent Claims 2-3, 4-8, 10-11, 14-16, and 18-19 recite wherein current state data for the particular resource comprises the current state data for the particular resource in an environment in which the resource task requests are made; store updated state data for each of the plurality of resources in the respective particular memory location of the RAM allocated to the respective resource; the current state data for the particular resource includes a market price of the particular resource; the environment includes at least one trading venue; and further narrowing the abstract idea. These recited limitations in the dependent claims do not amount to significantly more than the above-identified judicial exceptions in Claims 1, 9 and 17. Regarding Claims, 6 and 14, and the additional elements of “memory”; “RAM”; “processor”; “software code”; “system”- it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-11 and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Burhani et al., EP3576038A1, [hereinafter Burhani], in view of Jain et al., US Publication No. 20240007414 A1, [hereinafter Jain].
Regarding Claim 1,
Burhani teaches
A computer-implemented system for training an automated agent, the system comprising: a communication interface; at least one processor; memory in communication with the at least one processor; and software code stored in the memory, which when executed at the at least one processor causes the system to: (Burhani (par. [0047]: The matching engine 114 can be a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning”)
instantiate a plurality of automated agents for generating resource task requests for a plurality of resources, each of the automated agents configured to train over a plurality of training cycles; (Burhani (par. [0039]: The platform 100 can train one or more reinforcement learning neural networks 110; par. [0047]: [..] a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning; par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities; par. [0079]: [..] platform 100 instantiates an automated agent 180 that maintains a reinforcement learning neural network[..] The automated agent 180 generates, according to outputs of its reinforcement learning neural network, signals for communicating resource task requests for a given resource;)
receive a request for state data for a particular resource from a subset of the plurality of automated agents; (Burhani par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities;)
Burhani teaches agent platform and the feature is expounded upon by Jain:
for each resource of the plurality of resources, allocate by a memory controller using a memory allocation signal, a particular memory location on random access memory (RAM) as a dedicated portion of the memory device to store all state data for the respective resource; (Jain Par. 276-277- Examples disclosed herein generate a resource utilization model which includes generating any number of candidate models (e.g., runtime models) with varying resource utilization to determine how to allocate resources to different workloads (e.g., AI models) using a rewards-based system. Generally speaking, a given workload typically involves numerous models to be invoked to accomplish computation objectives. “)
obtain, by a market data manager, current state data for the respective resource, said current state data being a feature vector including feature data for the respective resource (Jain Par. 92- Edge gateway servers may be equipped with pools of memory and storage resources to perform computation in real-time for low latency use-cases (e.g., autonomous driving or video surveillance) for connected client devices. Or as an example, base stations may be augmented with compute and acceleration resources to directly process service workloads for connected user equipment, without further communicating data via backhaul networks”; Par. 393; Par. 647) ;
format, by said market data manager, said current state data for said respective resource for storage in said particular memory location of said RAM (Par. 396-397- “Automated featurization can help drive AutoML by automatically identifying, capturing, and leveraging relevant, high-quality feature data for model training, testing, validation, etc. For example, featurization automation can assist an ML algorithm to learn better, forming an improved ML model for deployment and utilization. Featurization can include feature normalization, missing data identification and interpolation, format conversion and/or scaling, etc. Featurization can transform raw data into a set of features to be consumed by a training algorithm, etc. In certain examples, featurization is based on an analysis of an underlying platform (e.g., hardware, software, firmware, etc.) on which a model will be operating.”);
for each of the training cycles for the subset of the plurality of automated agents, store formatted current state data for the particular resource in the particular memory location of said RAM allocated to the particular resource by appending the formatted current state data to an existing state data array for the particular resource during each training cycle at the particular memory location of said RAM allocated to the particular resource, wherein the formatted current state data becomes the newest member of the state data array as the last data instance of the state data array ; (Jain Par. 108- The memory D106 may be embodied as any type of volatile (e.g., dynamic random access memory (DRAM), etc.) or non-volatile memory or data storage capable of performing the functions described herein. Volatile memory may be a storage medium that requires power to maintain the state of data stored by the medium. Non-limiting examples of volatile memory may include various types of random access memory (RAM), such as DRAM or static random access memory (SRAM). One particular type of DRAM that may be used in a memory module is synchronous dynamic random access memory (SDRAM).”;Par. 227; Par. 242-243- Considering that one or more conditions have changed, the example agent managing circuitry ID5_C310 assigns another agent to re-assess performance of the selected alternate path (block ID5_F312). The example benchmark managing circuitry ID5_C302 updates the workload with new information corresponding to the newly selected path and the current conditions (block ID5_F314).; Par. 261; 265; Par. 409; Par. 418-419)
and transmit a pointer to said particular memory location of said RAM for the particular resource to the subset of the automated agents, to facilitate asynchronous reading of the current state data for the particular resource during each training cycle with reduced overhead and delay . (Jain Par. 149; Par. 476; Par. 478-479- Existing techniques to compress models are unscalable and inefficient for large-scale model adaptation. The current techniques (e.g., an optimizer/learning agent) are repeated independently from scratch for every instance of a model (e.g., a neural network), its target platform and corresponding performance goals of the model. As a result, for every new workload having one or more models to be optimized (e.g., compressed), optimization resources (e.g., exploration agents) are spawned from a ground state absent of anything learned from previously spawned agents. Efficient example solutions to support scaling are disclosed herein. Examples disclosed herein create a scaling technique to support multiple XPU platforms (e.g., mix of architectures collectively described as XPU includes CPU, GPU, VPU, FPGA, etc.) where target deployment can be any combination of XPU platforms (heterogeneous inference). Examples disclosed herein include an adaptable system that supports different neural network topologies, datasets from different customers (e.g., tenants), and different target performance-accuracy trade-offs to support scaling and create a model (e.g., neural network) with improved efficiency.)
Burhani and Jain are directed to resource processing. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Burhani in view of, as taught by Jain, by utilizing additional data storage and delivery processing with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Burhani with the motivation of improving execution performance across the multiple AI models (Jain Par. 276).
Regarding Claim 2, Claim 10 and Claim 18,
wherein said current state data for the particular resource comprises the current state data for the particular resource in an environment in which the resource task requests are made. (Burhani Par. 89-90-“ As the automated agents 602 learn during operation, platform 100' obtains update data 606 from one or more of the automated agents 602 reflective of learnings at the automated agents 602. Update data 606 includes data descriptive of an "experience" of an automated agent in generating a task request. Update data 606 may include one or more of: (i) input data to the given automated agent 602 and applied normalizations (ii) a list of possible resource task requests evaluated by the given automated agent with associated probabilities of making each requests, and (iii) one or more rewards for generating a task request.).
Regarding Claim 3, Claim 11 and Claim 19,
wherein the updated state data for the particular resource further comprises historical state data for the particular resource in the environment in which the resource task requests are made. (Burhani Par. 89-90-“ Platform 100' processes update data 606 to update master model 600 according to the experience of the automated agent 602 providing the update data 606. Consequently, automated agents 602 instantiated thereafter will have benefit of the learnings reflected in update data 606. Platform 100' may also sends model changes 608 to the other automated agents 602 so that these pre-existing automated agents 602 will also have benefit of the learnings reflected in update data 606..”;Par. 133).
Regarding Claim 4, Claim 12 and Claim 20- Cancelled
Regarding Claim 5 and Claim 13, - Cancelled
Regarding Claim 6 and Claim 14,
store updated state data for each of the plurality of resources ….. ; (Burhani Par. 89-92-“ In some embodiments, platform 100' obtains update data 606 after each time step. In other embodiments, platform 100' obtains update data 606 after a predefined number of time steps, e.g., 2, 5, 10, etc. In some embodiments, platform 100' updates master model 600 upon each receipt update data 606. In other embodiments, platform 100' updates master model 600 upon reaching a predefined number of receipts of update data 606, which may all be from one automated agents 602 or from a plurality of automated agents 602.; In one example, platform 100' instantiates a first automated agent 602 and a second automated agent 602, each from master model 600. Platform 100' obtains update data 606 from the first automated agents 602. Platform 100' modifies master model 600 in response to the update data 606 and then applies a corresponding modification to the second automated agent 602. Of course, the roles of the automated agents 602 could be reversed in another example such that platform 100' obtains update data 606 from the second automated agent 602 and applies a corresponding modification to the first automated agent 602.”)
in the respective particular memory location of the RAM allocated to the respective resource (Jain Par. 276-277- Examples disclosed herein generate a resource utilization model which includes generating any number of candidate models (e.g., runtime models) with varying resource utilization to determine how to allocate resources to different workloads (e.g., AI models) using a rewards-based system. Generally speaking, a given workload typically involves numerous models to be invoked to accomplish computation objectives. “)
Burhani and Jain are directed to resource processing. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Burhani in view of, as taught by Jain, by utilizing additional data storage and delivery processing with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Burhani with the motivation of improving execution performance across the multiple AI models (Jain Par. 276).
Regarding Claim 7 and Claim 15,
wherein the current state data for the particular resource includes a market price of the particular resource. (Burhani Par. 97-98-“ Pricing Features: In some embodiments, input normalization may involve the training engine 118 computing pricing features. In some embodiments, pricing features for input normalization may involve price comparison features, passive price features, gap features, and aggressive price features.”)
Regarding Claim 8 and Claim 16,
wherein the environment includes at least one trading venue. (Burhani Par. 94-“ In some embodiments, the plurality of automated agents 602 may be distributed geographically, e.g., with certain of the automated agent 602 placed for geographic proximity to certain trading venues.”)
Regarding Claim 9,
Burhani teaches
A computer-implemented method for training an automated agent, the method comprising: (Burhani (par. [0047]: The matching engine 114 can be a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning”)
instantiating a plurality of automated agents for generating resource task requests for a plurality of resources, each of the automated agents configured to train over a plurality of training cycles; (Burhani (par. [0039]: The platform 100 can train one or more reinforcement learning neural networks 110; par. [0047]: [..] a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning; par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities; par. [0079]: [..] platform 100 instantiates an automated agent 180 that maintains a reinforcement learning neural network[..] The automated agent 180 generates, according to outputs of its reinforcement learning neural network, signals for communicating resource task requests for a given resource;)
receiving a request for state data for a particular resource from a subset of the plurality of automated agents (Burhani par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities;)
Burhani teaches agent platform and the feature is expounded upon by Jain:
for each resource of the plurality of resources, allocating by a memory controller using a memory allocation signal, a particular memory location on random access memory (RAM) as a dedicated portion of the memory device to store all state data for the respective resource; (Jain Par. 276-277- Examples disclosed herein generate a resource utilization model which includes generating any number of candidate models (e.g., runtime models) with varying resource utilization to determine how to allocate resources to different workloads (e.g., AI models) using a rewards-based system. Generally speaking, a given workload typically involves numerous models to be invoked to accomplish computation objectives. “)
obtaining, by a market data manager, current state data for the respective resource, said current state data being a feature vector including feature data for the respective resource (Jain Par. 92- Edge gateway servers may be equipped with pools of memory and storage resources to perform computation in real-time for low latency use-cases (e.g., autonomous driving or video surveillance) for connected client devices. Or as an example, base stations may be augmented with compute and acceleration resources to directly process service workloads for connected user equipment, without further communicating data via backhaul networks”; Par. 393; Par. 647) ;
formatting, by said market data manager, said current state data for said respective resource for storage in said particular memory location of said RAM (Par. 396-397- “Automated featurization can help drive AutoML by automatically identifying, capturing, and leveraging relevant, high-quality feature data for model training, testing, validation, etc. For example, featurization automation can assist an ML algorithm to learn better, forming an improved ML model for deployment and utilization. Featurization can include feature normalization, missing data identification and interpolation, format conversion and/or scaling, etc. Featurization can transform raw data into a set of features to be consumed by a training algorithm, etc. In certain examples, featurization is based on an analysis of an underlying platform (e.g., hardware, software, firmware, etc.) on which a model will be operating.”);
for each of the training cycles for the subset of the plurality of automated agents, storing formatted current state data for the particular resource in the particular memory location of said RAM allocated to the particular resource by appending the formatted current state data to an existing state data array for the particular resource during each training cycle at the particular memory location of said RAM allocated to the particular resource, wherein the formatted current state data becomes the newest member of the state data array as the last data instance of the state data array ; (Jain Par. 108- The memory D106 may be embodied as any type of volatile (e.g., dynamic random access memory (DRAM), etc.) or non-volatile memory or data storage capable of performing the functions described herein. Volatile memory may be a storage medium that requires power to maintain the state of data stored by the medium. Non-limiting examples of volatile memory may include various types of random access memory (RAM), such as DRAM or static random access memory (SRAM). One particular type of DRAM that may be used in a memory module is synchronous dynamic random access memory (SDRAM).”;Par. 227; Par. 242-243- Considering that one or more conditions have changed, the example agent managing circuitry ID5_C310 assigns another agent to re-assess performance of the selected alternate path (block ID5_F312). The example benchmark managing circuitry ID5_C302 updates the workload with new information corresponding to the newly selected path and the current conditions (block ID5_F314).; Par. 261; 265; Par. 409; Par. 418-419)
and transmitting a pointer to said particular memory location of said RAM for the particular resource to the subset of the automated agents, to facilitate asynchronous reading of the current state data for the particular resource during each training cycle with reduced overhead and delay . (Jain Par. 149; Par. 476; Par. 478-479- Existing techniques to compress models are unscalable and inefficient for large-scale model adaptation. The current techniques (e.g., an optimizer/learning agent) are repeated independently from scratch for every instance of a model (e.g., a neural network), its target platform and corresponding performance goals of the model. As a result, for every new workload having one or more models to be optimized (e.g., compressed), optimization resources (e.g., exploration agents) are spawned from a ground state absent of anything learned from previously spawned agents. Efficient example solutions to support scaling are disclosed herein. Examples disclosed herein create a scaling technique to support multiple XPU platforms (e.g., mix of architectures collectively described as XPU includes CPU, GPU, VPU, FPGA, etc.) where target deployment can be any combination of XPU platforms (heterogeneous inference). Examples disclosed herein include an adaptable system that supports different neural network topologies, datasets from different customers (e.g., tenants), and different target performance-accuracy trade-offs to support scaling and create a model (e.g., neural network) with improved efficiency.)
Burhani and Jain are directed to resource processing. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Burhani in view of, as taught by Jain, by utilizing additional data storage and delivery processing with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Burhani with the motivation of improving execution performance across the multiple AI models (Jain Par. 276).
Regarding Claim 17,
Burhani teaches
A non-transitory computer-readable storage medium storing instructions which when executed adapt at least one computing device to: (Burhani (par. [0047]: The matching engine 114 can be a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning”)
instantiate a plurality of automated agents for generating resource task requests for a plurality of resources, each of the automated agents configured to train over a plurality of training cycles; (Burhani (par. [0039]: The platform 100 can train one or more reinforcement learning neural networks 110; par. [0047]: [..] a highly performant stock market simulation environment designed to provide rich datasets and ever changing experiences to reinforcement learning networks 110 (e.g. of agents 180) in order to accelerate and improve their learning; par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities; par. [0079]: [..] platform 100 instantiates an automated agent 180 that maintains a reinforcement learning neural network[..] The automated agent 180 generates, according to outputs of its reinforcement learning neural network, signals for communicating resource task requests for a given resource;)
receive a request for state data for a particular resource from a subset of the plurality of automated agents; (Burhani par. [0033]: [..] the automated agent may generate requests for tasks to be performed in relation to securities;)
Burhani teaches agent platform and the feature is expounded upon by Jain:
for each resource of the plurality of resources, allocate by a memory controller using a memory allocation signal, a particular memory location on random access memory (RAM) as a dedicated portion of the memory device to store all state data for the respective resource; (Jain Par. 276-277- Examples disclosed herein generate a resource utilization model which includes generating any number of candidate models (e.g., runtime models) with varying resource utilization to determine how to allocate resources to different workloads (e.g., AI models) using a rewards-based system. Generally speaking, a given workload typically involves numerous models to be invoked to accomplish computation objectives. “)
obtain, by a market data manager, current state data for the respective resource, said current state data being a feature vector including feature data for the respective resource (Jain Par. 92- Edge gateway servers may be equipped with pools of memory and storage resources to perform computation in real-time for low latency use-cases (e.g., autonomous driving or video surveillance) for connected client devices. Or as an example, base stations may be augmented with compute and acceleration resources to directly process service workloads for connected user equipment, without further communicating data via backhaul networks”; Par. 393; Par. 647) ;
format, by said market data manager, said current state data for said respective resource for storage in said particular memory location of said RAM (Par. 396-397- “Automated featurization can help drive AutoML by automatically identifying, capturing, and leveraging relevant, high-quality feature data for model training, testing, validation, etc. For example, featurization automation can assist an ML algorithm to learn better, forming an improved ML model for deployment and utilization. Featurization can include feature normalization, missing data identification and interpolation, format conversion and/or scaling, etc. Featurization can transform raw data into a set of features to be consumed by a training algorithm, etc. In certain examples, featurization is based on an analysis of an underlying platform (e.g., hardware, software, firmware, etc.) on which a model will be operating.”);
for each of the training cycles for the subset of the plurality of automated agents, store formatted current state data for the particular resource in the particular memory location of said RAM allocated to the particular resource by appending the formatted current state data to an existing state data array for the particular resource during each training cycle at the particular memory location of said RAM allocated to the particular resource, wherein the formatted current state data becomes the newest member of the state data array as the last data instance of the state data array ; (Jain Par. 108- The memory D106 may be embodied as any type of volatile (e.g., dynamic random access memory (DRAM), etc.) or non-volatile memory or data storage capable of performing the functions described herein. Volatile memory may be a storage medium that requires power to maintain the state of data stored by the medium. Non-limiting examples of volatile memory may include various types of random access memory (RAM), such as DRAM or static random access memory (SRAM). One particular type of DRAM that may be used in a memory module is synchronous dynamic random access memory (SDRAM).”;Par. 227; Par. 242-243- Considering that one or more conditions have changed, the example agent managing circuitry ID5_C310 assigns another agent to re-assess performance of the selected alternate path (block ID5_F312). The example benchmark managing circuitry ID5_C302 updates the workload with new information corresponding to the newly selected path and the current conditions (block ID5_F314).; Par. 261; 265; Par. 409; Par. 418-419)
and transmit a pointer to said particular memory location of said RAM for the particular resource to the subset of the automated agents, to facilitate asynchronous reading of the current state data for the particular resource during each training cycle with reduced overhead and delay . (Jain Par. 149; Par. 476; Par. 478-479- Existing techniques to compress models are unscalable and inefficient for large-scale model adaptation. The current techniques (e.g., an optimizer/learning agent) are repeated independently from scratch for every instance of a model (e.g., a neural network), its target platform and corresponding performance goals of the model. As a result, for every new workload having one or more models to be optimized (e.g., compressed), optimization resources (e.g., exploration agents) are spawned from a ground state absent of anything learned from previously spawned agents. Efficient example solutions to support scaling are disclosed herein. Examples disclosed herein create a scaling technique to support multiple XPU platforms (e.g., mix of architectures collectively described as XPU includes CPU, GPU, VPU, FPGA, etc.) where target deployment can be any combination of XPU platforms (heterogeneous inference). Examples disclosed herein include an adaptable system that supports different neural network topologies, datasets from different customers (e.g., tenants), and different target performance-accuracy trade-offs to support scaling and create a model (e.g., neural network) with improved efficiency.)
Burhani and Jain are directed to resource processing. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Burhani in view of, as taught by Jain, by utilizing additional data storage and delivery processing with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Burhani with the motivation of improving execution performance across the multiple AI models (Jain Par. 276).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure US Publication No. 20220383100 A1 to Zhu et al.- Abstract- A first neural network can be trained to approximate a state-action value function to estimate an expected cumulative return for an agent to perform an action in a given state, the agent being an autonomous reinforcement learning agent running on the processor. A second neural network can be trained to generate a simulated experience, the second network trained to predict a simulated state at a next time step after performing a given action, the second neural network being trained using real experience in a real environment. The first neural network is trained based on the simulated experience and a real experience from a real environment. A selected action selected by the second neural network given a current state of the real environment can be performed. The agent can explore an action space by uniformly sampling an action from all possible remaining action-state space combinations and performing the sampled action.”
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chesiree Walton, whose telephone number is (571) 272-5219. The examiner can normally be reached from Monday to Friday between 8 AM and 5 PM. If any attempt to reach the examiner by telephone is unsuccessful, the examiner’s supervisor, Patricia Munson, can be reached at (571) 270-5396. The fax telephone numbers for this group are either (571) 273-8300 or (703) 872-9326 (for official communications including After Final communications labeled “Box AF”).
Another resource that is available to applicants is the Patent Application Information Retrieval (PAIR). Information regarding the status of an application can be obtained from the (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAX. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, please feel free to contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Applicants are invited to contact the Office to schedule an in-person interview to discuss and resolve the issues set forth in this Office Action. Although an interview is not required, the Office believes that an interview can be of use to resolve any issues related to a patent application in an efficient and prompt manner.
Sincerely,
/CHESIREE A WALTON/Examiner, Art Unit 3624