Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 1 requires “processing the usage data with a first machine learning model to determine one or more states corresponding to one or more respective services of the set of services, wherein the first machine learning model evaluates one or more resource allocations for one or more services in the set of services based on the usage data” and “assigning a new resource allocation to at least one service in the set of services using a second machine learning model comprising: (i) a prediction component that predicts the new resource allocation based at least in part on the determined one or more states corresponding to the at least one service; and (ii) a feedback component that updates the prediction component based on an evaluation of the predicted new resource allocation.”
The limitations of processing to determine and assigning a new resource allocation, as drafted, under its broadest reasonable interpretation, covers performance of the limitation in the mind or using mathematical concepts but for the recitation of generic computer components. That is, other than reciting “a first machine learning model” and “a second machine learning model,” nothing in the claim element precludes the step from practically being performed in the mind or by using mathematical concepts. For example, but for the “machine learning model” language, processing to determine and assigning a new resource allocation in the context of this claim encompasses the user mentally determining states based on usage data viewed on a terminal or implementing a mathematical formula to determine states based on usage data. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind (or recites mathematical concepts) but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim additionally recites “obtaining usage data relating to execution of a set of services” and “the method is performed by at least one processing device comprising a processor coupled to a memory.” The processing device coupled to memory is recited at a high-level of generality (i.e., as a generic computer hardware) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Further, the obtaining amounts to insignificant extra solution activity of mere data gathering. See MPEP § 2106.05(g). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “obtaining usage data relating to execution of a set of services” and “the method is performed by at least one processing device comprising a processor coupled to a memory” amount to no more than mere instructions to apply the exception using a generic computer component and mere data gathering. Mere instructions to apply an exception using a generic computer component and data gathering cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 2, it additionally recites evaluating using a loss function and providing feedback, which can be performed in the human mind (mentally) or by mathematical concepts. Accordingly, claim 2 is ineligible.
Regarding claim 3, it additionally recites a dynamic advantage function and baseline rewards for reward boosting, which can be performed in the human mind (mentally) or by mathematical concepts. Accordingly, claim 3 is ineligible.
Regarding claim 4, it additionally recites the machine learning model comprises a feed-forward neural network, which amounts to a mathematical concept. Accordingly, claim 4 is ineligible.
Regarding claim 5, the components comprising an actor network and critic network, which amount to mathematical concepts. Accordingly, claim 5 is ineligible.
Regarding claim 6, the further limits the determined to comprise information and a priority level, which can be performed in the human mind, i.e., a human can determine a state comprising information and priority. Accordingly, claim 6 is ineligible.
Regarding claim 7, the further limits the usage data to be collected from edge nodes; however, the claim is still merely directed to insignificant extra solution activity of data gathering. Accordingly, claim 7 is ineligible.
Regarding claims 8-20, they correspond to claims 1-7. Therefore, they are ineligible for the same reasons.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 6, it refers to “the resource allocation;” however, claim 1 defines “one or more resource allocations” and “a new resource allocation;” therefore, reference to “the resource allocation” is ambiguous. Claim 13 is indefinite for the same reason.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5, 8, 12, 15, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eker (US 2024/0403131) and further in view of Melodia (US 2022/0167236).
Regarding claim 1, Eker teaches: A computer-implemented method, comprising:
obtaining usage data relating to execution of a set of services (¶ 44, “Data on historical resource allocation may be collected for a period of time”);
processing the usage data with a first machine learning model (claim 1, “running a Reinforcement Learning, RL, model on the information representing the workload, current and historical resource allocations”) to determine one or more states corresponding to one or more respective services of the set of services (¶ 49, “the state 305 may be the current resource allocation of the service mesh 100, historical resource allocation and an information representing an input workload”), wherein the first machine learning model evaluates one or more resource allocations for one or more services in the set of services based on the usage data (claim 1, “determining a reward, wherein the reward is indicative of completed jobs and allocated resources of the service mesh”); and
assigning a new resource allocation to at least one service in the set of services (¶ 43, “The output of the RL model is a further resource allocation, obtained in step 211”)
(i) a prediction component that predicts the new resource allocation based at least in part on the determined one or more states corresponding to the at least one service (¶ 48, “A purpose of the RL algorithm is for an agent to learn a policy that maximizes a reward, wherein a policy comprises suggested actions that the agent should take for every possible state”); and
(ii) a feedback component that updates the prediction component based on an evaluation of the predicted new resource allocation (¶ 31, “uses a reinforcement learning, RL, model with a feedback signal to provide scaling of resources of the microservice-based application”);
wherein the method is performed by at least one processing device comprising a processor coupled to a memory (¶ 65, “The device 300 comprises a processor, 601, a memory, 602, and communication circuitry, 603”).
Eker does not teach; however, Melodia discloses: assigning a new resource allocation to at least one service in the set of services (¶ 16, “The action instructed may be configured to control, for non-limiting example, at the at least one network element, slicing of resources, selection of a scheduling policy, load balancing”) using a second machine learning model (¶ 11, “the second machine learning component may be configured to instruct the action”); and
a prediction component that predicts the new resource allocation based at least in part on the determined one or more states corresponding to the at least one service (¶ 16, “The RIC may further comprise at least one neural network or other type of machine learning component configured to produce a prediction or classification based on the representation”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of assigning a new resource allocation to at least one service in the set of services using a second machine learning model; and a prediction component that predicts the new resource allocation based at least in part on the determined one or more states corresponding to the at least one service, as taught by Melodia, in the same way to the at least one service, as taught by Eker. Both inventions are in the field of machine learning based resource allocation, and combining them would have predictably resulted in “scaling microservices in a service mesh,” as indicated by Melodia (¶ 1).
Regarding claim 5, Melodia teaches: The computer-implemented method of claim 1, wherein the prediction component comprises an actor network (¶ 14, “The second learning component may be a DRL agent that includes an actor neural network”) and the feedback component comprises a critic network (¶ 314, “a value network (scoring the actions taken by the actor network) implemented”).
Claims 8, 12, 15, and 19 recite commensurate subject matter as claims 1 and 5. Therefore, they are rejected for the same reasons.
Claim(s) 2, 7, 9, 14, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eker and Melodia, as applied above, and further in view of Yeh (US 2022/0014963).
Regarding claim 2, Eker and Melodia do not teach; however, Yeh discloses: the feedback component: evaluates the predicted new resource allocation using a loss function (¶ 144, “the critic network (θ.sup.Q) 905 is trained to minimize the following loss function (LF)”); and provides feedback (¶ 55, “The policy function (actor network 303) is the decision making entity with one or more tunable parameters, and the value function (critic network 305) evaluates the determined action a to produce feedback to help improve the training process for the policy function (actor network 303)”) for updating the prediction component according to a reward structure (¶ 57, “When the critic network 305 uses quantization techniques, it predicts the reward values produced using a particular policy”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of the feedback component: evaluates the predicted new resource allocation using a loss function; and provides feedback for updating the prediction component according to a reward structure, as taught by Yeh, in the same way to the feedback component, as taught by Eker and Melodia. Both inventions are in the field of machine learning based resource predictions, and combining them would have predictably resulted in a method to “minimize the risk of performance degradation during online learning, thereby improving network resource consumption efficiencies,” as indicated by Yeh (¶ 81).
Regarding claim 7, Eker and Melodia do not teach; however, Yeh discloses: the usage data is collected from one or more edge nodes that implement the set of services (¶ 41, “the agent 140 (or the iMTM 114) is responsible for deciding the traffic steering/splitting strategy (e.g., Action a) for individual multi-access UEs 101 and/or RANs/RAN nodes 130 in multi-access environment 100x based on the observation collected from the environment 100x (state s)” and “The reward r can be provided by one or more network elements from environment 100x, generated by the edge compute node 140, and/or by some other element/entity”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of the usage data is collected from one or more edge nodes that implement the set of services, as taught by Yeh, in the same way to the feedback component, as taught by Eker and Melodia. Both inventions are in the field of machine learning based resource predictions, and combining them would have predictably resulted in a method to “minimize the risk of performance degradation during online learning, thereby improving network resource consumption efficiencies,” as indicated by Yeh (¶ 81).
Claims 9, 14, 16, and 20 recite commensurate subject matter as claims 2 and 7. Therefore, they are rejected for the same reasons.
Claim(s) 3, 10, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eker, Melodia, and Yeh, as applied above, and further in view of Chen (US 2025/0013500).
Regarding claim 3, Eker, Melodia, and Yeh do not teach; however, Chen discloses: a dynamic reward boosting process (abstract, “a resource allocation policy is updated through gradient boosting, and a variance of a policy gradient is reduced by using an advantage function, to improve training efficiency”) comprising at least one of: a dynamic advantage function using static baseline rewards; and a static advantage function (¶ 29, “Aπθt(st,at)=Qπθt(st,at)-Vπθt(st) is an advantage function”) using dynamic baseline rewards (¶ 22, “the reward function: a DRL agent is guided to learn a better job scheduling policy with higher discounted long-term reward through the reward function”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of the reward structure is based on a dynamic reward boosting process comprising at least one of: a dynamic advantage function using static baseline rewards; and a static advantage function using dynamic baseline rewards, as taught by Yeh, in the same way to the reward structure, as taught by Eker, Melodia, and Yeh. Both inventions are in the field of machine learning based resource predictions, and combining them would have predictably resulted in “adaptive efficient resource allocation,” as indicated by Chen (¶ 1).
Claims 10 and 17 recite commensurate subject matter as claim 3. Therefore, they are rejected for the same reasons.
Claim(s) 4, 11, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eker and Melodia, as applied above, and further in view of Ayala Romero (US 2020/0296741).
Regarding claim 4, Eker and Melodia do not teach; however, Ayala Romero discloses: the first machine learning model comprises a feed-forward neural network (¶ 150, “A SAE includes two feedforward neural networks, an encoder (the one we use) e.sub.ξ and a decoder d.sub.ψ characterized by weights ξ and ψ, respectively. They are trained together so that a measure of the difference between the reconstructed output of the decoder and the input signal of the encoder x is minimal, d(y)=d(e(x))≈x”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of the first machine learning model comprises a feed-forward neural network, as taught by Ayala Romero, in the same way to the first machine learning model, as taught by Eker and Melodia. Both inventions are in the field of machine learning based resource allocation, and combining them would have predictably resulted in a method to “discover[ing] more complex, multi-modal structures,” as indicated by Ayala Romero (¶ 151).
Claims 11 and 18 recite commensurate subject matter as claim 4. Therefore, they are rejected for the same reasons.
Claim(s) 6 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eker and Melodia, as applied above, and further in view of Malvankar (US 2024/0220329).
Regarding claim 6, Eker and Melodia do not teach; however, Malvankar discloses: information indicating: whether the resource allocation can be improved for the corresponding service (¶ 70, “Method 600 includes identifying (606), based on retrieved computing resource details, which clusters can be downscaled using a trained deep reinforcement learning model”); and a priority level specified for the corresponding service (¶ 44, “Account priority 217 may include a priority designation (designated priority) for each user, e.g., low priority, medium priority, high priority, and so on”).
It would have been obvious to a person having ordinary skill in the art, at the effective filing date of the invention, to have applied the known technique of the first machine learning model comprises a feed-forward neural network, as taught by Malvankar, in the same way to the one or more states, as taught by Eker and Melodia. Both inventions are in the field of machine learning based resource allocation, and combining them would have predictably resulted in a method to “enable each computational workload or task to scale to threshold (e.g., maximum, minimum, etc.) limits,” as indicated by Malvankar (¶ 20).
Claim 13 recites commensurate subject matter as claim 6. Therefore, it is rejected for the same reason.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ma (US 2019/0286486) teaches “[d]ynamic and predictive resource allocation in both of the two forms for client applications is thus a key to reducing over allocation and waste of resources on one hand, and reducing under-allocation and loss of QoS for clients on the other hand” (¶ 17), which relates to the disclosed ML-driven scaling when service volume and resource demand vary over time.
Subramanian (US 2020/0104184) teaches “an artificial intelligence (AI) model or models that use a supervised or unsupervised reinforcement learning scheme to guide its suggestions of compute resources” (¶ 22), which relates to the disclosed ML based resource allocations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB D DASCOMB whose telephone number is (571)272-9993. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Vital can be reached at (571) 272-4215. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACOB D DASCOMB/ Primary Examiner, Art Unit 2198
Prosecution Insights