Detailed Action
1. This office action is in response to communication filed December 22, 2023. Claims 1-20 are currently pending and claims 1, 12, and 18 are the independent claims.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
3. Claims 13-17 are objected to because of the following informalities:
The claims 13-17 depend on independent claim 12 which discloses “a computing system”, but claims 13-17 only disclose a “system”. Please amend claims 13-17 to also disclose “a computing system”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
4. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation is: "the processing circuitry configured to..." in independent claim 12 and claims 13-17 that depend on independent claim 12.
Because this claim limitation is being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
5. Claim 9 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 9 discloses “determining an assignment of a workload to a second processing core of the plurality of processing cores based on real-time metrics” which does not make clear and obvious if the workload disclosed is the same workload or a different workload that is claimed in claim 1 which claim 9 depends on. Clarification by the Applicant is required moving forward.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
As per independent claim 1, the claim recites “A method comprising: applying, by a reinforcement learning agent, a policy model to a forecasted network traffic load associated with a workload to assign the workload to a first processing core of a plurality of processing cores of a computing device; and processing, by a virtual router and based on the assignment of the workload to the first processing core, network traffic for the workload using the first processing core.”
Under Step 2A, Prong I, the limitation "… assign the workload to a first processing core of a plurality of processing cores of a computing device" is a process that, under their broadest reasonable interpretation, covers performance of the limitation in the mind, but for the recitation of generic computer components. That is, other than reciting a "processing core", "learning agent" or "virtual router", nothing in the claim element precludes the step from practically being performed in a human mind or with the aid of pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the “Mental Processes” grouping of abstract ideas (concepts performed in the human mind including an observation, evaluation, judgment, and opinion). Accordingly, the claim recites an abstract idea.
These judicial exceptions are not integrated into a practical application. In particular under step 2A Prong II, claim 1 recites the additional element "applying, by a reinforcement learning agent, a policy model to a forecasted network traffic load associated with a workload …" This limitation amounts to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)). Claim 1 also recites the additional element "processing, by a virtual router and based on the assignment of the workload to the first processing core, network traffic for the workload using the first processing core." This limitation is a mere generic transmission and presentation of collected and analyzed data which is considered to be insignificant extra solution activity (MPEP 2106.05(g)). The "processing core", "learning agent" or "virtual router" in these steps are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts to no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. (see MPEP 2106.05(f)). The claim is directed to an abstract idea.
Under Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract ideas into a practical application, the limitations "applying..." and "processing..." are recognized by the courts as well-understood, routine, and conventional activities when they are claimed in a merely generic manner (see MPEP 2106.05(d)(II)(iv) Storing and retrieving information in memory, Versata Dev. Group Inc.). The additional elements that are well- understood, routine, conventional activity do not amount to significantly more, and are thus, not an inventive concept. Accordingly, the claim does not appear to be patent eligible under 35 U.S.C. 101.
As per claim 2, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “computing the forecasted network traffic load by: obtaining historical data associated with a plurality of workloads and the plurality of processing cores, wherein the plurality of workloads comprises the workload; and determining the forecasted network traffic load based on the historical data”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the computing of forecasted network traffic based on historical data, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 2 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 3, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “collecting state data associated with the plurality of processing cores and the workload; calculating a reward signal for the assignment of the workload to the first processing core based on a reward function and the state data; and updating the policy model based on the reward signal”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the state data collection to update the policy model, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 3 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 4, it incorporates the deficiencies of dependent claim 3 upon which it depends, and further recites, “wherein the state data comprises one or more of utilization of the plurality of processing cores, throughput of each processing core of the plurality of processing cores, a number of queues associated with each processing core of the plurality of processing cores, tail drops, jitter, or packet latency”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the state data collected on the workload, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 4 fails to correct the deficiencies of claim 3 and is rejected for similar reasoning as claim 3, above.
As per claim 5, it incorporates the deficiencies of dependent claim 3 upon which it depends, and further recites, “wherein the reward function is defined to one or more of minimize latency associated with an average time the first processing core processes the network traffic for the workload, minimize a number of idle processing cores of the plurality of processing cores, minimize a number of packet drops, or maximize overall throughput of the plurality of processing cores”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the reward function’s purpose, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 5 fails to correct the deficiencies of claim 3 and is rejected for similar reasoning as claim 3, above.
As per claim 6, it incorporates the deficiencies of dependent claim 3 upon which it depends, and further recites, “wherein updating the policy model comprises applying a reinforcement learning algorithm comprising a policy gradient method”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the policy model’s updating based on a policy gradient method, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 6 fails to correct the deficiencies of claim 3 and is rejected for similar reasoning as claim 3, above.
As per claim 7, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “wherein the policy model is trained with one or more of historical assignment data, historical throughput data, usage of the plurality of processing cores, type of the plurality of processing cores, a requirement of the workload, type of the workload, or a profile associated with the workload”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the policy model’s training inputs, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 7 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 8, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “wherein the policy model comprises a neural network”, which conceptually, with broadest reasonable interpretation, merely provides further clarification as to the policy model being a neural network, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 8 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 9, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “determining an assignment of a workload to a second processing core of the plurality of processing cores based on real-time metrics; and selecting the assignment of the workload to the first processing core rather than the assignment of the workload to the second processing core, wherein processing the network traffic for the workload using the first processing core is based on the selecting”, which conceptually, merely provides further clarification as to the assignment of a workload to a specific processing core, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 9 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 10, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “wherein the reinforcement learning agent is executed by one of the computing device or a controller for a virtualized computing infrastructure that includes the computing device”, which conceptually, merely provides further clarification as to the execution handling of the reinforcement learning agent by a computing device or virtualized computing, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 10 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
As per claim 11, it incorporates the deficiencies of independent claim 1 upon which it depends, and further recites, “assigning, by the virtual router, a queue to the first processing core; enqueueing, based on the assigning of network traffic processing for the workload to the first processing core, the network traffic for the workload to the queue; and obtaining the network traffic for the workload based on the queue, prior to processing the network traffic for the workload”, which conceptually, merely provides further clarification as to the queueing nature of network traffic processing, which does not integrate the abstract idea/mental process into a practical application and is not significantly more than the abstract idea/mental process. Therefore, claim 11 fails to correct the deficiencies of claim 1 and is rejected for similar reasoning as claim 1, above.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 1 above other than being a computing system claim rather than a method claim. Claim 12 is rejected under 35 U.S.C. 101 for the same reasons as claim 1 above.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 2 above other than being a system claim rather than a method claim. Claim 13 is rejected under 35 U.S.C. 101 for the same reasons as claim 2 above.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 3 above other than being a system claim rather than a method claim. Claim 14 is rejected under 35 U.S.C. 101 for the same reasons as claim 3 above.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 5 above other than being a system claim rather than a method claim. Claim 15 is rejected under 35 U.S.C. 101 for the same reasons as claim 5 above.
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 6 above other than being a system claim rather than a method claim. Claim 16 is rejected under 35 U.S.C. 101 for the same reasons as claim 6 above.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 9 above other than being a system claim rather than a method claim. Claim 17 is rejected under 35 U.S.C. 101 for the same reasons as claim 9 above.
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 1 above other than being a computer-readable storage media claim rather than a method claim. Claim 18 is rejected under 35 U.S.C. 101 for the same reasons as claim 1 above.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 2 above other than being a computer-readable storage media claim rather than a method claim. Claim 19 is rejected under 35 U.S.C. 101 for the same reasons as claim 2 above.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites the same claimed language as claim 3 above other than being a computer-readable storage media claim rather than a method claim. Claim 20 is rejected under 35 U.S.C. 101 for the same reasons as claim 3 above.
7. Claims 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because the claims are directed to software per se because claim 17 may cover transitory media. The recitation of “a computer-readable storage media” in claim 20 can be interpreted as transitory media because the Specification of the application includes paragraph [0117] which discloses "In some examples, the computer-readable storage media may comprise non-transitory media."
Therefore, claim 18 is considered as software per se claim. Additionally, dependent claims 19-20 which depend on claim 18 od not provide any non-software structure not previously seen in independent claim 18, so claims 18-20 are rejected under 35 U.S.C. 101 as directing to software per se. The Examiner recommends the Applicant amends the claims to include non-transitory consistent with the Specification.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (U.S. Pub. No. 2022/0329539) – hereinafter “Kim” in view of Xue et al. (U.S. Pub. No. 2024/0054020) – hereinafter “Xue”.
Regarding independent claim 1, Kim discloses:
A method comprising:
applying, by a reinforcement learning agent, a policy model to a forecasted network traffic load associated with a workload … (Fig. 2, M2 and Predicted Traffic 233 and [0041] “Alternatively, the second AI model includes a multi-agent deep reinforcement learning model and is trained to adjust the allocation of the computing resources in the server using, as input values, the predicted traffic, status information of the computing resources in the server, and the status information of the computing resources in the at least one associated server.” and [0011] “According to an example embodiment of the disclosure, a method, performed by a server, of adjusting allocation of computing resources to a plurality of VNFs includes: identifying, for processing at least one task related to user equipments (UEs) connected to the server, a plurality of VNFs related to the task; obtaining predicted traffic expected to be generated in the server by processing the task via the plurality of VNFs; obtaining, from at least one associated server, status information of computing resources in the at least one associated server; and adjusting allocation of computing resources to the plurality of VNFs based on the status information of the computing resources in the at least one associated server and the predicted traffic. The at least one associated server includes another server that processes the task related to the UEs connected to the server.” and [0063] “In the disclosure, a “computing resource” may refer to a CPU as a resource for computing. For example, a millisecond (ms) may be used as a unit to define computing resources, and about 1000 ms of computing resources may constitute 1 vCore (unit virtual CPU core).”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the learning model uses predicted traffic from processing a task to allocate a CPU to VNFs.
processing, by a virtual router and based on the assignment of the workload to the first processing core, network traffic for the workload using the first processing core. ([0011] “processing the task via the plurality of VNFs;” [0056] “a VNF may include one or more VMs for routing”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the network traffic/tasks are processed via VNFs/VMs.
Kim does not explicitly disclose:
… to assign the workload to a first processing core of a plurality of processing cores of a computing device;
However, Xue discloses:
… to assign the workload to a first processing core of a plurality of processing cores of a computing device; ([0083] “This step can correspond to the CPU utilization prediction module in FIG. 2. It can be learned from the above-mentioned concept of unit workload that, if configuration of the computing resource share remains unchanged and traffic changes, the unit workload changes, and therefore, the CPU utilization of the computing resource changes. Therefore, a relationship between workload (traffic) of an application and CPU utilization needs to be mined. This mapping relationship is heterogeneous. For example, (1) for different applications, mapping from workload to CPU utilization is different; (2) for a same application, there is different correlation between a subtype of workload and CPU utilization. In view of this, in this specification, an idea of meta-learning is used for reference, to train a general model for all tasks. The model maps workload to CPU utilization based on commonness and a difference between tasks.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the workload has a direct relationship with CPU utilization based on assigning specific workloads to specific CPU allocations.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add to assign the workload to a first processing core of a plurality of processing cores of a computing device as seen in Xue's invention into Kim's invention because these modifications allow the use of a known technique to improve similar devices in the same way such that the workload has a direct relationship with CPU utilization based on assigning specific workloads to specific CPU allocations so that a workload can be contained to a singular processing core.
Regarding claim 2, Kim discloses the method of claim 1, further comprising computing the forecasted network traffic load by:
obtaining historical data associated with a plurality of workloads and the plurality of processing cores, wherein the plurality of workloads comprises the workload; and (Fig. 2, 232 Traffic History Information and [0011] “According to an example embodiment of the disclosure, a method, performed by a server, of adjusting allocation of computing resources to a plurality of VNFs includes: identifying, for processing at least one task related to user equipments (UEs) connected to the server, a plurality of VNFs related to the task; obtaining predicted traffic expected to be generated in the server by processing the task via the plurality of VNFs; obtaining, from at least one associated server, status information of computing resources in the at least one associated server; and adjusting allocation of computing resources to the plurality of VNFs based on the status information of the computing resources in the at least one associated server and the predicted traffic. The at least one associated server includes another server that processes the task related to the UEs connected to the server.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the historical data associated with the tasks of the workload performed via the plurality of VNFs is obtained for future usage.
determining the forecasted network traffic load based on the historical data. ([0107] “The DB 353 included in the storage 350 may include a collection of massive amounts of data. In various example embodiments, the DB 353 may include an adjustment history DB 353-1 related to an allocation adjustment history of the computing resources 310. In various example embodiments, the DB 353 may be used in an operation of obtaining the predicted traffic or an operation of training and refining the AI model 370. All pieces of data obtained in a method, performed by a server, of adjusting allocation of computing resources for a plurality of VNFs by the server according to various embodiments may be stored in the DB 353, and thus, the obtained data may be used in a recursive manner.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the predicted traffic is determined based on the historical data found in the adjustment history DB.
Regarding claim 3, Kim discloses the method of claim 1, further comprising:
collecting state data associated with the plurality of processing cores and the workload; (Fig. 2, Computing Resource Status Information 234 and [0088] “In an example embodiment, the second AI model M2 may further include the status information 234 of the computing resources 210 in the server 200 itself as an input value. For example, the status information 234 of the computing resources 210 may include at least one of information about allocation of the computing resources 210 to the VNFs 251 in the server 200, information about an occupancy rate of the computing resources 210 in the server 200, information related to whether CPU cores in the server 200 are turned on or off, or information about a clock speed of the CPU cores in the server 200.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the computing resource status information is data including allocation information to the VNFs or CPU core information.
calculating a reward signal for the assignment of the workload to the first processing core based on a reward function and the state data; and ([0087] “In various example embodiments, the second AI model M2 may be an AI model for adjusting allocation of the computing resources 210 to the VNFs 251. For example, the second AI model M2 may include a multi-agent deep reinforcement learning model. Reinforcement learning is an area of machine learning in which an agent defined in an environment recognizes a current state and selects an action or sequence of actions that maximizes rewards among the selectable actions. Models built through reinforcement learning may be used in multi-agent systems or optimization control theory. Thus, the multi-agent deep reinforcement learning model may be effectively used to improve or optimize adjustment of computing resource allocation while having multiple inputs, as in the present disclosure.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI model is updated based on its reinforcement learning that rewards correct actions among a plurality of possible actions.
updating the policy model based on the reward signal. ([0087] “In various example embodiments, the second AI model M2 may be an AI model for adjusting allocation of the computing resources 210 to the VNFs 251. For example, the second AI model M2 may include a multi-agent deep reinforcement learning model. Reinforcement learning is an area of machine learning in which an agent defined in an environment recognizes a current state and selects an action or sequence of actions that maximizes rewards among the selectable actions. Models built through reinforcement learning may be used in multi-agent systems or optimization control theory. Thus, the multi-agent deep reinforcement learning model may be effectively used to improve or optimize adjustment of computing resource allocation while having multiple inputs, as in the present disclosure.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI model is updated based on its reinforcement learning that rewards correct actions among a plurality of possible actions.
Regarding claim 4, Kim discloses the method of claim 3, wherein the state data comprises one or more of utilization of the plurality of processing cores, throughput of each processing core of the plurality of processing cores, a number of queues associated with each processing core of the plurality of processing cores, tail drops, jitter, or packet latency. ([0036] “Alternatively, the status information of the computing resources in the at least one associated server includes at least one of information about allocation of the computing resources to a plurality of VNFs in the at least one associated server, information about an occupancy rate of the computing resources in the at least one associated server, information related to whether central processing unit (CPU) cores in the at least one associated server are turned on or off, or information about a clock speed of the CPU cores in the at least one associated server.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the state data includes CPU core utilization/occupancy rate.
Regarding claim 5, Kim discloses the method of claim 3, wherein the reward function is defined to one or more of minimize latency associated with an average time the first processing core processes the network traffic for the workload, minimize a number of idle processing cores of the plurality of processing cores, minimize a number of packet drops, or maximize overall throughput of the plurality of processing cores. ([0087] “In various example embodiments, the second AI model M2 may be an AI model for adjusting allocation of the computing resources 210 to the VNFs 251. For example, the second AI model M2 may include a multi-agent deep reinforcement learning model. Reinforcement learning is an area of machine learning in which an agent defined in an environment recognizes a current state and selects an action or sequence of actions that maximizes rewards among the selectable actions. Models built through reinforcement learning may be used in multi-agent systems or optimization control theory. Thus, the multi-agent deep reinforcement learning model may be effectively used to improve or optimize adjustment of computing resource allocation while having multiple inputs, as in the present disclosure.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the selection of the best action among a plurality of possible actions is rewarded because of the focus on improving computing resource allocation to minimize network traffic latency time.
Regarding claim 6, Kim discloses the method of claim 3, wherein updating the policy model comprises applying a reinforcement learning algorithm comprising a policy gradient method. ([0077] “In various example embodiments, the first AI model M1 is an AI model for obtaining the predicted traffic 233 expected to be generated in the server 200, and may include a DNN. A DNN refers to a type of artificial neural network, and may have a characteristic of including several hidden layers between an input layer and an output layer. DNNs may model complex nonlinear relationships like general artificial neural networks. For example, in a DNN architecture for an object detection model, each object may be expressed as a layered composition of image primitives. In this case, additional layers may gather features gradually propagated from lower layers. These characteristics of a DNN enable modeling of complex data with fewer units. A DNN may be used to obtain a future predicted value as described herein in the present disclosure. For example, the first AI model M1 may include a DNN with a long short-term memory (LSTM) structure.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI model M1 includes an artificial neural network that gradually gathers data from lower layers of each object similar to a gradient of learning.
Regarding claim 7, Kim discloses the method of claim 1, wherein the policy model is trained with one or more of historical assignment data, historical throughput data, usage of the plurality of processing cores, type of the plurality of processing cores, a requirement of the workload, type of the workload, or a profile associated with the workload. ([0088] “In various example embodiments, the second AI model M2 may be trained to adjust allocation of the computing resources 210 in the server 200 (236) using, as input values, the predicted traffic 233 and the status information 235 of the computing resources 210R in the associated server 200R. In an example embodiment, the second AI model M2 may further include the status information 234 of the computing resources 210 in the server 200 itself as an input value. For example, the status information 234 of the computing resources 210 may include at least one of information about allocation of the computing resources 210 to the VNFs 251 in the server 200, information about an occupancy rate of the computing resources 210 in the server 200, information related to whether CPU cores in the server 200 are turned on or off, or information about a clock speed of the CPU cores in the server 200.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI model M2 is trained with assignment and state data/status information including computing resource allocations.
Regarding claim 8, Kim discloses the method of claim 1, wherein the policy model comprises a neural network. ([0077] “In various example embodiments, the first AI model M1 is an AI model for obtaining the predicted traffic 233 expected to be generated in the server 200, and may include a DNN. A DNN refers to a type of artificial neural network, and may have a characteristic of including several hidden layers between an input layer and an output layer.” and [0087] “In various example embodiments, the second AI model M2 may be an AI model for adjusting allocation of the computing resources 210 to the VNFs 251. For example, the second AI model M2 may include a multi-agent deep reinforcement learning model. Reinforcement learning is an area of machine learning in which an agent defined in an environment recognizes a current state and selects an action or sequence of actions that maximizes rewards among the selectable actions. Models built through reinforcement learning may be used in multi-agent systems or optimization control theory. Thus, the multi-agent deep reinforcement learning model may be effectively used to improve or optimize adjustment of computing resource allocation while having multiple inputs, as in the present disclosure.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI models M1 and M22 are neural networks.
Regarding claim 9, Kim discloses the method of claim 1, but does not explicitly disclose:
determining an assignment of a workload to a second processing core of the plurality of processing cores based on real-time metrics; and
selecting the assignment of the workload to the first processing core rather than the assignment of the workload to the second processing core,
wherein processing the network traffic for the workload using the first processing core is based on the selecting.
However, Xue discloses:
determining an assignment of a workload to a second processing core of the plurality of processing cores based on real-time metrics; and (Fig. 3 and [0106] “To adjust the computing resource configuration policy, at least a neural network that obtains the representation vector through coding and decoding can be adjusted, to repeat step 301, step 302, and step 303. Usually, in a model training phase, CPU utilization of each application and a corresponding computing resource configuration share can be used as a decision label, and an evaluation label of the policy evaluation network is determined based on a gap between CPU utilization corresponding to a decision and target CPU utilization and a cost of adjusting the computing resource share, to determine a model loss. Then, each model parameter in the CPU utilization prediction module and the scaling decision making module is adjusted and computed with an objective of reducing the model loss. After model training is completed, the parameter Ψ of the decision network is determined, and a network parameter corresponding to the representation vector z can be adjusted with an objective of maximizing a long-term reward based on a policy evaluation module, so as to adjust the representation vector z, so that the representation vector z better represents a task category, to further adjust the output result of the decision network, and adjust the computing resource configuration policy through forward propagation.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the application workload is determined to be assigned to a target CPU of the multitude of CPUs in the computing resource configuration share.
selecting the assignment of the workload to the first processing core rather than the assignment of the workload to the second processing core, ([0106] “To adjust the computing resource configuration policy, at least a neural network that obtains the representation vector through coding and decoding can be adjusted, to repeat step 301, step 302, and step 303. Usually, in a model training phase, CPU utilization of each application and a corresponding computing resource configuration share can be used as a decision label, and an evaluation label of the policy evaluation network is determined based on a gap between CPU utilization corresponding to a decision and target CPU utilization and a cost of adjusting the computing resource share, to determine a model loss. Then, each model parameter in the CPU utilization prediction module and the scaling decision making module is adjusted and computed with an objective of reducing the model loss. After model training is completed, the parameter Ψ of the decision network is determined, and a network parameter corresponding to the representation vector z can be adjusted with an objective of maximizing a long-term reward based on a policy evaluation module, so as to adjust the representation vector z, so that the representation vector z better represents a task category, to further adjust the output result of the decision network, and adjust the computing resource configuration policy through forward propagation.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the workload is assigned based upon the CPU utilization performed by the computing resource configuration policy’s model to maximize long-term rewards for efficient processing.
wherein processing the network traffic for the workload using the first processing core is based on the selecting. ([0106] “To adjust the computing resource configuration policy, at least a neural network that obtains the representation vector through coding and decoding can be adjusted, to repeat step 301, step 302, and step 303. Usually, in a model training phase, CPU utilization of each application and a corresponding computing resource configuration share can be used as a decision label, and an evaluation label of the policy evaluation network is determined based on a gap between CPU utilization corresponding to a decision and target CPU utilization and a cost of adjusting the computing resource share, to determine a model loss. Then, each model parameter in the CPU utilization prediction module and the scaling decision making module is adjusted and computed with an objective of reducing the model loss. After model training is completed, the parameter Ψ of the decision network is determined, and a network parameter corresponding to the representation vector z can be adjusted with an objective of maximizing a long-term reward based on a policy evaluation module, so as to adjust the representation vector z, so that the representation vector z better represents a task category, to further adjust the output result of the decision network, and adjust the computing resource configuration policy through forward propagation.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the workload is assigned based upon the CPU utilization performed by the computing resource configuration policy’s model to maximize long-term rewards for efficient processing.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add determining an assignment of a workload to a second processing core of the plurality of processing cores based on real-time metrics and selecting the assignment of the workload to the first processing core rather than the assignment of the workload to the second processing core, wherein processing the network traffic for the workload using the first processing core is based on the selecting as seen in Xue's invention into Kim's invention because these modifications allow applying a known technique to a known device ready for improvement to yield predictable results such that the determination of which processing core a workload is assigned to is based on current utilization of a plurality of processing cores and network traffic is processed following assignment.
Regarding claim 10, Kim discloses the method of claim 1, wherein the reinforcement learning agent is executed by one of the computing device or a controller for a virtualized computing infrastructure that includes the computing device. (Fig. 2 AI Model M2 run via Server 200 comprising Processor 230 and Computing Resources 210) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the AI model M2 executes on server resources including computing devices.
Regarding claim 11, Kim discloses the method of claim 1, further comprising:
assigning, by the virtual router, a queue to the first processing core; ([0062-0063] “In the disclosure, “waiting traffic” may refer to traffic waiting in a buffer of a server. For example, the “waiting traffic” may refer to a workload associated with tasks waiting in the buffer to be processed. A “buffer” may refer to an area of memory that temporarily stores data while transferring the data from one place to another in a computing operation. The buffer may also be referred to as a ‘queue’ or a ‘waiting queue’. In the disclosure, a “computing resource” may refer to a CPU as a resource for computing. For example, a millisecond (ms) may be used as a unit to define computing resources, and about 1000 ms of computing resources may constitute 1 vCore (unit virtual CPU core).”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the queue/buffer containing the waiting traffic transfers data from memory to CPUs to be processed.
enqueueing, based on the assigning of network traffic processing for the workload to the first processing core, the network traffic for the workload to the queue; and ([0062] “In the disclosure, “waiting traffic” may refer to traffic waiting in a buffer of a server. For example, the “waiting traffic” may refer to a workload associated with tasks waiting in the buffer to be processed. A “buffer” may refer to an area of memory that temporarily stores data while transferring the data from one place to another in a computing operation. The buffer may also be referred to as a ‘queue’ or a ‘waiting queue’.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the traffic is enqueued on a buffer while waiting to be processed.
obtaining the network traffic for the workload based on the queue, prior to processing the network traffic for the workload. ([0062] “In the disclosure, “waiting traffic” may refer to traffic waiting in a buffer of a server. For example, the “waiting traffic” may refer to a workload associated with tasks waiting in the buffer to be processed. A “buffer” may refer to an area of memory that temporarily stores data while transferring the data from one place to another in a computing operation. The buffer may also be referred to as a ‘queue’ or a ‘waiting queue’.”) The citation is interpreted to read on the claimed invention because under broadest reasonable interpretation, the traffic is enqueued on a buffer while waiting to be processed and then is transferred from buffer to processing computing resource.
Regarding independent claim 12, it is a computing system claim having the same limitations as cited in method claim 1. Thus, claim 12 is also rejected under the same rationale as addressed in the rejection of claim 1 above.
Regarding claim 13, it is a computing system claim having the same limitations as cited in method claim 2. Thus, claim 13 is also rejected under the same rationale as addressed in the rejection of claim 2 above.
Regarding claim 14, it is a computing system claim having the same limitations as cited in method claim 3. Thus, claim 14 is also rejected under the same rationale as addressed in the rejection of claim 3 above.
Regarding claim 15, it is a computing system claim having the same limitations as cited in method claim 5. Thus, claim 15 is also rejected under the same rationale as addressed in the rejection of claim 5 above.
Regarding claim 16, it is a computing system claim having the same limitations as cited in method claim 6. Thus, claim 16 is also rejected under the same rationale as addressed in the rejection of claim 6 above.
Regarding claim 17, it is a computing system claim having the same limitations as cited in method claim 9. Thus, claim 17 is also rejected under the same rationale as addressed in the rejection of claim 9 above.
Regarding independent claim 18, it is a computer-readable storage media claim having the same limitations as cited in method claim 1. Thus, claim 18 is also rejected under the same rationale as addressed in the rejection of claim 1 above.
Regarding claim 19, it is a computer-readable storage media claim having the same limitations as cited in method claim 2. Thus, claim 19 is also rejected under the same rationale as addressed in the rejection of claim 2 above.
Regarding claim 20, it is a computer-readable storage media claim having the same limitations as cited in method claim 3. Thus, claim 20 is also rejected under the same rationale as addressed in the rejection of claim 3 above.
Conclusion
9. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Such prior art includes Palermo et al. (U.S. Patent No. 11,650,851) which discloses handling network traffic with edge server computing processing which can virtualize router functions and Spain et al. (U.S. Pub. No. 2024/0362047) which discloses processing historical IP packet data via a trained ML model to instantiate virtual machines including virtual networking devices such as virtual routers.
Examiner has cited particular columns/paragraphs/sections and line numbers in the references applied and not relied upon to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
When responding to the Office action, applicant is advised to clearly point out the patentable novelty the claims present in view of the state of the art disclosed by the reference(s) cited or the objections made. A showing of how the amendments avoid such references or objections must also be present. See 37 C.F.R. 1.111(c).
When responding to this Office action, applicant is advised to provide the line and page numbers in the application and/or reference(s) cited to assist in locating the appropriate paragraphs.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL B TRAINOR whose telephone number is (571)272-3710. The examiner can normally be reached Monday-Friday 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Vital can be reached at (571) 272-4215. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T./Examiner, Art Unit 2198
/PIERRE VITAL/Supervisory Patent Examiner, Art Unit 2198
Prosecution Insights