DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 06/25/2024 for application number 18/754,007. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1-23 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 2A Prong 1:
Claim 1 recites:
“Performing… genetic operations on the initial set of solutions to produce the evolved set of solutions;” Performing genetic operations is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A computer-implemented method for producing an evolved set of solutions to a combinatorial problem;” “at a reinforcement learning model; by the reinforcement learning model;” “at a genetic solution processor;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“Receiving… a problem definition and problem constraints for the combinatorial problem;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“processing the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A computer-implemented method for producing an evolved set of solutions to a combinatorial problem;” “at a reinforcement learning model; by the reinforcement learning model;” “at a genetic solution processor;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“Receiving… a problem definition and problem constraints for the combinatorial problem;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“processing the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 2
Step 2A Prong 1:
Claim 2 recites:
“removing a portion of the evolved set of solutions to produce a reduced set of solutions;” Removing a portion of solutions is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“inserting additional solutions into the reduced set of solutions to produce a second set of solutions;” Inserting additional solutions is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“performing additional genetic operations on the second set of solutions to produce a further evolved set of solutions;” Performing additional genetic operations is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 3
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the additional solutions are generated by the reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the additional solutions are generated by the reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 2A Prong 1:
Claim 4 recites:
“wherein the additional solutions are generated by a genetic operation including at least one of mutation or crossover;” Generating by at least mutation or crossover is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 5
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the combinatorial problem is a vehicle routing problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the combinatorial problem is a vehicle routing problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the problem definition comprises delivery locations and the problem constraints comprise at least one of vehicle capacity, delivery load information, demand at each drop-point, delivery and pickup time windows, delivery priorities, and pickup and drop-point pairs;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the problem definition comprises delivery locations and the problem constraints comprise at least one of vehicle capacity, delivery load information, demand at each drop-point, delivery and pickup time windows, delivery priorities, and pickup and drop-point pairs;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 2A Prong 1:
Claim 7 recites:
“wherein processing the problem definition selects one or more solutions having at least one of a minimal combined delivery time duration, a minimal distance travelled, or a minimal number of delivery vehicles as the initial set of solutions to the problem;” Selecting solutions having at least one of a minimal combined delivery time duration, a minimal distance travelled, or a minimal number of delivery vehicles as the initial set of solutions to the problem is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 8
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“configuring one or more vehicles to implement at least one solution in the evolved set of solutions;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“configuring one or more vehicles to implement at least one solution in the evolved set of solutions;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 2A Prong 1:
Claim 9 recites:
“wherein the reinforcement learning model implements a beam search to compute the initial set of solutions by: generating a set of partial solutions;” Implementing a beam search to compute solutions by generating a set of partial solutions is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
“selecting a subset of the partial solutions based on shared sequences of traversed destinations (shared segments) as a diverse set of partial solutions;” Selecting a subset of the partial solutions based on shared sequences of traversed destinations is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“extending the partial solutions in the diverse set of partial solutions to generate another set of partial solutions;” Extending the partial solutions is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“repeating the selecting and extending until the initial set of solutions is complete;” Repeating the selecting and extending is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 10
Step 2A Prong 1:
Claim 10 recites:
“performs genetic operations on the initial set of solutions to produce the evolved set of solutions;” Performing genetic operations is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A system for producing an evolved set of solutions to a combinatorial problem, comprising: a memory that stores a problem definition and problem constraints for the combinatorial problem; and a processor that is connected to the memory;” “by a reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
wherein the processor: processes the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A system for producing an evolved set of solutions to a combinatorial problem, comprising: a memory that stores a problem definition and problem constraints for the combinatorial problem; and a processor that is connected to the memory;” “by a reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
wherein the processor: processes the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
System claim 11 recites similar limitations to method claim 2. Therefore, claim 11 is rejected using the same rationale as claim 2.
System claim 12 recites similar limitations to method claim 9. Therefore, claim 12 is rejected using the same rationale as claim 9.
Claim 13
Step 2A Prong 1:
Claim 13 recites:
“performing genetic operations on the initial set of solutions to produce the evolved set of solutions;” Performing genetic operations is a claim limitation that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A non-transitory computer-readable media storing computer instructions for producing an evolved set of solutions to a combinatorial problem that, when executed by one or more processors, cause the one or more processors to perform the steps;” “by a reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“receiving the problem definition and problem constraints for the combinatorial problem;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“processing the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A non-transitory computer-readable media storing computer instructions for producing an evolved set of solutions to a combinatorial problem that, when executed by one or more processors, cause the one or more processors to perform the steps;” “by a reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“receiving the problem definition and problem constraints for the combinatorial problem;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“processing the problem definition and the problem constraints… to generate an initial set of solutions to the combinatorial problem;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Non-transitory computer-readable media claim 14 recites similar limitations to method claim 9. Therefore, claim 14 is rejected using the same rationale as claim 9.
Claim 15
Step 2A Prong 1:
Claim 15 recites:
“selecting a subset of the partial solutions based on shared sequences of traversed destinations as a diverse set of partial solutions;” Selecting a subset of solutions is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“extending the partial solutions in the diverse set of partial paths to generate an additional set of partial solutions;” Extending solutions to generate additional solutions is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“repeating the selecting and extending until the additional set of solutions is complete;” Repeating the selecting and extending is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A computer-implemented method for producing a set of solutions to a problem using a reinforcement learning model;” “by the reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“receiving the problem definition, problem constraints, and parameters;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
“processing the problem definition and problem constraints, according to the parameters… to generate a set of partial solutions;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
“outputting the additional set of partial solutions as the set of solutions;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A computer-implemented method for producing a set of solutions to a problem using a reinforcement learning model;” “by the reinforcement learning model;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“receiving the problem definition, problem constraints, and parameters;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
“processing the problem definition and problem constraints, according to the parameters… to generate a set of partial solutions;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
“outputting the additional set of partial solutions as the set of solutions;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the additional set of solutions is complete when all of the destinations are traversed;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the additional set of solutions is complete when all of the destinations are traversed;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 17
Step 2A Prong 1:
Claim 17 recites:
“identifying a best partial solution based on performance criteria; and selecting the best partial solution and at least one additional partial solution that does not share at least a portion of the sequence of traversed destination in the best partial solution for inclusion in the subset;” Identifying a solution based on performance criteria and selecting a best solution and at least one additional solution that does not share at least a portion of the sequence of traversed destination in the best partial solution are actions that can be performed mentally with the aid of pen and paper, and are therefore a mental processes.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 18
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the reinforcement learning model is trained using an objective that maximizes a return;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the reinforcement learning model is trained using an objective that maximizes a return;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 19 recites similar limitations to claim 8. Therefore, claim 19 is rejected using the same rationale as claim 8.
Claim 20
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed on a server or in a data center to generate the set of solutions;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“the set of solutions is streamed to a user device;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed on a server or in a data center to generate the set of solutions;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“the set of solutions is streamed to a user device;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 21
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed within a cloud computing environment;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed within a cloud computing environment;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 22
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 23 recites similar limitations to claim 22. Therefore, claim 23 is rejected using the same rationale as claim 22.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3, 4, 10, and 13 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by VENUGOPAL MINIMOL et al. (US 20230306271 A1), hereinafter Venugopal.
Regarding claim 1, Venugopal teaches,
A computer-implemented method for producing an evolved set of solutions to a combinatorial problem [Abstract, Embodiments herein provide a method and system for evolved State-Action-Reward-State-Action (evolved SARSA) RL for flow shop scheduling, which is a hybrid framework of hierarchical RL with evolutionary techniques and heuristics method to solve FSSP… The framework refines FSSP solution provided by Reinforced-SARSA (R-SARSA) using the evolutionary Genetic Algorithms (GAs)], comprising:
receiving, at a reinforcement learning model, a problem definition (Para 0005, a plurality of jobs to be sequenced during flow shop scheduling) and problem constraints (Para 0005, obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan) for the combinatorial problem [Para 0005, a method for evolved State-Action-Reward-State-Action (evolved SARSA) reinforcement learning for flow shop scheduling is provided. The method includes receiving a plurality of jobs to be sequenced during flow shop scheduling on a plurality of machines. Further, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan];
processing the problem definition and the problem constraints by the reinforcement learning model to generate an initial set of solutions (Para 0005, a plurality parent job sequence pairs) to the combinatorial problem [Para 0005, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan, by iteratively performing steps comprising: (A) obtaining via a Reinforced State-Action-Reward-State-Action (R-SARSA) learning module a current job sequence of the plurality of jobs. (B) Identifying the current job sequence as a best job sequence if a current makespan estimated for the current job sequence is less than a preceding makespan computed for a previous iteration. The current makespan is identified as a best makespan, and if the current makespan is greater than the preceding makespan, a policy learnt by the R-SARSA learning module in the previous iteration is unlearnt and is assigned a penalty for a reward to prevent from drifting away from obtaining the optimum job sequence, and wherein a first makespan estimated in a first iteration is compared with an initial preset best makespan. (C) Storing the identified best job sequence in a repository, wherein the repository stores a plurality of best sequences identified in each iteration. (D) Generating via an evolutionary crossover module a plurality parent job sequence pairs, randomly picked from the plurality of best sequences]; and
performing, at a genetic solution processor, genetic operations on the initial set of solutions to produce the evolved set of solutions [Para 0005, Processing via the evolutionary crossover module each of the plurality of parent job sequence pairs, to generate a plurality of evolved job sequences using genetic operators].
Regarding claim 3, Venugopal teach the limitations of claim 1.
Venugopal further teaches,
wherein the additional solutions are generated by the reinforcement learning model [Para 0027, the system 100, interchangeably referred herein as evolved SARSA system; Para 0033, FIGS. 2A through 2C (collectively referred as FIG. 2) is a flow diagram illustrating a method 200 for evolved SARSA reinforcement learning for flow shop scheduling, using the system of FIG. 1; Para 0046, At step 204e the evolutionary crossover module processes each of the plurality of parent job sequence pairs, to generate a plurality of evolved job sequences using genetic operators].
Regarding claim 4, Venugopal teach the limitations of claim 1.
Venugopal further teaches,
wherein the additional solutions are generated by a genetic operation including at least one of mutation or crossover [Para 0046, At step 204e the evolutionary crossover module processes each of the plurality of parent job sequence pairs, to generate a plurality of evolved job sequences using genetic operators].
Regarding claim 10, Venugopal further teaches,
A system for producing an evolved set of solutions to a combinatorial problem [Para 0007, a system for evolved State-action-reward-state-action (evolved SARSA) reinforcement learning for flow shop scheduling is provided], comprising:
a memory that stores a problem definition and problem constraints for the combinatorial problem [Para 0005, a method for evolved State-Action-Reward-State-Action (evolved SARSA) reinforcement learning for flow shop scheduling is provided. The method includes receiving a plurality of jobs to be sequenced during flow shop scheduling on a plurality of machines. Further, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan; Para 0007, The system comprises a memory storing instructions]; and
a processor that is connected to the memory [Para 0007, one or more hardware processors coupled to the memory], wherein the processor: processes the problem definition and the problem constraints by a reinforcement learning model to generate an initial set of solutions to the combinatorial problem [Para 0005, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan, by iteratively performing steps comprising: (A) obtaining via a Reinforced State-Action-Reward-State-Action (R-SARSA) learning module a current job sequence of the plurality of jobs. (B) Identifying the current job sequence as a best job sequence if a current makespan estimated for the current job sequence is less than a preceding makespan computed for a previous iteration. The current makespan is identified as a best makespan, and if the current makespan is greater than the preceding makespan, a policy learnt by the R-SARSA learning module in the previous iteration is unlearnt and is assigned a penalty for a reward to prevent from drifting away from obtaining the optimum job sequence, and wherein a first makespan estimated in a first iteration is compared with an initial preset best makespan. (C) Storing the identified best job sequence in a repository, wherein the repository stores a plurality of best sequences identified in each iteration. (D) Generating via an evolutionary crossover module a plurality parent job sequence pairs, randomly picked from the plurality of best sequences]; and
performs genetic operations on the initial set of solutions to produce the evolved set of solutions [Para 0005, Processing via the evolutionary crossover module each of the plurality of parent job sequence pairs, to generate a plurality of evolved job sequences using genetic operators].
Regarding claim 13, Venugopal teaches,
A non-transitory computer-readable media storing computer instructions for producing an evolved set of solutions to a combinatorial problem that, when executed by one or more processors [Abstract, Embodiments herein provide a method and system for evolved State-Action-Reward-State-Action (evolved SARSA) RL for flow shop scheduling, which is a hybrid framework of hierarchical RL with evolutionary techniques and heuristics method to solve FSSP… The framework refines FSSP solution provided by Reinforced-SARSA (R-SARSA) using the evolutionary Genetic Algorithms (GAs); Para 0028, Referring to the components of system 100… the one or more hardware processors 104 are configured to fetch and execute computer-readable instructions stored in the memory 102], cause the one or more processors to perform the steps of:
receiving the problem definition and problem constraints for the combinatorial problem [Para 0005, a method for evolved State-Action-Reward-State-Action (evolved SARSA) reinforcement learning for flow shop scheduling is provided. The method includes receiving a plurality of jobs to be sequenced during flow shop scheduling on a plurality of machines. Further, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan];
processing the problem definition and the problem constraints by a reinforcement learning model to generate an initial set of solutions to the combinatorial problem [Para 0005, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan, by iteratively performing steps comprising: (A) obtaining via a Reinforced State-Action-Reward-State-Action (R-SARSA) learning module a current job sequence of the plurality of jobs. (B) Identifying the current job sequence as a best job sequence if a current makespan estimated for the current job sequence is less than a preceding makespan computed for a previous iteration. The current makespan is identified as a best makespan, and if the current makespan is greater than the preceding makespan, a policy learnt by the R-SARSA learning module in the previous iteration is unlearnt and is assigned a penalty for a reward to prevent from drifting away from obtaining the optimum job sequence, and wherein a first makespan estimated in a first iteration is compared with an initial preset best makespan. (C) Storing the identified best job sequence in a repository, wherein the repository stores a plurality of best sequences identified in each iteration. (D) Generating via an evolutionary crossover module a plurality parent job sequence pairs, randomly picked from the plurality of best sequences]; and
performing genetic operations on the initial set of solutions to produce the evolved set of solutions [Para 0005, Processing via the evolutionary crossover module each of the plurality of parent job sequence pairs, to generate a plurality of evolved job sequences using genetic operators].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2, 5, 6, and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Cai et al. (CN116384602A, see attached translation), hereinafter Cai.
Regarding claim 2 Venugopal teaches the limitations of claim 1.
Venugopal further teaches,
performing additional genetic operations on solutions to produce a further evolved set of solutions [Para 0005, Applying a mutation technique, via a mutation module executed by the one or more hardware processors, on the plurality of evolved job sequences to generate a plurality of mutated job sequences].
Venugopal teaches the above limitations of claim 2 including the evolved set of solutions (para 0005).
Venugopal does not teach removing a portion of solutions to produce a reduced set of solutions; inserting additional solutions into the reduced set of solutions to produce a second set of solutions.
Cai teaches,
removing a portion of solutions to produce a reduced set of solutions; inserting additional solutions into the reduced set of solutions to produce a second set of solutions [Para 0215, if scheme x′ is superior to any existing planning scheme in A, then the superior scheme in A is deleted, and scheme x′ is added to the external archive A].
Cai is analogous to the claimed invention as they both relate to vehicle routing optimization via reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Cai and provide producing an updated set of solutions in order to improve model performance by enhancing the dataset.
Regarding claim 5, Venugopal teaches the limitations of claim 1.
Venugopal does not teach wherein the combinatorial problem is a vehicle routing problem.
Cai teaches,
wherein the combinatorial problem is a vehicle routing problem [Para 0003, This invention relates to the field of vehicle scheduling and intelligent optimization, and in particular to a multi-objective vehicle route optimization method, system, electronic device and medium].
Cai is analogous to the claimed invention as they both relate to vehicle routing optimization via reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Cai and provide a vehicle routing problem in order to [Cai, para 0007] improve the planning efficiency and accuracy of vehicle routes.
Regarding claim 6, Venugopal-Cai teach the limitations of claim 5.
Cai further teaches,
wherein the problem definition comprises delivery locations [Para 0200, Each customer point… is a node in the graph, and each node has corresponding location information] and the problem constraints comprise at least one of vehicle capacity [Para 0117, The constraints include: capacity constraints, time constraints, and service constraints. The capacity constraints include: the number of passengers carried by vehicles on each route must not exceed the total vehicle capacity at any time; Para 0220, For the five-objective vehicle routing problem with time windows, the main point is to allocate the customer point sequence into multiple different paths, and assign one vehicle to each path to complete the service to all customer points], delivery load information, demand at each drop-point, delivery and pickup time windows, delivery priorities, and pickup and drop-point pairs (alternates).
Cai is analogous to the claimed invention as they both relate to vehicle routing optimization via reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Cai and provide delivery locations and vehicle constraints in order to [Cai, para 0007] improve the planning efficiency and accuracy of vehicle routes.
System claim 11 recites similar limitations to method claim 2. Therefore, claim 11 is rejected using the same rationale as claim 2.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Cai, and in further view of VOLOS et al. (US 20190234752 A1), hereinafter Volos.
Regarding claim 7, Venugopal-Cai teach the limitations of claim 5.
Venugopal-Cai do not teach wherein processing the problem definition selects one or more solutions having at least one of a minimal combined delivery time duration, a minimal distance travelled, or a minimal number of delivery vehicles as the initial set of solutions to the problem.
Volos teaches,
wherein processing the problem definition selects one or more solutions having at least one of a minimal combined delivery time duration (alternate), a minimal distance travelled [Para 0099, The travel route module 144 may select: the shortest traveling route], or a minimal number of delivery vehicles as the initial set of solutions to the problem (alternate).
Volos is analogous to the claimed invention as they both relate to vehicle routing systems. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Volos and provide selecting an optimal path in order to [Volos, para 0004] improve fuel efficiency and reduce vehicle congestion within a given geographical area.
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Volos.
Regarding claim 8, Venugopal teaches the limitations of claim 1 including the evolved set of solutions (para 0005).
Venugopal does not teach configuring one or more vehicles to implement at least one solution.
Volos teaches,
configuring one or more vehicles to implement at least one solution [Para 0099, the travel route module 144 selects the travel route with the best metrics and/or least overall latency and/or overall predicted latency. This may include… the shortest traveling route… The travel route or paths with the minimal overall latencies may allow for enablement of certain vehicle applications, which may increase vehicle and occupant safety].
Volos is analogous to the claimed invention as they both relate to vehicle routing systems. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Volos and provide selecting an optimal path in order to [Volos, para 0099] increase vehicle and occupant safety.
Claim(s) 9, 12, and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb et al. (A Beam Search Based Algorithm for the Capacitated Vehicle Routing Problem with Time Windows, published 2013), hereinafter Akeb.
Regarding claim 9, Venugopal teaches the limitations of claim 1 including the reinforcement learning model (para 0005) and the initial set of solutions (para 0005).
Venugopal does not teach wherein model implements a beam search to compute solutions by: generating a set of partial solutions; selecting a subset of the partial solutions based on shared sequences of traversed destinations (shared segments) as a diverse set of partial solutions; and extending the partial solutions in the diverse set of partial solutions to generate another set of partial solutions; and repeating the selecting and extending until the solutions is complete.
Akeb teaches,
wherein model implements a beam search to compute solutions [Sect III (B), para 4, The best solution, among the different solutions obtained, is then retained as the final result] by:
generating a set of partial solutions [Sect III (B), para 4, the method starts by creating the root node which may contains an initial (starting) partial solution];
selecting a subset of the partial solutions based on shared sequences of traversed destinations (shared segments) as a diverse set of partial solutions; and extending the partial solutions in the diverse set of partial solutions to generate another set of partial solutions [Sect III (B), para 4, After that, each node at level ℓ generates a set of descendants, these correspond to level ℓ+1. Each node of the new level is then evaluated by using an evaluation criterion and only a subset containing the ω best nodes are retained, the other nodes are discarded]; and
repeating the selecting and extending until the solutions is complete [Sect III (B), para 4, The beam search stops when no branching becomes possible from any node of the current level; See also Algorithm 2].
Akeb is analogous to the claimed invention as they both relate to the vehicle routing problem. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Akeb and provide a beam search in order to explore various route combinations, in parallel, in a computationally effective manner.
System claim 12 and non-transitory computer-readable media claim 14 recite similar limitations to method claim 9. Therefore, claims 12 and 14 are rejected using the same rationale as claim 9.
Regarding claim 15, Venugopal teaches
A computer-implemented method for producing a set of solutions to a problem using a reinforcement learning model [Abstract, Embodiments herein provide a method and system for evolved State-Action-Reward-State-Action (evolved SARSA) RL for flow shop scheduling, which is a hybrid framework of hierarchical RL with evolutionary techniques and heuristics method to solve FSSP… The framework refines FSSP solution provided by Reinforced-SARSA (R-SARSA) using the evolutionary Genetic Algorithms (GAs)], comprising:
receiving the problem definition (Para 0005, a plurality of jobs to be sequenced during flow shop scheduling), problem constraints (Para 0005, a minimum makespan), and parameters (Para 0005, obtaining an optimum job sequence) [Para 0005, a method for evolved State-Action-Reward-State-Action (evolved SARSA) reinforcement learning for flow shop scheduling is provided. The method includes receiving a plurality of jobs to be sequenced during flow shop scheduling on a plurality of machines. Further, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan]; and
processing the problem definition and problem constraints, according to the parameters, by the reinforcement learning model to generate a set of solutions [Para 0005, the method includes obtaining an optimum job sequence for the plurality of jobs that has a minimum makespan, by iteratively performing steps comprising: (A) obtaining via a Reinforced State-Action-Reward-State-Action (R-SARSA) learning module a current job sequence of the plurality of jobs. (B) Identifying the current job sequence as a best job sequence if a current makespan estimated for the current job sequence is less than a preceding makespan computed for a previous iteration. The current makespan is identified as a best makespan, and if the current makespan is greater than the preceding makespan, a policy learnt by the R-SARSA learning module in the previous iteration is unlearnt and is assigned a penalty for a reward to prevent from drifting away from obtaining the optimum job sequence, and wherein a first makespan estimated in a first iteration is compared with an initial preset best makespan. (C) Storing the identified best job sequence in a repository, wherein the repository stores a plurality of best sequences identified in each iteration. (D) Generating via an evolutionary crossover module a plurality parent job sequence pairs, randomly picked from the plurality of best sequences];
Venugopal does not teach generate a set of partial solutions; selecting a subset of the partial solutions based on shared sequences of traversed destinations as a diverse set of partial solutions; and extending the partial solutions in the diverse set of partial paths to generate an additional set of partial solutions; repeating the selecting and extending until the additional set of solutions is complete; and outputting the additional set of partial solutions as the set of solutions.
Akeb teaches,
generate a set of partial solutions [Sect III (B), para 4, the method starts by creating the root node which may contains an initial (starting) partial solution];
selecting a subset of the partial solutions based on shared sequences of traversed destinations as a diverse set of partial solutions; and extending the partial solutions in the diverse set of partial paths to generate an additional set of partial solutions [Sect III (B), para 4, After that, each node at level ℓ generates a set of descendants, these correspond to level ℓ+1. Each node of the new level is then evaluated by using an evaluation criterion and only a subset containing the ω best nodes are retained, the other nodes are discarded];
repeating the selecting and extending until the additional set of solutions is complete [Sect III (B), para 4, The beam search stops when no branching becomes possible from any node of the current level; See also Algorithm 2]; and
outputting the additional set of partial solutions as the set of solutions [Sect III (D), para 1, The three-phase algorithm based on clustering, beam search, and 2-opt local search is given in algorithm 4… The algorithm’s output corresponds to a set containing m feasible routes].
Akeb is analogous to the claimed invention as they both relate to the vehicle routing problem. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Akeb and provide a beam search in order to explore various route combinations, in parallel, in a computationally effective manner.
Regarding claim 16, Venugopal-Akeb teach the limitations of claim 15.
Akeb further teaches,
wherein the additional set of solutions is complete when all of the destinations are traversed [Sect II, para 1, each customer (or vertex) has coordinates (xi, yi) in the Euclidean plan; Algorithm 2, Ensure:… visiting all the vertices of the cluster].
Akeb is analogous to the claimed invention as they both relate to the vehicle routing problem. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Akeb and provide traversing all the indices in order to improve model robustness by ensuring all options are considered.
Regarding claim 17, Venugopal-Akeb teach the limitations of claim 15.
Akeb further teaches,
wherein selecting the subset comprises: identifying a best partial solution based on performance criteria [Sect III (B), para 4, Each node of the new level is then evaluated by using an evaluation criterion and only a subset containing the ω best nodes are retained]; and
selecting the best partial solution and at least one additional partial solution that does not share at least a portion of the sequence of traversed destination in the best partial solution for inclusion in the subset [Sect III (C), paras 1-3, In order to try to improve the result obtained after the second phase (beam search), a local search is performed on each cluster. 2-opt is an iterative method that consists, at each iteration, to break two nonconsecutive arcs in the route and to link the four extremities in order to form another path and by respecting the time windows of course. The replacement is kept if the obtained solution is better. The 2-opt method is given in algorithm 3. In each iteration, the algorithm examines each two distinct arcs vi → vi+1 and vj → vj+1 in the route R. These two arcs are replaced by the arcs vi → vj and vi+1 → vj+1 if and only if the distance decreases and the time windows are not violated].
Akeb is analogous to the claimed invention as they both relate to the vehicle routing problem. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Akeb and provide identifying best solutions based on performance criteria and uniqueness in order to [Akeb, Sect III (C), para 1] improve results obtained after beam search.
Claim(s) 18 is rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb, and in further view of Wada et al. (US 20250333165 A1), hereinafter Wada.
Regarding claim 18, Venugopal-Akeb teach the limitations of claim 15.
Venugopal-Akeb do not teach wherein the reinforcement learning model is trained using an objective that maximizes a return.
Wada teaches,
wherein the reinforcement learning model is trained using an objective that maximizes a return [Para 0048, the deep reinforcement learning model MDL trained by the value-based method may be trained to output an action (an action variable) a.sub.t in which the value (Q value) is maximized among one or more actions].
Wada is analogous to the claimed invention as they both relate to reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Wada and provide a reinforcement learning model trained using an objective to maximize a return in order to improve system results by optimizing output.
Claim(s) 19 is rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb, and in further view of Volos.
Regarding claim 19, Venugopal-Akeb teach the limitations of claim 15 including the set of solutions (Venugopal, Abstract).
Venugopal-Akeb do not teach configuring one or more vehicles to implement at least one solution.
Volos teaches,
configuring one or more vehicles to implement at least one solution [Para 0099, the travel route module 144 selects the travel route with the best metrics and/or least overall latency and/or overall predicted latency. This may include… the shortest traveling route… The travel route or paths with the minimal overall latencies may allow for enablement of certain vehicle applications, which may increase vehicle and occupant safety].
Volos is analogous to the claimed invention as they both relate to vehicle routing systems. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Volos and provide selecting an optimal path in order to [Volos, para 0099] increase vehicle and occupant safety.
Claim(s) 20 is rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb, and in further view of Takimoto et al. (WO2023175910A1, see attached translation), hereinafter Takimoto.
Regarding claim 20, Venugopal-Akeb teach the limitations of claim 15 including the set of solutions (Venugopal, para 0005).
Venugopal-Akeb do not teach wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed on a server or in a data center to generate solutions, and the solutions is streamed to a user device.
Takimoto teaches,
wherein at least one of the steps of receiving, processing, selecting, extending, repeating (alternates), and outputting (Para 0024, outputs) is performed on a server (Para 0024, information management server) or in a data center (alternate) to generate solutions, and the is streamed to a user device (outputs the optimal solution… to the user terminal device 20) [Para 0024, The decision support system 10 then uses information about public sports races obtained from the information management server 30 and an objective function to determine the optimal solution for predicting the race results. The decision support system 10 then outputs the optimal solution for predicting the race result to the user terminal device 20].
Takimoto is analogous to the claimed invention as they both relate to reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Takimoto and provide streaming results to a user device in order to increase public access to the functionalities of machine learning system.
Claim(s) 21 is rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb, and in further view of Chen et al. (US 20250013500 A1), hereinafter Chen.
Regarding claim 21, Venugopal-Akeb teach the limitations of claim 15.
Venugopal-Akeb do not teach wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed within a cloud computing environment.
Chen teaches,
wherein at least one of the steps of receiving, processing (alternates), selecting (Abstract, selected), extending, repeating (alternates), and outputting (para 0021, outputted) is performed within a cloud computing environment [Abstract, The present invention relates to a deep reinforcement learning-based cloud data center adaptive efficient resource allocation method. First, an operation (a scheduling job) is selected according to a score evaluated by critic (an evaluation operation) by using an actor parameterized policy (resource allocation); Para 0021, a value of the transition probability is obtained by running a DRL algorithm, and probabilities that different actions are adopted in a state are outputted by using the algorithm; and; Para 0022, the reward function: a DRL agent is guided to learn a better job scheduling policy with higher discounted long-term reward through the reward function, to improve system performance of cloud resource allocation].
Chen is analogous to the claimed invention as they both relate to reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Chen and provide reinforcement learning in a cloud computing environment in order to [Chen, para 0008] have a considerably large action space to continuously meet the requirements of scheduling jobs.
Claim(s) 22 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Venugopal in view of Akeb, and in further view of Chen-Fu et al. (Agent-based approach integrating deep reinforcement learning and hybrid genetic algorithm for dynamic scheduling for Industry 3.5 smart production, published 2021), hereinafter Chen(2).
Regarding claim 22, Venugopal-Akeb teach the limitations of claim 15.
Venugopal-Akeb do not teach wherein at least one of the steps of receiving, processing, selecting, extending, repeating, and outputting is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
Chen(2) teaches,
wherein at least one of the steps of receiving, processing, selecting (alternates), extending (Sect 3.3, para 1, to generate better solutions), repeating, and outputting (alternates) is performed for training [Sect 1, para 4, The proposed framework developed the HGA module to enhance searching effectiveness and efficiency during the training process; Sect 3.3, para 1, In the agent training process, the proposed approach employed the hybrid genetic algorithm (HGA) as an optimizer to generate better solutions at each episode for enhancing training efficiency], testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle (alternates).
Chen(2) is analogous to the claimed invention as they both relate to reinforcement learning. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Venugopal’s teachings to incorporate the teachings of Chen(2) and provide reinforcement methodologies for training a system in order to iteratively and autonomously improve the machine learning system, especially for robotic machines.
Claim 23 recites similar limitations to claim 22. Therefore, claim 23 is rejected using the same rationale as claim 22.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED RAYHAN AHMED whose telephone number is (571)270-0286. The examiner can normally be reached Mon-Fri ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED RAYHAN AHMED/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights