DETAILED ACTION
This communication is in response to Application No. 18/427,408 filed on January 30th, 2024
in which claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement submitted on 08/15/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement was considered by the examiner.
Specification
The contents of the specification are sufficient for examination purposes.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Regarding Claim 1, the claim recites the terms “historical” (ln. 2), “current” (ln. 3 and 5) and “future” (ln. 3), which are relative terms that render the claim indefinite. The terms “historical”, “current”, and “future” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree of these terms, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Specifically, the terms “historical” and “current” lie on a continuum, such that it is not clear what should be considered the delineation point between these terms. Similarly, “current” and “future” lie on a continuum, such that it is not clear what should be considered the delineation point between these terms. As a result, a person of ordinary skill in the art would not be reasonably appraised on what qualifies as “historical backhauls” (ln. 2), “historical releases” (ln. 2-3), “current releases” (ln. 3 and 5), or “future releases” (ln. 3). As a result, the claim is rejected. The claim should be amended to provide a standard for ascertaining the requisite degree for the terms “historical”, “current”, and “future”.
Regarding Claims 2-4, the claims are rejected because they are dependent upon a rejected claim.
Regarding Claim 5, the claim recites the term “historical” (ln. 2), which is indefinite for substantially the same reasoning as articulated in regard to the rejection of Claim 1. As a result, the claim is similarly rejected and should be amended in a similar manner.
Additionally, the claim is rejected because it is dependent upon a rejected claim.
Regarding Claim 6, the claim recites the terms “current” (ln. 2) and “future” (ln. 2), which are indefinite for substantially the same reasoning as articulated in regard to the rejection of Claim 1. As a result, the claim is similarly rejected and should be amended in a similar manner.
Additionally, the claim is rejected because it is dependent upon a rejected claim.
Regarding Claims 7-10, the claims are rejected because they are dependent upon a rejected claim.
Regarding Claim 11, the claim recites the terms “historical” (ln. 4), “current” (ln. 5 and 7), and “future” (ln. 5), which are indefinite for substantially the same reasoning as articulated in regard to the rejection of Claim 1. As a result, the claim is similarly rejected and should be amended in a similar manner.
Regarding Claims 12-14, the claims are rejected because they are dependent upon a rejected claim.
Regarding Claim 15, the claim recites the term “historical” (ln. 2), which is indefinite for substantially the same reasoning as articulated in regard to the rejection of Claim 1. As a result, the claim is similarly rejected and should be amended in a similar manner.
Additionally, the claim is rejected because it is dependent upon a rejected claim.
Regarding Claim 16, the claim recites the terms “current” (ln. 1) and “future” (ln. 1), which are indefinite for substantially the same reasoning as articulated in regard to the rejection of Claim 1. As a result, the claim is similarly rejected and should be amended in a similar manner.
Additionally, the claim is rejected because it is dependent upon a rejected claim.
Regarding Claims 17-20, the claims are rejected because they are dependent upon a rejected claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract ideas without significantly more.
Regarding Claim 1:
Step 1: Claim 1 is a process claim. Therefore, claims 1-10 are directed to a statutory category of eligible subject matter.
Step 2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the "Mental Processes" grouping of abstract ideas. Here, steps of the claimed process are mental processes. Specifically, the claim recites
“A . . . method comprising . . . to recommend current releases or future releases . . . determining . . . recommended backhaul loads to be assigned to current releases from among available backhaul loads” (mental process – amounts to exercising judgment to form a recommendation opinion about known or observed backhaul loads, which may be aided by pen and paper) and
“automatically generating matches for outbound routes with the recommended backhaul loads” (mental process – amounts to exercising judgement to form a match opinion about known or observed routes, which can happen automatically depending on a person’s knowledge base, and which may be aided by pen and paper to memorialize the matches).
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“computer-implemented . . . training a machine-learning model . . . using the machine-learning model” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea) and
“based on pairs of historical backhauls and historical releases” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“computer-implemented . . . training a machine-learning model . . . using the machine-learning model” (mere instructions to apply the exception using generic computer components does not provide an inventive concept) and
“based on pairs of historical backhauls and historical releases” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
For the reasons above, Claim 1 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 2-10. The additional limitations of the dependent claims are addressed below.
Regarding Claim 2:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 2 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea) and
“based on (i) a first type of input training data comprising backhaul pickup windows and service times and (ii) a second type of the input training data comprising arrival time windows to pick up backhauls after single-stop store deliveries” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (mere instructions to apply the exception using generic computer components does not provide an inventive concept) and
“based on (i) a first type of input training data comprising backhaul pickup windows and service times and (ii) a second type of the input training data comprising arrival time windows to pick up backhauls after single-stop store deliveries” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 2 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 3:
Step 2A Prong 1: See the rejection of Claim 2 above, which Claim 3 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea) and
“based on incremental performance metrics of backhaul attachment, using the first type of the input training data and the second type of the input training data, and performance factors” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (mere instructions to apply the exception using generic computer components does not provide an inventive concept) and
“based on incremental performance metrics of backhaul attachment, using the first type of the input training data and the second type of the input training data, and performance factors” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 3 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 4:
Step 2A Prong 1: See the rejection of Claim 3 above, which Claim 4 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the performance factors comprise one or more of: an incremental duration based on outbound routing; an incremental distance based on outbound routing; a threshold for empty miles; a penalty for empty miles; stops before a backhaul pickup; or an additional penalty for a dummy empty route” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the performance factors comprise one or more of: an incremental duration based on outbound routing; an incremental distance based on outbound routing; a threshold for empty miles; a penalty for empty miles; stops before a backhaul pickup; or an additional penalty for a dummy empty route” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 4 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 5:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 5 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea) and
“based on output training data of binary classifications based on historical backhaul outcomes” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the machine-learning model is further trained” (mere instructions to apply the exception using generic computer components does not provide an inventive concept) and
“based on output training data of binary classifications based on historical backhaul outcomes” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 5 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 6:
Step 2A Prong 1: See the rejection of Claim 5 above, which Claim 6 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the binary classifications are each current or future” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the binary classifications are each current or future” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 6 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 7:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 7 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the machine-learning model comprises a tree-based gradient boosting model” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the machine-learning model comprises a tree-based gradient boosting model” (mere instructions to apply the exception using generic computer components does not provide an inventive concept).
Accordingly, Claim 7 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 8:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 8 depends on. Here, the claim recites additional elements that are mental processes. Specifically, the claim recites
“wherein automatically generating matches further comprises . . . automatically assigning the recommended backhaul loads to the outbound routes to minimize an overall performance objective” (mental process – amounts to exercising judgement to form a match opinion about known or observed routes, which can happen automatically depending on a person’s knowledge base, which can be done with reference to the goal of minimization of a known performance objective, and which may be aided by pen and paper to memorialize the matches);
“determining feasibility for each pair of the outbound routes and the recommended backhaul loads” (mental process – amounts to exercising judgement to form a feasibility opinion, with reference to known pairs); and
“calculating a performance metric for each pair of the outbound routes and the recommended backhaul loads” (mental process – amounts to exercising judgement to evaluate known information to generate a calculation as a performance metric).
Step 2A Prong 2 & Step 2B: There are no elements left for consideration of implementation within a practical application or for consideration of significantly more.
Accordingly, Claim 8 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 9:
Step 2A Prong 1: See the rejection of Claim 8 above, which Claim 9 depends on. Here, the claim recites additional elements that are mental processes. Specifically, the claim recites
“wherein automatically assigning the recommended backhaul loads to the outbound routes comprises solving a mixed integer programming formulation” (mental process – amounts to exercising judgement to form a match opinion about known or observed routes, which can happen automatically depending on a person’s knowledge base, which may be made with reference to a mixed integer programming formulation goals and constraints, and which may be aided by pen and paper to memorialize the matches).
Step 2A Prong 2 & Step 2B: There are no elements left for consideration of implementation within a practical application or for consideration of significantly more.
Accordingly, Claim 9 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 10:
Step 2A Prong 1: See the rejection of Claim 9 above, which Claim 10 depends on.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“wherein the mixed integer programming formulation comprises constraints comprising: each of the recommended backhaul loads is assigned to a single one of the outbound routes; and each of the outbound routes has at most one backhaul assignment from among the recommended backhaul loads” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“wherein the mixed integer programming formulation comprises constraints comprising: each of the recommended backhaul loads is assigned to a single one of the outbound routes; and each of the outbound routes has at most one backhaul assignment from among the recommended backhaul loads” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
Accordingly, Claim 10 is rejected as being directed to an abstract idea without significantly more.
Regarding Claim 11:
Step 1: Claim 11 is a machine claim. Therefore, claims 11-20 are directed to a statutory category of eligible subject matter.
Step 2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the "Mental Processes" grouping of abstract ideas. Here, the claim recites limitations that are substantially the same as the limitations of Claim 1. As a result, and as elaborated above, these limitations are abstract ideas because they are mental processes.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites the additional elements:
“A system comprising one or more processors and one or more non-transitory computer- readable media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: training a machine-learning model . . . using the machine-learning model” (amounts to mere instructions to apply the judicial exception on generic and unspecialized computer components, which do not impose any meaningful limits on practicing the abstract idea) and
“based on pairs of historical backhauls and historical releases” (amounts to merely generally linking the use of the judicial exception to a particular technological environment or field of use, which do not impose any meaningful limits on practicing the abstract idea).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
The claim recites the additional elements:
“A system comprising one or more processors and one or more non-transitory computer- readable media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: training a machine-learning model . . . using the machine-learning model” (mere instructions to apply the exception using generic computer components does not provide an inventive concept) and
“based on pairs of historical backhauls and historical releases” (merely generally linking the use of the judicial exception to a particular technological environment or field of use does not provide an inventive concept).
For the reasons above, Claim 11 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 12-20. The additional limitations of the dependent claims are addressed below.
Regarding Claim 12, the claim recites limitations that are all substantially the same as limitations of
Claim 2, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 12 is rejected under the same rationale.
Regarding Claim 13, the claim recites limitations that are all substantially the same as limitations of
Claim 3, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 13 is rejected under the same rationale.
Regarding Claim 14, the claim recites limitations that are all substantially the same as limitations of
Claim 4, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 14 is rejected under the same rationale.
Regarding Claim 15, the claim recites limitations that are all substantially the same as limitations of
Claim 5, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 15 is rejected under the same rationale.
Regarding Claim 16, the claim recites limitations that are all substantially the same as limitations of
Claim 6, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 16 is rejected under the same rationale.
Regarding Claim 17, the claim recites limitations that are all substantially the same as limitations of
Claim 7, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 17 is rejected under the same rationale.
Regarding Claim 18, the claim recites limitations that are all substantially the same as limitations of
Claim 8, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 18 is rejected under the same rationale.
Regarding Claim 19, the claim recites limitations that are all substantially the same as limitations of
Claim 9, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 19 is rejected under the same rationale.
Regarding Claim 20, the claim recites limitations that are all substantially the same as limitations of
Claim 10, in the form of a system. The claim is also directed to performing mental processes without
integration into a practical component or significantly more.
Accordingly, Claim 20 is rejected under the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-11, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (hereinafter Chen) (“DeepFreight: Integrating Deep Reinforcement Learning and Mixed Integer Programming for Multi-transfer Truck Freight Delivery”) in view of Alajkovic et al. (hereinafter Alajkovic) (“Delivery pattern planning in retailing with transport and warehouse workload balancing”).
Regarding Claim 1, Chen teaches a computer-implemented method comprising (Pg. 1, Col. 1, Abstract, “we propose DeepFreight, a model-free deep-reinforcement-learning-based algorithm for multi-transfer freight delivery . . . DeepFreight is integrated with a Mixed Integer Linear Programming optimizer for further optimization . . . The codes are available at https://github.com/LucasCJYSDL/DeepFreight”, where the “DeepFreight” “algorithm” “integrated with a Mixed Integer Linear Programming optimizer” is a method, which must be computer-implemented to execute the “codes” and run “deep-reinforcement-learning”; see also Pg. 3, Col. 2, Fig. 2, “System Structure. DeepFreight . . . decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment (introduced in Section V)” and Pg. 9, Col. 2, Para. 1, “the MILP solver runs on a device with an Intel i7-10850H processor”, where “MILP solver”, which is part of the “System Structure” that implements the method, “runs on a device with an Intel i7-10850H processor”):
training a machine-learning model based on pairs of . . . [generated] backhauls and . . . [generated] releases (Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX”, where the “dispatch policy” is “trained through QMIX”; see also Pg. 4, Col. 1, Algo. 1, where, as depicted in Algo. 1, the training, “Update Qsingle and Qmix by minimizing loss function”, is based on, “Sample a random batch of experiences from B”, generated delivery requests, “Generate Nr delivery requests randomly”, that are collectively paired, “Match the delivery requests”, with “dispatch decisions”, which are comprised of generated “destination[s]”, see Pg. 7, Col. 2, Para. 4-5, “Simulation Setup . . . After confirming the source of a delivery request, its destination is randomly generated”; see also Pg. 4, Col. 1, Algo. 1 and Pg. 1, Col. 2, Para. 2, “assign the packages to the trucks”, where the packages associated with “delivery requests”, as collectively “assign[ed]” to a “truck”, are within the broadest reasonable interpretation of backhauls; see also Pg. 2, Col. 2, Fig. 1; Pg. 2, Col. 2, Para. 2, “As an example, consider the scenario shown in Figure 1, where Loads 1 and 4 need to be shipped to a distribution center in Petersburg, VA, while Loads 2 and 3 to the distribution center in New Castle, DE”; and Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck”, where the “dispatch” decisions are within the broadest reasonable interpretation of releases because they “determine[] the itinerary for each truck” for a release of “Loads” at a “distribution center”, thus the training is based on pairs of generated backhauls, e.g. “Loads 1 and 4”, and generated releases, e.g. delivery to “to a distribution center in Petersburg, VA”; see also Pg. 4, Col. 2, Para. 6, “the dispatch policy for agent a can be defined as πa(ua|za) : Z×U → [0,1]” and Pg. 3, Col. 2, Para. 1, “The dispatch policy is trained in a multi-agent reinforcement learning setting called QMIX”, where a probabilistic model, “πa(ua|za) : Z×U → [0,1]”, “trained in a multi-agent reinforcement learning setting” is within the broadest reasonable interpretation of a machine learning model)
to recommend current releases or future releases (Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX”, where, as discussed above, the dispatch decisions comprise “determine[d]” releases, which are not historical classification of past releases and, as a result, are either current or future releases, which are “evaluated” by “perform[ance]” “for multi-transfer freight delivery”, Pg. 10, Col. 2, Para. 2-3, “Overall, DeepFreight+MILP performs best among these algorithms, because it not only has better scalability and lower time complexity but also can ensure a 100% delivery success with fairly low fuel consumption . . . This paper proposes DeepFreight, a novel model-free approach for multi-transfer freight delivery . . . This approach is then integrated with MILP for further optimization. The evaluation results show superior scalability and improved performance of the combined system”; see also Pg. 3, Col. 2, Fig. 2, where, as depicted, the “Dispatch Agent” is used to generate an “Initial Dispatch”, which is finalized as the “final Dispatch” after use of the “MILP Optimizer”, and as a result, the dispatch policy determinations are within the broadest reasonable interpretation of recommendations);
determining, using the machine-learning model, recommended backhaul loads to be assigned to current releases from among available backhaul loads (Pg. 1, Col. 2, Para. 2, “The dispatch policy is adopted to determine the routes of the trucks, and then matching policy is executed to assign the packages to the trucks efficiently”, where an “efficien[t]” recommendation of backhaul loads is determined, the “packages” “assign[ed]” as a backhaul load to each of the “trucks” in a manner that conforms with “efficiency” goals, from the available backhaul loads, the set of possible permutations of “packages” that could be “assign[ed]” to the “trucks”, which indirectly assigns the recommended backhaul loads to the releases because the “trucks” are previously assigned to the releases, “the routes of the trucks”, which uses the machine learning model, “The dispatch policy is adopted to determine”; see also Pg. 2, Col. 2, Fig. 1; Pg. 2, Col. 2, Para. 2, “As an example, consider the scenario shown in Figure 1, where Loads 1 and 4 need to be shipped to a distribution center in Petersburg, VA, while Loads 2 and 3 to the distribution center in New Castle, DE”; and Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck”, where the “dispatch” decisions are within the broadest reasonable interpretation of releases because they “determine[] the itinerary for each truck” for a release of “Loads” at a “distribution center”, which, as best understood in light of the 112(b)-related issues identified above, can reasonable be described as current, where “the itinerary for each truck” is based on the current shipping needs, “to be shipped to a distribution center”); and
automatically generating matches for . . . routes with the recommended backhaul loads (Pg. 6, Col. 1, Para. 2, “We further propose a hybrid approach that harnesses Deep-Freight and MILP to ensure successful delivery of all the packages. Specifically, experiments show that when most of the requests have been completed and only a small number of packages remain to be optimized, DeepFreight may have unstable training dynamics resulting in undelivered packages (as evidenced in Figure 9(b)). To this end, we leverage MILP to find the exact routing decisions for the small portion of requests that are not efficiently handled by DeepFreight”, where “MILP” is used to generate matches for routes, “to find the exact routing decisions for the small portion of requests”, with the recommended backhaul loads already incorporated into the routes, “hybrid approach that harnesses Deep-Freight and MILP . . . when most of the requests have been completed and only a small number of packages remain to be optimized”; see also Pg. 2, Col. 2, Fig. 1, where the itineraries of “Truck 1” and “Truck 2” are depicted using color-coded lines in Fig. 1, such that each itinerary is comprised of at least one route, which can reasonably be defined as a direct travel segment, such as “A” to “B”; see also Pg. 3, Col. 2, Fig. 2, “DeepFreight (introduced in Section IV) decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see also Pg. 6-7, Section “A. MILP Formulation”, where “MILP” is an algorithm to find the “optimal solution” to an “Optimization Objective”, which is performed automatically using “a device with an Intel i7-10850H processor”, see Pg. 9, Col. 2, Para. 1, “Note that the MILP solver runs on a device with an Intel i7-10850H processor”; see generally Pg. 7, Col. 1-2, Section “B. The Workflow of DeepFreight+MILP”, where step “1)” corresponds with the training and determining steps of the method, whereas steps “2)” through “6)” correspond with the generating step of the method, which can be implemented automatically as part of a “Run[] Time” “algorithm”, see Pg. 10, Col. 2, Para. 2, “DeepFreight+MILP performs best among these algorithms” and Pg. 10, Fig. 10, where the automatic performance of the “DeepFreight+MILP . . . algorithm[]” at “Run[] Time” is depicted).
Chen does not explicitly disclose . . . historical . . . (wherein training based on generated data is disclosed, which is not specifically described as historical; redundant recitations of historical omitted)
. . . outbound . . . (wherein the routes are not specifically described as outbound).
However, Alajkovic teaches . . . [using] historical [data to recommend releases] . . . (Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops . . . we first decompose the total problem by isolating the delivery scheduling . . . Delivery patterns are optimized for the quality criteria regarding specific store-warehouse pair types, with a special focus on fresh food delivery that aims at reducing inventory write-offs due to aging”, where “Delivery patterns are optimized for . . . delivery” is within the broadest reasonable interpretation of release recommendation, which uses “historical data”, see Pg. 102, Para. 3, “we have decided to learn shop assignments to route clusters by clustering from the historical data and correcting with expert interventions”; see also Pg. 107, Para. 4, “We have used historical weekly data to create 36 instances” and Pg. 102, Para. 5, “several decisions are already made before optimization, based on forecasting and historical performance. The number of weekly deliveries is fixed by several parameters, expressed as business rules that reflect historical best practices. Additionally, activation of delivery routes throughout a week is also fixed at the input. These fixed decisions are input into the model as parameters Ni,j and RADi,j”)
. . . [for use in downstream operations relating to] outbound [routes] . . . (Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”, where the “routing system” is downstream of the release recommendations, “delivery scheduling” “in advance”, and performs operations relating to outbound routes, “Goods from warehouses must be . . . delivered to shops”).
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the training of a machine-learning model based on pairs of generated backhauls and releases to recommend current or future releases, determining, using the machine-learning model, recommended backhaul loads to be assigned to current releases, and the automatically generating matches for routes with recommended backhaul loads of Chen with the use of historical data to recommend releases for use in downstream operations relating to outbound routes of Alajkovic in order to utilize historical data to train the model to learn patterns from real-world insights (compare Chen, Pg. 4, Col. 1, Algo. 1, “Generate Nr delivery requests randomly”, where the training data is “generate[d] . . . randomly”, which prevents instilling the model with learned patterns from real-world insights, with Alajkovic, Pg. 100, Para. 4, “Demand, which is usually patterned repetitively across a week, is consuming goods at stores” and Alajkovic, Pg. 107, Para. 4, “We have used historical weekly data to create 36 instances”, where real-life historical data, “historical weekly data”, contains insights that can be transformed into learned patterns, “Demand, which is usually patterned repetitively across a week”), which will contribute to increased model performance (Alajkovic, Pg. 109, para. 2, “We have noticed a 3% improvement in the quality of delivery schedules (back-measured against historical data using our objective)”), and to utilize Chen’s method with improved performance, increased scalability, lower time complexity, and high delivery success (Chen, Pg. 10, Col. 2, Para. 2, “Overall, DeepFreight+MILP performs best among these algorithms, because it not only has better scalability and low time complexity but also can ensure a 100% delivery success with fairly low fuel consumption”) to address problems associated with delivery to shops using outbound routes, which will increase method revenue by increasing the applicability of the system to additional use cases (Chen, Pg. 1, Col. 1, Para. 2, “According to American Trucking Associations1, the U.S. trucking industry shipped 11.84 billion tons of goods in 2019 and had a market of 791.7 billion dollars . . . We propose DeepFreight, a model-free learning framework for the freight delivery problem”, where modification of “DeepFreight” to include outbound “deliver[ies]” would allow for monetization of a greater percentage of the “market of 791.7 billion dollars”), and which Chen suggests as an important future direction (compare Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to Shops . . . Delivery patterns are optimized for the quality criteria regarding specific store-warehouse pair types, with a special focus on fresh food delivery that aims at reducing inventory write-offs due to aging” with Chen, Pg. 10, Col. 2, Para. 4, “Extending the work with different priority packages is an important future direction”).
Regarding Claim 8, Chen in view of Alajkovic teach the computer-implemented method of claim 1, wherein automatically generating matches (Chen, Pg. 6, Col. 1, Para. 2, “We further propose a hybrid approach that harnesses Deep-Freight and MILP to ensure successful delivery of all the packages. Specifically, experiments show that when most of the requests have been completed and only a small number of packages remain to be optimized, DeepFreight may have unstable training dynamics resulting in undelivered packages (as evidenced in Figure 9(b)). To this end, we leverage MILP to find the exact routing decisions for the small portion of requests that are not efficiently handled by DeepFreight”, where “MILP” is used to generate matches for routes, “to find the exact routing decisions for the small portion of requests”, with the recommended backhaul loads already incorporated into the routes, “hybrid approach that harnesses Deep-Freight and MILP . . . when most of the requests have been completed and only a small number of packages remain to be optimized”; see also Chen, Pg. 3, Col. 2, Fig. 2, “DeepFreight (introduced in Section IV) decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see also Chen, Pg. 6-7, Section “A. MILP Formulation”, where “MILP” is an algorithm to find the “optimal solution” to an “Optimization Objective”, which is performed automatically using “a device with an Intel i7-10850H processor”, see Chen, Pg. 9, Col. 2, Para. 1, “Note that the MILP solver runs on a device with an Intel i7-10850H processor”)
further comprises (Chen, Pg. 7, Col. 1-2, Para. 3-1, “To exploit the advantages of DeepFreight and MILP, we propose an integration of the two . . . Its workflow is described as below: 1) Get the initial dispatch decisions and matching results using Algorithm 1; 2) Define . . . efficiency . . . and calculate efficiency for each truck; 3) Eliminate all the dispatch decisions of the trucks whose efficiency is lower than the threshold; 4) Rematch the package list with the pruned dispatch decisions, and get the unmatched package list; 5) Pick the new truck list: choose two trucks from the initial truck list for each distribution center that has unmatched packages . . . 6) Adopt the MILP optimizer to get the routing result for the new truck list to serve the unmatched package list”, where, as discussed above, the automatically generating matches comprises steps “2)” through “6)”):
determining feasibility for each pair of the outbound routes and the recommended backhaul loads (Chen, Pg. 7, Col. 1, Para. 3, “Get the initial dispatch decisions and matching results using Algorithm 1 . . . Eliminate all the dispatch decisions of the trucks whose efficiency is lower than the threshold”, where each pair of “initial dispatch decisions and matching results” “of the trucks” is evaluated to determine whether each pair is feasible within a given efficiency constraint, such that an evaluation decision determines feasibility for all pairs of loads and routes associated with the truck, “Eliminate all the dispatch decisions of the trucks whose efficiency is lower than the threshold”; see also Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS”, where, as discussed above, “the requests” that are “assign[ed]” “to the trucks” correspond with the recommended backhaul loads and the “the itinerary for each truck” contain the routes, which in view of Alajkovic include outbound routes, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”);
calculating a performance metric for each pair of the outbound routes and the recommended backhaul loads (Chen, Pg. 7, Col. 1, Para. 3, “Get the initial dispatch decisions and matching results using Algorithm 1 . . . Define a key parameter called efficiency, which equals the number of packages delivered by the truck divided by its driving time, and calculate efficiency for each truck”, where, for each pair of “initial dispatch decisions and matching results” “of the trucks”, a performance matric, “efficiency”, is calculated, such that it is for all pairs of loads and routes associated with the truck, “Define a key parameter called efficiency, which equals the number of packages delivered by the truck divided by its driving time, and calculate efficiency for each truck”; see also Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS”, where, as discussed above, “the requests” that are “assign[ed]” “to the trucks” correspond with the recommended backhaul loads and the “the itinerary for each truck” contain the routs, which in view of Alajkovic include outbound routes, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”); and
automatically assigning the recommended backhaul loads to the outbound routes to minimize an overall performance objective (Chen, Pg. 7, Col. 1-2, Para. 3-1, “4) Rematch the package list with the pruned dispatch decisions, and get the unmatched package list; 5) Pick the new truck list: choose two trucks from the initial truck list for each distribution center that has unmatched packages . . . 6) Adopt the MILP optimizer to get the routing result for the new truck list to serve the unmatched package list”, where “Rematch[ing] the package list with the pruned dispatch decisions” and then using the “MILP optimizer” to add the remaining packages from the “unmatched package list” to the “truck[s]” assigns the recommended backhaul loads to the routes, which in view of Alajkovic include outbound routes, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”, and which happens automatically, see Chen, Pg. 10, Col. 2, Para. 2, “DeepFreight+MILP performs best among these algorithms” and Chen, Pg. 10, Fig. 10, where the automatic performance of the “DeepFreight+MILP . . . algorithm[]” at “Run[] Time” is depicted; see also Chen, Pg. 3, Col. 2, Fig. 2, “DeepFreight (introduced in Section IV) decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see also Chen, Pg. 6-7, Section “A. MILP Formulation”, where “MILP” is an algorithm to find the “optimal solution” to an “Optimization Objective”, which minimizes an overall performance objective, see Chen, Pg. 6, Col. 2, Para. 3, “The optimal solution should minimize the objective function”, and which is performed automatically using “a device with an Intel i7-10850H processor”, see Chen, Pg. 9, Col. 2, Para. 1, “Note that the MILP solver runs on a device with an Intel i7-10850H processor”).
The reasons for obviousness were discussed in regard to the rejection of Claim 1 and remain applicable here.
Regarding Claim 9, Chen in view of Alajkovic teach the computer-implemented method of claim 8, wherein automatically assigning the recommended backhaul loads to the outbound routes comprises (Chen, Pg. 7, Col. 1-2, Para. 3-1, “4) Rematch the package list with the pruned dispatch decisions, and get the unmatched package list; 5) Pick the new truck list: choose two trucks from the initial truck list for each distribution center that has unmatched packages . . . 6) Adopt the MILP optimizer to get the routing result for the new truck list to serve the unmatched package list”, where “Rematch[ing] the package list with the pruned dispatch decisions” and then using the “MILP optimizer” to add the remaining packages from the “unmatched package list” to the “truck[s]” assigns the recommended backhaul loads to the routes, which in view of Alajkovic include outbound routes, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”, and which happens automatically, see Chen, Pg. 10, Col. 2, Para. 2, “DeepFreight+MILP performs best among these algorithms” and Chen, Pg. 10, Fig. 10, where the automatic performance of the “DeepFreight+MILP . . . algorithm[]” at “Run[] Time” is depicted; see also Chen, Pg. 3, Col. 2, Fig. 2, “DeepFreight (introduced in Section IV) decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see also Chen, Pg. 6-7, Section “A. MILP Formulation”, where “MILP” is an algorithm to find the “optimal solution” to an “Optimization Objective”, which minimizes an overall performance objective, see Chen, Pg. 6, Col. 2, Para. 3, “The optimal solution should minimize the objective function”, and which is performed automatically using “a device with an Intel i7-10850H processor”, see Chen, Pg. 9, Col. 2, Para. 1, “Note that the MILP solver runs on a device with an Intel i7-10850H processor”)
solving a mixed integer programming formulation (Chen, Pg. 1, Col. 2, Para. 1, “DeepFreight is integrated with a Mixed-Integer Linear Programming (MILP) optimizer for more efficient and
reliable dispatch and assignment decisions”; see also Chen, Pg. 3, Col. 2, Fig. 2, “a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see generally Chen, Pg. 6-7, Section “A. MILP Formulation”).
The reasons for obviousness were discussed in regard to the rejection of Claim 1 and remain applicable here.
Regarding Claim 10, Chen in view of Alajkovic teach the computer-implemented method of claim 9, wherein the mixed integer programming formulation comprises constraints comprising (Chen, Pg. 1, Col. 2, Para. 1, “DeepFreight is integrated with a Mixed-Integer Linear Programming (MILP) optimizer for more efficient and reliable dispatch and assignment decisions”; see also Chen, Pg. 3, Col. 2, Fig. 2, “a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment”; see generally Chen, Pg. 6-7, Section “A. MILP Formulation”, Subsection “Constraints”):
each of the recommended backhaul loads is assigned to a single one of the outbound routes;
and each of the outbound routes has at most one backhaul assignment from among the recommended backhaul loads (Chen, Pg. 3, Col. 2, Fig. 2, “System Structure. DeepFreight . . . decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck”; Chen, Pg. 1, Col. 2, Para. 2, “The dispatch policy is adopted to determine the routes of the trucks, and then matching policy is executed to assign the packages to the trucks efficiently” and Chen, Pg. 2, Col. 2, Fig. 1, where, as discussed in detail above, each of the recommended backhaul loads are the unique combinations of the “request[ed]” “package[s]” “assign[ed]” . . . to trucks” and the routes are the direct travel segments of a trucks “itinerary”, such as “A” to “B” in Fig. 1, which in view of Alajkovic include outbound routes, which are the direct travel segments to for “deliver[y] to shops”, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”, such that constraint “(17)” ensures that each recommended backhaul load is assigned to a single outbound route ensures each of the outbound routes has at most one backhaul assignment from among the recommended backhaul loads because only “one truck can be assigned” to “each delivery demand” and thus the associated outbound route, “deliver[y] to shops”, is only matched with a single trucks recommended backhaul at the time of delivery, see Chen, Pg. 7, Col. 1, Para. 1, “For each delivery demand, one truck can be assigned at most:
PNG
media_image1.png
62
346
media_image1.png
Greyscale
”; see also Chen, Pg. 7, Col. 1, Para. 1, “If truck k takes the demand from location i to location j, truck k’s trajectory(tour) should include i, j and j appears after i: [EQUATION (18)]”; see generally Chen, Pg. 9, Col. 1, Para. 1, “For the system without multi-transfer, each package will be delivered to its destination by a fixed truck, so there is an extra restriction on the truck used when executing the matching policy (Algorithm 2)” and Chen, Pg. 6, Col. 1, Para. 2, “We further propose a hybrid approach that harnesses Deep-Freight and MILP to ensure successful delivery of all the Packages”, where it is taught that “Deep-Freight and MILP [can be combined] to ensure successful delivery of all the Packages” and that “the system” can be configured “without multi-transfer”).
Regarding Claim 11, Chen teaches a system comprising (Pg. 3, Col. 2, Fig. 2, “System Structure. DeepFreight . . . decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck through DFS. Further, a MILP solver is adopted to optimize the initial dispatch & matching decisions and feedback the final decisions to the environment (introduced in Section V)”, where, as depicted in Fig. 2, the “DeepFreight+MILP” “hybrid approach” is implemented as part of a “System Structure”, see Pg. 1, Col. 2, Para. 3, “a hybrid approach that harnesses DeepFreight and MILP: DeepFreight+MILP, including the formulation of MILP and the workflow of this hybrid approach”)
one or more processors and one or more non-transitory computer-readable media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: . . . (Pg. 1, Col. 1, Abstract, “we propose DeepFreight, a model-free deep-reinforcement-learning-based algorithm for multi-transfer freight delivery . . . The codes are available at https://github.com/LucasCJYSDL/DeepFreight” and Pg. 4, Col. 1, Algo. 1, “DeepFreight Algorithm”, where the “DeepFreight Algorithm” is computing instructions, which must be stored in one or more non-transitory computer-readable media to be “codes are available at https://github.com/LucasCJYSDL/DeepFreight” and executed by the system to produce an algorithm output, and were the system must include one or more processors to execute the operations of the “DeepFreight Algorithm”, as contained in the computer “codes”; see also Pg. 10, Col. 2, Para. 3, “This paper proposes DeepFreight, a novel model-free approach for multi-transfer freight delivery based on deep reinforcement learning”, where a person of ordinary skill in the art would understand “deep reinforcement learning” to require a system comprising processors and non-transitory computer-readable media storing computing instructions, which is executed by the processors to perform operations; see also Pg. 9, Col. 2, Para. 1, “the MILP solver runs on a device with an Intel i7-10850H processor”, where “MILP solver”, which is part of the system, “runs on a device with an Intel i7-10850H processor”).
The remaining limitations are substantially the same as limitations of Claim 1, therefore it is
rejected under the same rationale.
Regarding Claim 18, the additional elements of the dependent claim are substantially the same as limitations of Claim 8, therefore it is rejected under the same rationale.
Regarding Claim 19, the additional elements of the dependent claim are substantially the same as limitations of Claim 9, therefore it is rejected under the same rationale.
Regarding Claim 20, the additional elements of the dependent claim are substantially the same as limitations of Claim 10, therefore it is rejected under the same rationale.
Claims 2-4 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Alajkovic and Liu et al. (hereinafter Liu) (“Memetic search for vehicle routing with simultaneous pickup-delivery and time windows”).
Regarding Claim 2, Chen in view of Alajkovic teach the computer-implemented method of claim 1, wherein the machine-learning model is further trained (Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck”, where the “dispatch policy” is “trained through QMIX”; see also Chen, Pg. 3, Col. 2, Para. 1, “the matching results provide reward feedback for the training of the dispatch policy”; Chen, Pg. 4, Col. 2, Para. 6, “the dispatch policy for agent a can be defined as πa(ua|za) : Z×U → [0,1]”; and Chen, Pg. 3, Col. 2, Para. 1, “The dispatch policy is trained in a multi-agent reinforcement learning setting called QMIX”, where a probabilistic model, “πa(ua|za) : Z×U → [0,1]”, “trained in a multi-agent reinforcement learning setting” is within the broadest reasonable interpretation of a machine learning mode)
based on (Chen, Pg. 3, Col. 2, Para. 1, “the matching results provide reward feedback for the training of the dispatch policy”, where “the matching results”, which uses a first type of input data, “for every delivery request do”, and a second type of input data, “Find all the available paths from source to destination”, as training data, see Chen, Pg. 6, Algo. 2, to generate performance factors, “reward feedback” “for epoch k = 1 to Ne” of Algo. 1, see Chen, Pg. 4, Col. 1, Algo. 1, for “training of the dispatch policy”)
(i) a first type of input training data comprising backhaul pickup [data] . . . (Chen, Pg. 6, Algo. 2, where, as discussed above, the “delivery request[s]” are the first type of input data, received data, which comprises backhaul pickup data, see Chen, Pg. 4, Col. 1, Algo. 1 and Pg. 1, Col. 2, Para. 2, “assign the packages to the trucks”, where the packages associated with “delivery requests”, as collectively “assign[ed]” to a “truck”, are within the broadest reasonable interpretation of backhauls, thus, the “delivery requests” are backhaul data)
and (ii) a second type of the input training data comprising arrival [data] . . . to pick up backhauls . . . (Chen, Pg. 6, Algo. 2, where, as discussed above, the “source” and “destination” information are the second type of input data, calculated data that is subsequently used as input, which comprise arrival data “source” to pick up backhauls, in the form of package components, for transportation to their “destination[s]”; see also Chen, Pg. 7, Col. 2, Para. 4-5, “ten Amazon distribution centers in the eastern U.S. are chosen as the origins and destinations of delivery requests in the simulation. For each episode, Nt trucks should complete Nr randomly generated requests . . . After confirming the source of a delivery request, its destination is randomly generated”)
. . . store deliveries (Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”).
The reasons for obviousness were discussed in regard to the rejection of Claim 1 and remain applicable here.
Chen in view of Alajkovic do not explicitly disclose . . . windows and service times . . . time windows . . . after single-stop . . . .
However, Liu teaches . . . [a method for vehicle routing with simultaneous pickup-delivery and time windows, comprising] (Pg. 1, Abstract, “The Vehicle Routing Problem with Simultaneous Pickup-Delivery and Time Windows (VRPSPDTW) has attracted much research interest in the last decade, due to its wide application in modern logistics . . . we propose a novel Memetic Algorithm with efficienT local search and Extended neighborhood, dubbed MATE, to solve this problem”)
[using a first type of data comprising backhaul pickup] windows and service times (Pg. 3, Col. 1, Para. 3, “Each node 𝑖 ∈ 𝑉 is associated with 5 attributes, i.e., a delivery demand 𝑑𝑖 , a pickup demand 𝑝𝑖 , a time window [𝑎𝑖 , 𝑏𝑖] and a service time 𝑠𝑖 . 𝑑𝑖 is the amount of goods to deliver from the depot to customer 𝑖 and 𝑝𝑖 is the amount of goods to pick up from customer 𝑖 that must be delivered to the depot. 𝑎𝑖 and 𝑏𝑖 are the start and the end of the time window in which the customer receives service”, where the first type of data, the “5 attributes”, which can reasonably be described as data that is received from an external source, such as a “customer” request, comprises backhaul pickup windows, “a time window [𝑎𝑖 , 𝑏𝑖]”, and service times, “a service time 𝑠𝑖”) . . .
[and using a second type of data comprising arrival] time windows [to pick up backhauls] after single-stop [deliveries] (Pg. 3, Col. 2, Para. 1, “The time of arrival at . . . ℎ 𝑗 , denoted as arr(hj) . . . can be computed recursively” and Pg. 3, Col. 1, Para. 3, “Arrival of a vehicle at customer 𝑖 before 𝑎𝑖 results in a wait before service can begin; while arrival after 𝑏𝑖 is infeasible”, where the second type of data is “computed” data, such as the arrival time window, which is the pick-up time window, “𝑎𝑖” through “𝑏𝑖”, as further restricted by the “computed” “time of arrival at . . . ℎ 𝑗”, which may be the same window if “Arrival of a vehicle at customer 𝑖 before 𝑎𝑖” or a reduced window; see also Pg. 3, Col. 2, Para. 3, “The main characteristics of VRPSPDTW lie in the capacity aspect and the temporal aspect . . . customers can simultaneously have delivery demand and pick-up demand”, where a single stop is used to “simultaneously” satisfy “delivery demand and pick-up demand”, and where, as shown in Fig. (3), and as reasonably concluded by a person of ordinary skill in the art, the pickup occurs after the delivery, see Pg. 3, Col. 2, Para. 1, “load (ℎ𝑗) = load (ℎ𝑗−1 ) − 𝑑ℎ 𝑗−1 + 𝑝ℎ 𝑗−1 (3)”, where the delivery load is subtracted before the pickup load is added; alternatively, the pickup will occur after a delivery for all but the first “vehicle visit[]” in the “sequence”, see Pg. 3, Col. 1, Para. 4, “each route 𝑅 𝑖 consists of a sequence of nodes that the vehicle visits”).
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the training of the machine learning model based on a first type of input training data comprising backhaul pickup data and a second type of the input training data comprising arrival data to pick up backhauls within the context of shop deliveries of Chen in view of Alajkovic with the method for vehicle routing with simultaneous pickup delivery and time windows, comprising using a first type of data comprising backhaul pickup windows and service times and using a second type of data comprising arrival time windows to pick up backhauls after single-stop deliveries of Liu in order to receive backhaul pickup windows and service times as part of the delivery request input data (compare Chen, Pg. 6, Algo. 2, where, as discussed above, the “delivery request[s]” are the first type of input data, received data, with Liu, Pg. 3, Col. 1, Para. 3, “Each node 𝑖 ∈ 𝑉 is associated with 5 attributes, i.e., a delivery demand 𝑑𝑖 , a pickup demand 𝑝𝑖 , a time window [𝑎𝑖 , 𝑏𝑖] and a service time 𝑠𝑖 . 𝑑𝑖 is the amount of goods to deliver from the depot to customer 𝑖 and 𝑝𝑖 is the amount of goods to pick up from customer 𝑖 that must be delivered to the depot. 𝑎𝑖 and 𝑏𝑖 are the start and the end of the time window in which the customer receives service”) and to determine arrival time windows alongside source and destination input data (compare Chen, Pg. 6, Algo. 2, where, as discussed above, the “source” and “destination” information are the second type of input data, calculated data that is subsequently used as input, with Liu, Pg. 3, Col. 2, Para. 1, “The time of arrival at . . . ℎ 𝑗 , denoted as arr(hj) . . . can be computed recursively” and Liu, Pg. 3, Col. 1, Para. 3, “Arrival of a vehicle at customer 𝑖 before 𝑎𝑖 results in a wait before service can begin; while arrival after 𝑏𝑖 is infeasible”), which will allow for the incorporation of input data sufficient for determining feasible solutions to the vehicle routing with simultaneous pickup delivery and time windows problem (Liu, Pg. 4, Col. 1, Para. 1, “these components ensure that the generated solutions always satisfy the constraints of capacity and time windows. Moreover, if a generated solutions route number (i.e., the number of used vehicles) exceeds the number of available ones, then the solution will be immediately discarded. In other words, during the entire run of MATE, each individual in the population always corresponds to a feasible solution to the VRPSPDTW instance”; see also Liu, Pg. 3, Col. 2, Para. 3, “each customer is associated with a hard time window, which further increases the difficulty planning the service order for a particular vehicle under the time-window constraint”; see also Liu, Pg. 13, Col. 1, Para. 1, “In this paper, we proposed a memetic algorithm, dubbed MATE, for solving VRPSPDTW”), where solving this problem will lower energy consumption by substantially reducing travel time and distances, which will result in positive environmental impacts and financial savings (Liu, Pg. 2, col. 2, Para. 2, “The results in [1] showed that, com- pared to traditional one-directional logistics, bi-directional logistics can achieve substantial time/distance savings. Since then numerous studies have been conducted on VRPSPD”; see also Liu, Pg. 1, Col. 1, para. 1, “Reverse logistics plays an important role in modern transportation. Generally, it is related to bi-directional flow of goods regarding delivery and pickup activities, where the former refers to shipping goods to the customers, while the latter refers to the opposite. Because of its significant effect on lowering costs associated with energy consumption and reducing the environmental impact, reverse logistics has been incorporated into many regular delivery systems”).
Regarding Claim 3, Chen in view of Alajkovic and Liu teach the computer-implemented method of claim 2, wherein the machine-learning model is further trained based on incremental performance metrics of backhaul attachment (Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign
the requests to the trucks based on the route of each truck”, where the “dispatch policy” is “trained through QMIX”; see also Chen, Pg. 4, Col. 1, Algo. 1, where, as depicted in Algo. 1, the training, “Update Qsingle and Qmix”, is based on incremental backhaul performance metrics, “Generate Nr delivery requests randomly . . . Calculate joint reward rk . . . [and] Update . . . by minimizing loss function”, which, are incremental performance metrics of backhaul attachment, “11: Match the delivery requests with the dispatch decisions using Algorithm 2 12: Calculate joint reward rk”; see also Chen, Pg. 3, Col. 2, Para. 1, “the matching results provide reward feedback for the training of the dispatch policy”; Chen, Pg. 4, Col. 2, Para. 6, “the dispatch policy for agent a can be defined as πa(ua|za) : Z×U → [0,1]”; and Chen, Pg. 3, Col. 2, Para. 1, “The dispatch policy is trained in a multi-agent reinforcement learning setting called QMIX”, where a probabilistic model, “πa(ua|za) : Z×U → [0,1]”, “trained in a multi-agent reinforcement learning setting” is within the broadest reasonable interpretation of a machine learning mode),
using the first type of the input training data and the second type of the input training data, and performance factors (Chen, Pg. 3, Col. 2, Para. 1, “the matching results provide reward feedback for the training of the dispatch policy”, where “the matching results”, which uses the first, “for every delivery request do”, and second, “Find all the available paths from source to destination”, types of input training data, see Chen, Pg. 6, Algo. 2, to generate the performance factors, “reward feedback” “for epoch k = 1 to Ne” of Algo. 1, see Chen, Pg. 4, Col. 1, Algo. 1, for “training of the dispatch policy”).
Regarding Claim 4, Chen in view of Alajkovic and Liu teach the computer-implemented method of claim 3, wherein the performance factors comprise one or more of: an incremental duration based on outbound routing; an incremental distance based on outbound routing; a threshold for empty miles; a penalty for empty miles; stops before a backhaul pickup; or an additional penalty for a dummy empty route (Chen, Pg. 4, Col. 2, Para. 5, “Equation (3) . . . rk = β1Nkrs − β2Fktotal” and Chen, Pg. 4, Col. 1, Algo. 1, where the “joint reward rk” “for epoch k = 1 to Ne” are the performance factors, which comprises “Fktotal”, the “total fuel consumption”, that in turn comprises a duration, “driving time”, incrementally added across the fleet, “k”, to amount to the “total”, see Chen, Pg. 3, Col. 1, Para. 3-4, “Minimizing the total fuel consumption of the fleet Ftotal during this process. The second component Ftotal is viewed as the function of the total driving time”, which is based on routing, see Chen, Pg. 3, Col. 2, Fig. 2, “System Structure. DeepFreight (introduced in Section IV) decomposes the freight delivery problem into two components: truck-dispatch and package-matching. The dispatch policy determines the itinerary for each truck, which is trained through QMIX. The matching policy is then executed to assign the requests to the trucks based on the route of each truck”, which in view of Alajkovic includes outbound routes, see Alajkovic, Pg. 99, Abstract, “Goods from warehouses must be scheduled in advance, prepared, routed, and delivered to shops. At least three systems directly interact within such a process: warehouse workforce scheduling, delivery scheduling, and routing system”).
The reasons for obviousness were discussed in regard to the rejection of Claim 1 and remain applicable here.
Regarding Claim 12, the additional elements of the dependent claim are substantially the same as limitations of Claim 2, therefore it is rejected under the same rationale.
Regarding Claim 13, the additional elements of the dependent claim are substantially the same as limitations of Claim 3, therefore it is rejected under the same rationale.
Regarding Claim 14, the additional elements of the dependent claim are substantially the same as limitations of Claim 4, therefore it is rejected under the same rationale.
Claims 5-6 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Alajkovic and Kumar et al. (hereinafter Kumar) (“Binary-Classifiers-Enabled Filters for Semi-Supervised Learning”).
Regarding Claim 5, Chen in view of Alajkovic teach the computer-implemented method of claim 1, wherein the machine-learning model is further trained . . . based on historical backhaul outcomes (Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX”, where the “dispatch policy” is “trained through QMIX”; see also Chen, Pg. 4, Col. 1, Algo. 1, where, as depicted in Algo. 1, the training, “Update Qsingle and Qmix”, is based on backhaul outcomes, “Generate Nr delivery requests randomly . . . [and] Update . . . by minimizing loss function”, which, in view of Alajkovic, is historical, see Alajkovic, Pg. 107, Para. 4, “We have used historical weekly data to create 36 instances”; see also Chen, Pg. 4, Col. 2, Para. 6, “the dispatch policy for agent a can be defined as πa(ua|za) : Z×U → [0,1]” and Chen, Pg. 3, Col. 2, Para. 1, “The dispatch policy is trained in a multi-agent reinforcement learning setting called QMIX”, where a probabilistic model, “πa(ua|za) : Z×U → [0,1]”, “trained in a multi-agent reinforcement learning setting” is within the broadest reasonable interpretation of a machine learning mode).
The reasons for obviousness were discussed in regard to the rejection of Claim 1 and remain applicable here.
Chen in view of Alajkovic do not explicitly disclose . . . based on output training data of binary classifications . . . .
However, Kumar teaches . . . [training a machine learning model] based on output training data of binary classifications . . . (Pg. 5, Col. 2, Para. 1, “we use the trained binary classifiers to assign pseudo labels for the unlabeled data . . . The pseudo labeled data (labeled by any of the above techniques) is then combined with the labeled data to train a single model to obtain he final model”).
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the machine-learning model that is further trained based on historical backhaul outcomes of Chen in view of Alajkovic with the training a machine learning model] based on output training data of binary classifications of Kumar in order to generate pseudo-labeled training data to augment sparsely labeled training data (Kumar, Pg. 2, Col. 1, Para. 2, “SSL methods have been successful in numerous tasks i.e. image classification [20]–[22], text classification [23], sound classification [24], [25] [26]. In traditional SSL approaches, the network is first trained on labeled data, then the trained network is used as a pseudo labels predictor for unlabeled data. In this case, samples are filtered using a probability threshold. Labeled data and pseudo labeled data are now combined into a large scale data, and the model is finally trained on the large scale dataset”), which will improve model accuracy by increasing training data size (Kumar, Pg. 8-9, Col. 2-1, Para. 4-1, “The proposed method achieves significant performance improvement and outperforms both of the SL and SSL methods. It is to be noted that the model accuracy is increased with the increase in number of training samples for all of the methods in comparison”) in a manner that reduces training data noise in order to further improve model performance (Kumar, Pg. 2, Col. 2, Para. 1, “data being filtered by the model is noisy, which further degrades the overall performance of the model. To remedy this issue, we propose a novel frame work of Binary classifiers based Semi-Supervised Learning (BSSL) that eliminates the threshold issue to improves the performance of pseudo labeling in the conventional SSL”).
Regarding Claim 6, Chen in view of Alajkovic and Kumar teach the computer-implemented method of claim 5, wherein the binary classifications are each current or future (Kumar, Pg. 5, Col. 2, Para. 1, “we use the trained binary classifiers to assign pseudo labels for the unlabeled data . . . The pseudo labeled data (labeled by any of the above techniques) is then combined with the labeled data to train a single model to obtain the final model”, where, as best understood in regard to the 112-related issues discussed above, each of the classifications could reasonably be described as future because they are used for future “train[ing of] a single model” located downstream or, alternatively, the classifications could be considered current because they are part of the current “framework”, see Kumar, Pg. 10, Col. 1, Para. 2, “In this work, we addressed the vital issue of thresholding in semi-supervised learning and proposed a new framework based on binary classifiers that does not require any thresholding during the samples selection process”).
The reasons for obviousness were discussed in regard to the rejection of Claim 5 and remain applicable here.
Regarding Claim 15, the additional elements of the dependent claim are substantially the same as limitations of Claim 5, therefore it is rejected under the same rationale.
Regarding Claim 16, the additional elements of the dependent claim are substantially the same as limitations of Claim 6, therefore it is rejected under the same rationale.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Alajkovic and Anghel et al. (hereinafter Anghel) (“Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms”).
Regarding Claim 7, Chen in view of Alajkovic teach the computer-implemented method of claim 1, wherein the machine-learning model comprises a . . . model (Chen, Pg. 3, Col. 2, Fig. 2, “The dispatch policy determines the itinerary for each truck, which is trained through QMIX”, where, as discussed above, the “dispatch policy” is a machine learning model, and as a result, comprises a model; see also Chen, Pg. 4, Col. 2, Para. 6, “the dispatch policy for agent a can be defined as πa(ua|za) : Z×U → [0,1]” and Chen, Pg. 3, Col. 2, Para. 1, “The dispatch policy is trained in a multi-agent reinforcement learning setting called QMIX”).
Chen in view of Alajkovic do not explicitly disclose . . . tree-based gradient boosting . . . .
However, Anghel teaches . . . [a machine-learning model comprising a] tree-based gradient boosting [model] (Pg. 1, Abstract, “Gradient boosting decision trees (GBDTs) have seen widespread adoption in academia, industry and competitive data science due to their state-of-the-art performance in many machine learning tasks”).
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the machine-learning model of Chen in view of Alajkovic with the machine-learning model comprising a tree-based gradient boosting model of Anghel in order to utilize a model with state-of-the-art performance and widespread adoption (Anghel, Pg. 1, Abstract, “Gradient boosting decision trees (GBDTs) have seen widespread adoption in academia, industry and competitive data science due to their state-of-the-art performance in many machine learning tasks”) to recommend current or future releases.
Regarding Claim 17, the additional elements of the dependent claim are substantially the same as limitations of Claim 7, therefore it is rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Granada-Echeverri et al. (“A mixed integer linear programming formulation for the vehicle routing problem with backhauls”) discloses applicable constraints associated with a mixed integer linear programming formulation for the vehicle routing problem with backhauls.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW BRYCE GOLAN whose telephone number is (571)272-5159. The examiner can normally be reached Monday through Friday, 8:00 AM to 5:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW BRYCE GOLAN/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights