DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 4, 2026, has been entered.
Claims 1, 8, and 15 are amended.
Claims 1-20 are pending.
Response to Arguments
35 USC §101 Rejections
The Applicant traverses the rejection of the claims as being directed to an ineligible abstract idea, contending that the identified abstract idea is not recited per se in the claims. See Remarks p. 12. In response, the Examiner points to exemplary independent claim 1 and the rejection, below. Exemplary independent claim 1 explicitly states: “generate a vehicle transportation itinerary.” The rejection identifies the abstract idea as “generating a vehicle transportation itinerary.” The abstract idea is recited per se in the claims.
The Applicant additionally contends that the present claims are subject matter eligible because the recited elements constitute a combination of algorithms that result in an improved itinerary. See Remarks p. 13. In response, the Examiner points out that mathematical concepts, such as algorithms, are ineligible abstract ideas. See MPEP §2106.04(a). Moreover, generating an algorithm to optimize vehicle itineraries is an abstract idea. The claims do not recite a practical application of the identified abstract idea, and no apparent improvement to technology or a technical field is recited in the claims.
The Applicant additionally contends that the present claims provide a practical application by reciting an improved combination of algorithms. See Remarks p. 14. Again, the Examiner reiterates that the claims do not recite any apparent improvement in the field of machine learning. The claims merely recite the application of various known machine learning methods to optimize vehicle itineraries. Contrary to the Applicant’s assertions, generation of new software code is not evidence of subject matter eligibility.
The Applicant additionally contends that the claims are subject matter eligible because the recited algorithms are more efficient than previous algorithms. See Remarks p. 15. In response, the Examiner submits that the claims merely recite the idea of efficiency and optimization at a high level. See exemplary independent claim 1: “training a reinforcement learning model to generate an optimal itinerary.” An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome. See MPEP §2106.05(a). Merely stating in the claims that the result is optimal is the idea of a solution. The steps for achieving an optimal solution are not recited in a particular manner.
The rejection for lack of subject matter eligibility is updated and maintained.
35 USC §112 Rejections
In light of the Applicant’s amendments, the rejection of the claims under 35 USC §112, is withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The Manual of Patent Examining Procedure (MPEP) provides detailed rules for determining subject matter eligibility for claims in §2106. Those rules provide a basis for the analysis and finding of ineligibility that follows.
Claims 1-20 are rejected under 35 U.S.C. 101. The claimed invention is directed to non-statutory subject matter because the claimed invention recites a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Under Step 1 of the subject matter eligibility analysis, claims(s) 1-20 are all directed to one of the four statutory categories of invention. However, under step 2A, prong one, the claims recite a judicial exception: generating a vehicle transportation itinerary (as evidenced by exemplary independent claim 1; “generate a vehicle transportation itinerary”), an abstract idea. Certain methods of organizing human activity are ineligible abstract ideas, including managing personal behavior or relationships or interactions between people. See MPEP §2106.04(a). The limitations of exemplary claim 1 include: [1] “obtaining a past itinerary;” [2] “obtaining a past cost function;” [3] “training a reinforcement learning model to generate an optimal itinerary;” [4] “generating the plurality of itineraries;” [5] “generating input features;” [6] “processing the input features;” [7] “processing outputs;” [8] “determining a first route;” [9] “determining a second route;“ [10] “determining that the second route includes a minimum result;” [11] “storing an itinerary;” [12] “storing the reinforcement learning model;” [13 “training [a] supervised learning model;” [14] “provide a set of input requirements;” and [15] “generate a vehicle transport itinerary;” Steps [1], [2], [4]-[11], [14], and [15] are steps for managing personal behavior related to the abstract idea of generating a vehicle transportation itinerary that, when considered alone and in combination, are part of the abstract idea of generating a vehicle transportation itinerary. The dependent claims further recite steps for managing personal behavior that are part of the abstract idea of generating a vehicle transportation itinerary. These claim elements, when considered alone and in combination, are considered to be abstract ideas because they are directed to a method of organizing human activity which includes optimizing a travel plan for a vehicle.
Under step 2A, prong two, of the subject matter eligibility analysis, a claim that recites a judicial exception must be evaluated to determine whether the claim provides a practical application of the judicial exception. Additional elements of the independent claims amount to generic computer hardware that does not provide a practical application (a processor, communication device, computer readable medium, and mobile device in independent claim 1; a computer readable medium and mobile device in independent claim 8; and a mobile device in independent claim 15). See MPEP §2106.04(d)[I]. Steps [3], [12], and [13 recite steps for training and storing learning models that merely amount to the use of machine learning in a computing environment. In effect, the abstract idea of generating a vehicle transportation itinerary is generally linked to a computing environment using machine learning for implementation. The amended language similarly recites steps for calculating residuals and policies, recited at a high level generality, as intermediate steps in a machine learning algorithm. These recitations amount to a technological environment for implementing the abstract idea that does not provide a practical application or significantly more than an abstract idea. See MPEP §2106.05(h). No apparent improvement to machine learning is recited in the claims. The claims do not recite an improvement to another technology or technical field, nor do they recite an improvement to the functioning of the computer itself. See MPEP §2106.05(a). Because the claims only recite use of a generic computer, they do not apply the judicial exception with a particular machine. See MPEP §2106.05(b). Under step 2B of the subject matter eligibility analysis, the claims do not integrate the abstract idea into a judicial exception. Referring to the additional elements provided in the analysis in step one, above, the generic computer hardware does not provide significantly more than the recited abstract idea. See MPEP §2106.05(f).
For these reasons, the claims do not provide a practical application of the abstract idea, nor do they amount to significantly more than an abstract idea under step 2B of the subject matter eligibility analysis. Using a generic computer to implement an abstract idea does not provide an inventive concept. Therefore, the claims recite ineligible subject matter under 35 USC §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 8, 12, 15, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20200111169 A1 to Halder et al. (hereinafter ‘HALDER’) in view of Silver, David, et al. "Mastering chess and shogi by self-play with a general reinforcement learning algorithm." arXiv preprint arXiv:1712.01815 (2017) (hereinafter ‘SILVER’), US 10133275 B1 to Kobilarov et al. (hereinafter ‘KOBILAROV’), US 20190304065 A1 to Bousmalis et al. (hereinafter ‘BOUSMALIS’), and US 20150170094 A1 to Ye et al. (hereinafter ‘YE’).
Claim 1 (Currently Amended)
HALDER discloses a system for training a (see abstract and ¶[0008]-[0009]; use a machine learning based technique. Use trained risk and price models) to generate an optimal vehicle transportation itinerary (see abstract and ¶[0066]; decide which route to take based on insurance premium values), the system comprising:
a data store (see ¶[0005]; non-transitory computer-readable storage media storing programs);
at least one processor (see again ¶[0005]; instructions executable by one or more processors);
a communication device (see ¶[0037] and Fig. 1; computing devices and systems can be communicatively coupled to each other through one or more communication networks 140); and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the at least one processor (see again ¶[0005]; various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors, and the like), perform a method of training the (see abstract and ¶[0066]; decide which route to take based on insurance premium values), the method comprising:
obtaining a past itinerary comprising a series of vehicle trips comprising at least a starting location, an ending location, and a distance traveled (see ¶[0050], [0055]-[0058], and [0131]; historical data may include attributes of a route. Pickup and drop-off complexity index is computed based on start/end location and total distance to travel historical data associated with real-world locations and their characteristics);
obtaining a past cost function associated with the past itinerary from a machine learning algorithm configured to generate vehicle transportation itineraries (see ¶[0015] and [0058]; training data for a pickup-dropoff complexity index can be fit to a mathematical function. A risk value is predicted based on the PDCI. See also ¶[0007]-[0008]; the risk or loss value for each category may be computed based on historical data); and
training a reinforcement learning model to generate an optimal itinerary (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle),
HALDER does not specifically disclose, but SILVER discloses, by setting the past itinerary against a plurality of itineraries (see title and abstract and p. 3; achieve superhuman levels of performance through reinforcement learning from games of self-play. Use a general purpose Monte-Carlo tree search (MCTS) algorithm. Each search consists of a series of simulated games of self-play that traverse a tree from root sroot to leaf. Each simulation proceeds by selecting in each state s a move a with low visit count, high move probability and high value (averaged over the leaf states of simulations that selected a from s) according to the current neural network f . Games are played by selecting moves for both players by MCTS, at t . At the end of the game, the terminal position sT is scored according to the rules of the game to compute the game outcome z: 1 for a loss, 0 for a draw, and +1 for a win.).
HALDER does not specifically disclose, but KOBILAROV discloses, wherein each itinerary comprises a plurality of nodes representing a plurality of decisions along a route associated with each itinerary (see abstract; determining a route can include a search algorithm. Trajectories can be selected based on various costs and constraints that are optimized for performance. Nodes can be generated and actions can be explored based on machine learning, including a deep neural network),
HALDER further discloses wherein the past itinerary is a best itinerary according to the past cost function (see ¶[0066]; the route to take is selected as whichever route is associated with the lowest insurance premium value, this would minimize the risk for the AV ride while also minimizing the insurance premium charged to the user);
HALDER does not specifically disclose, but KOBILAROV discloses, generating the plurality of itineraries by repeating steps until a new determination point is reached or a maximum step count threshold value is reached (see col 4, ln 49-col 5, ln 3; decisions can be made in real time. Determine trajectories to evaluate based on a current state or goal. See also col 17, ln 23-28; a policy can be learned iteratively).
HALDER does not specifically disclose, but BOUSMALIS discloses, repeating the steps of: generating input features comprising concatenated planes including activity assignment statuses for each of a first player and a second player (see ¶[0007]; concatenate input images with an additional channel to generate a combined input);
processing the input features by a residual tower comprising a convolution block and one or more residual blocks (see ¶[0007]; The convolutional sub-neural network may comprise a plurality of residual blocks each comprising a respective plurality of resolution-preserving convolutional layers.); and
processing outputs of the residual tower by policy head to calculate a policy at each node and a value head to calculate a value at each node (see ¶[0025] and [0055]; develop a control policy for a vehicle).
HALDER does not specifically disclose, but KOBILAROV discloses, determining a first route for a first driver by minimizing the past cost function at each node of the plurality of nodes based on the policy and the value (see abstract and col 1, ln 58-col 2, ln 26; ; trajectories can be selected based on various costs and constraints optimized for performance. Temporal-logic as applied to policy-learning for autonomous vehicles).
The combination of HALDER and KOBILAROV does not specifically disclose, but YE discloses, determining a second route for a second driver by minimizing a new cost function distinct from the past cost function at each node of the plurality of nodes based on the policy and the value (see claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate);
determining that the second route includes a minimum result by comparing the first route and the second route (see again claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate).
HALDER further discloses storing an itinerary including the second route as one of the plurality of itineraries (see ¶[0068] and [0100]-[0102]; the data depicted in FIG. 1 under data sources 106 is generally stored in a computer-readable format indicating values for different attributes under the vehicle, route, weather and road condition, or other categories);
storing the reinforcement learning model including the plurality of itineraries (see ¶[0140]; use a model trained using reinforcement learning);
training the (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle).
HALDER does not explicitly disclose: training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries.
However, KOBILAROV discloses training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries (see col 21, ln 9-46; Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. Gradient Boosted Regression Trees (GBRT), Random Forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc. See also col 1, ln 58-col 2, ln 26; reinforcement learning with deep neural networks).
HALDER further discloses provide a set of input requirements to the supervised learning model (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route. See also ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride); and
generate a vehicle transportation itinerary by the supervised learning model based on the set of input requirements (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. SILVER discloses achieving superhuman performance using reinforcement learning from games of self-play that compare outcomes from players to determine winners. It would have been obvious for one of ordinary skill in the art at the time of invention to use self-play as taught by SILVER in the system executing the method of HALDER with the motivation to achieve superhuman levels of performance in reinforcement learning and select a winning player with a best route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. KOBILAROV discloses trajectory generation that optimizes costs and constraints at nodes to generate actions. It would have been obvious for one of ordinary skill in the art at the time of invention to include the model as taught by KOBILAROV in the system executing the method of HALDER with the motivation to select a best route for a vehicle.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. BOUSMALIS discloses transforming source domain images into target domain images that uses concatenated input and a convolutional sub-neural network with residual blocks and convolutional layers to preserve resolution. It would have been obvious for one of ordinary skill in the art at the time of invention to include the neural network structure as taught by BOUSMALIS in the system executing the method of HALDER with the motivation to generate a predictive model for selecting a route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. YE discloses vessel scheduling that ranks vehicle candidates based on cost. It would have been obvious for one of ordinary skill in the art at the time of invention to include the scheduling as taught by YE in the system executing the method of HALDER with the motivation to select a vehicle and route with optimized cost.
Claim 5 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the system as set forth in claim 1.
HALDER further discloses wherein the set of input requirements further comprises one or more of a set of activities, activity start/end times, employees, employee clock- in/clock-out times, contractors, contractor clock-in/clock-out times, driver ratings, and vehicle types (see ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride).
Claim 8 (Currently Amended)
HALDER discloses one or more non-transitory computer-readable media storing computer-executable instructions (see ¶[0005]; computer-readable storage media) that, when executed by a processor (see again ¶[0005]; instructions executable by one or more processors), perform a method of training a (see abstract and ¶[0066]; decide which route to take based on insurance premium values), the method comprising;
obtaining a past itinerary (see ¶[0008]; historical data) comprising a series of vehicle trips comprising a starting location, an ending location, and a distance traveled (see ¶[0050], [0055]-[0058], and [0131]; historical data may include attributes of a route. Pickup and drop-off complexity index is computed based on start/end location and total distance to travel historical data associated with real-world locations and their characteristics);
obtaining a past cost function associated with the past itinerary from a machine learning algorithm configured to generate vehicle transportation itineraries (see ¶[0015] and [0058]; training data for a pickup-dropoff complexity index can be fit to a mathematical function. A risk value is predicted based on the PDCI. See also ¶[0007]-[0008]; the risk or loss value for each category may be computed based on historical data);
training a reinforcement learning model to generate an optimal itinerary (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle),
HALDER does not specifically disclose, but SILVER discloses, by setting the past itinerary against a plurality of itineraries (see title and abstract and p. 3; achieve superhuman levels of performance through reinforcement learning from games of self-play. Use a general purpose Monte-Carlo tree search (MCTS) algorithm. Each search consists of a series of simulated games of self-play that traverse a tree from root sroot to leaf. Each simulation proceeds by selecting in each state s a move a with low visit count, high move probability and high value (averaged over the leaf states of simulations that selected a from s) according to the current neural network f . Games are played by selecting moves for both players by MCTS, at t . At the end of the game, the terminal position sT is scored according to the rules of the game to compute the game outcome z: 1 for a loss, 0 for a draw, and +1 for a win.).
HALDER does not specifically disclose, but KOBILAROV discloses, wherein each itinerary comprises a plurality of nodes representing a plurality of decisions along a route associated with each itinerary (see abstract; determining a route can include a search algorithm. Trajectories can be selected based on various costs and constraints that are optimized for performance. Nodes can be generated and actions can be explored based on machine learning, including a deep neural network),
HALDER further discloses wherein the past itinerary is a best itinerary according to the past cost function (see ¶[0066]; the route to take is selected as whichever route is associated with the lowest insurance premium value, this would minimize the risk for the AV ride while also minimizing the insurance premium charged to the user);
HALDER does not specifically disclose, but KOBILAROV discloses, generating the plurality of itineraries by repeating steps until a new determination point is reached or a maximum step count threshold value is reached see col 4, ln 49-col 5, ln 3; decisions can be made in real time. Determine trajectories to evaluate based on a current state or goal. See also col 17, ln 23-28; a policy can be learned iteratively).
HALDER does not specifically disclose, but BOUSMALIS discloses, repeating the steps of: generating input features comprising concatenated planes including activity assignment statuses for each of a first player and a second player (see ¶[0007]; concatenate input images with an additional channel to generate a combined input);
processing the input features by a residual tower comprising a convolution block and one or more residual blocks (see ¶[0007]; The convolutional sub-neural network may comprise a plurality of residual blocks each comprising a respective plurality of resolution-preserving convolutional layers.); and
processing outputs of the residual tower by policy head to calculate a policy at each node and a value head to calculate a value at each node (see ¶[0025] and [0055]; develop a control policy for a vehicle).
HALDER does not specifically disclose, but KOBILAROV discloses, determining a first route for a first driver by minimizing the past cost function at each node of the plurality of nodes based on the policy and the value (see abstract and col 1, ln 58-col 2, ln 26; ; trajectories can be selected based on various costs and constraints optimized for performance. Temporal-logic as applied to policy-learning for autonomous vehicles).
The combination of HALDER and KOBILAROV does not specifically disclose, but YE discloses, determining a second route for a second driver by minimizing a new cost function distinct from the past cost function at each node of the plurality of nodes based on the policy and the value (see claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate);
determining that the second route includes a minimum result by comparing the first route and the second route (see again claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate).
HALDER further discloses storing an itinerary including the second route as one of the plurality of itineraries (see ¶[0068] and [0100]-[0102]; the data depicted in FIG. 1 under data sources 106 is generally stored in a computer-readable format indicating values for different attributes under the vehicle, route, weather and road condition, or other categories);
storing the reinforcement learning model including the plurality of itineraries (see ¶[0140]; use a model trained using reinforcement learning);
training the (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle).
HALDER does not explicitly disclose: training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries.
However, KOBILAROV discloses training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries (see col 21, ln 9-46; Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. Gradient Boosted Regression Trees (GBRT), Random Forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc. See also col 1, ln 58-col 2, ln 26; reinforcement learning with deep neural networks).
HALDER further discloses provide a set of input requirements to the supervised learning model (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route. See also ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride); and
generate a vehicle transportation itinerary by the supervised learning model based on the set of input requirements (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. SILVER discloses achieving superhuman performance using reinforcement learning from games of self-play that compare outcomes from players to determine winners. It would have been obvious for one of ordinary skill in the art at the time of invention to use self-play as taught by SILVER in the system executing the method of HALDER with the motivation to achieve superhuman levels of performance in reinforcement learning and select a winning player with a best route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. KOBILAROV discloses trajectory generation that optimizes costs and constraints at nodes to generate actions. It would have been obvious for one of ordinary skill in the art at the time of invention to include the model as taught by KOBILAROV in the system executing the method of HALDER with the motivation to select a best route for a vehicle.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. BOUSMALIS discloses transforming source domain images into target domain images that uses concatenated input and a convolutional sub-neural network with residual blocks and convolutional layers to preserve resolution. It would have been obvious for one of ordinary skill in the art at the time of invention to include the neural network structure as taught by BOUSMALIS in the system executing the method of HALDER with the motivation to generate a predictive model for selecting a route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. YE discloses vessel scheduling that ranks vehicle candidates based on cost. It would have been obvious for one of ordinary skill in the art at the time of invention to include the scheduling as taught by YE in the system executing the method of HALDER with the motivation to select a vehicle and route with optimized cost.
Claim 12 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the computer-readable media as set forth in claim 8.
HALDER further discloses wherein the set of input requirements further comprises one or more of a set of activities, activity start/end times, employees, employee clock-in/clock-out times, contractors, contractor clock-in/clock-out times, driver ratings, and vehicle types (see ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride).
Claim 15 (Currently Amended)
HALDER discloses a method for training a (see ¶[0050], [0055]-[0058], and [0131]; historical data may include attributes of a route. Pickup and drop-off complexity index is computed based on start/end location and total distance to travel historical data associated with real-world locations and their characteristics);
obtaining a past cost function associated with the past itinerary from a machine learning algorithm configured to generate vehicle transportation itineraries (see ¶[0015] and [0058]; training data for a pickup-dropoff complexity index can be fit to a mathematical function. A risk value is predicted based on the PDCI. See also ¶[0007]-[0008]; the risk or loss value for each category may be computed based on historical data);
training a reinforcement learning model to generate an optimal itinerary (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle),
HALDER does not specifically disclose, but SILVER discloses, by setting the past itinerary against a plurality of itineraries (see title and abstract and p. 3; achieve superhuman levels of performance through reinforcement learning from games of self-play. Use a general purpose Monte-Carlo tree search (MCTS) algorithm. Each search consists of a series of simulated games of self-play that traverse a tree from root sroot to leaf. Each simulation proceeds by selecting in each state s a move a with low visit count, high move probability and high value (averaged over the leaf states of simulations that selected a from s) according to the current neural network f . Games are played by selecting moves for both players by MCTS, at t . At the end of the game, the terminal position sT is scored according to the rules of the game to compute the game outcome z: 1 for a loss, 0 for a draw, and +1 for a win.).
HALDER further discloses, wherein each itinerary comprises a plurality of nodes representing a plurality of decisions along a route associated with each itinerary (see abstract; determining a route can include a search algorithm. Trajectories can be selected based on various costs and constraints that are optimized for performance. Nodes can be generated and actions can be explored based on machine learning, including a deep neural network),
HALDER further discloses, wherein the past itinerary is a best itinerary according to the past cost function (see ¶[0066]; the route to take is selected as whichever route is associated with the lowest insurance premium value, this would minimize the risk for the AV ride while also minimizing the insurance premium charged to the user);
HALDER does not specifically disclose, but KOBILAROV discloses, generating the plurality of itineraries by repeating steps until a new determination point is reached or a maximum step count threshold value is reached (see col 4, ln 49-col 5, ln 3; decisions can be made in real time. Determine trajectories to evaluate based on a current state or goal. See also col 17, ln 23-28; a policy can be learned iteratively).
HALDER does not specifically disclose, but BOUSMALIS discloses, repeating the steps of: generating input features comprising concatenated planes including activity assignment statuses for each of a first player and a second player (see ¶[0007]; concatenate input images with an additional channel to generate a combined input);
processing the input features by a residual tower comprising a convolution block and one or more residual blocks see ¶[0007]; The convolutional sub-neural network may comprise a plurality of residual blocks each comprising a respective plurality of resolution-preserving convolutional layers.); and
processing outputs of the residual tower by policy head to calculate a policy at each node and a value head to calculate a value at each node (see ¶[0025] and [0055]; develop a control policy for a vehicle).
HALDER does not specifically disclose, but KOBILAROV discloses, determining a first route for a first driver by minimizing the past cost function at each node of the plurality of nodes based on the policy and the value (see abstract and col 1, ln 58-col 2, ln 26; ; trajectories can be selected based on various costs and constraints optimized for performance. Temporal-logic as applied to policy-learning for autonomous vehicles).
The combination of HALDER and KOBILAROV does not specifically disclose, but YE discloses, determining a second route for a second driver by minimizing a new cost function distinct from the past cost function at each node of the plurality of nodes based on the policy and the value (see claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate);
determining that the second route includes a minimum result by comparing the first route and the second route (see again claims 1, 5, and 6; rank vehicle candidates based on cost and schedule the best vehicle candidate).
HALDER further discloses storing an itinerary including the second route as one of the plurality of itineraries (see ¶[0068] and [0100]-[0102]; the data depicted in FIG. 1 under data sources 106 is generally stored in a computer-readable format indicating values for different attributes under the vehicle, route, weather and road condition, or other categories);
storing the reinforcement learning model including the plurality of itineraries (see ¶[0140]; use a model trained using reinforcement learning);
training the (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner. Identify one or more goals of autonomous vehicle).
HALDER does not explicitly disclose training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries.
However, KOBILAROV discloses training the SUPERVISED learning model by the reinforcement learning model including the plurality of itineraries (see col 21, ln 9-46; Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. Gradient Boosted Regression Trees (GBRT), Random Forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc. See also col 1, ln 58-col 2, ln 26; reinforcement learning with deep neural networks).
HALDER further discloses, provide a set of input requirements to the supervised learning model (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route. See also ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride); and
generate a vehicle transportation itinerary by the supervised learning model based on the set of input requirements (see ¶[0136]-[0140]; use a model trained using reinforcement learning for generating and updating the plan of action in order to achieve a particular goal in a safe manner).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. SILVER discloses achieving superhuman performance using reinforcement learning from games of self-play that compare outcomes from players to determine winners. It would have been obvious for one of ordinary skill in the art at the time of invention to use self-play as taught by SILVER in the system executing the method of HALDER with the motivation to achieve superhuman levels of performance in reinforcement learning and select a winning player with a best route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. KOBILAROV discloses trajectory generation that optimizes costs and constraints at nodes to generate actions. It would have been obvious for one of ordinary skill in the art at the time of invention to include the model as taught by KOBILAROV in the system executing the method of HALDER with the motivation to select a best route for a vehicle.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. BOUSMALIS discloses transforming source domain images into target domain images that uses concatenated input and a convolutional sub-neural network with residual blocks and convolutional layers to preserve resolution. It would have been obvious for one of ordinary skill in the art at the time of invention to include the neural network structure as taught by BOUSMALIS in the system executing the method of HALDER with the motivation to generate a predictive model for selecting a route.
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using reinforcement learning. YE discloses vessel scheduling that ranks vehicle candidates based on cost. It would have been obvious for one of ordinary skill in the art at the time of invention to include the scheduling as taught by YE in the system executing the method of HALDER with the motivation to select a vehicle and route with optimized cost.
Claim 18 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the method as set forth in claim 15.
HALDER further discloses wherein the set of input requirements further comprises one or more of a set of activities, activity start/end times, employees, employee clock- in/clock-out times, contractors, contractor clock-in/clock-out times, driver ratings, and vehicle types (see ¶[0008], [0014], and [0053]-[0055]; risk value is calculated based on the time of the ride).
Claim(s) 2-4, 7, 9-11, 14, 16, 17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20200111169 A1 to HALDER et al. in view of SILVER and US 10133275 B1 to KOBILAROV et al., US 20190304065 A1 to BOUSMALIS et al., and US 20150170094 A1 to YE et al. as applied to claim 1 above, and further in view of US 20170032480 A1 to Wong et al. (hereinafter ‘WONG’).
Claim 2 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the system as set forth in claim 1.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: sending instructions for displaying the vehicle transportation itinerary to a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER further discloses sending further instructions for providing turn-by-turn navigation for each location on the vehicle transportation itinerary (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses a supervised learning model to provide a travel plan display the travel plan to a user. It would have been obvious to use the supervised learning and transmit the itinerary to a user as taught by WONG in the system executing the method of HALDER with the motivation to select a route and communicate the route to a user.
Claim 3 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, YE, and WONG discloses the system as set forth in claim 2.
HALDER further discloses wherein the set of input requirements further comprises a set of actions to be performed at one or more of a plurality of geographic coordinates, and wherein the vehicle transportation itinerary includes the set of actions (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
Claim 4 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the system as set forth in claim 1.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: querying an external data source to receive external data (see ¶[0007]; the method may comprise capturing, by a processor, user's input, user's personal data and user's social networking data, wherein the user's input comprises at least a travel destination and a travel duration); and
generating an updated vehicle transportation itinerary based on the external data (see ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses real-time updating of data to adapt to changes. It would have been obvious to use the updating of data in real-time as taught by WONG in the system executing the method of HALDER with the motivation to adapt to changes and provide a route to a user.
Claim 7 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the system as set forth in claim 1.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: receive updated input requirements (see ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive);
generating an updated vehicle transportation itinerary based on the updated input requirements (see again ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive); and sending the updated vehicle transportation itinerary to a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses real-time updating of data to adapt to changes. It would have been obvious to use the updating of data in real-time as taught by WONG in the system executing the method of HALDER with the motivation to adapt to changes and provide a route to a user.
Claim 9 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the computer-readable media as set forth in claim 8.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: sending instructions for displaying the vehicle transportation itinerary by a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER further discloses sending further instructions for providing turn-by-turn navigation for each location on the vehicle transportation itinerary (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses a supervised learning model to provide a travel plan display the travel plan to a user. It would have been obvious to use the supervised learning and transmit the itinerary to a user as taught by WONG in the system executing the method of HALDER with the motivation to select a route and communicate the route to a user.
Claim 10 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, YE, and WONG discloses the computer-readable media as set forth in claim 9.
HALDER further discloses wherein the set of input requirements further comprises a set of actions to be performed at one or more of a plurality of geographic coordinates, and wherein the vehicle transportation itinerary includes the set of actions (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
Claim 11 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the computer-readable media as set forth in claim 8.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: querying an external data source to receive external data (see ¶[0007]; the method may comprise capturing, by a processor, user's input, user's personal data and user's social networking data, wherein the user's input comprises at least a travel destination and a travel duration); and
generating an updated vehicle transportation itinerary based on the external data (see ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses real-time updating of data to adapt to changes. It would have been obvious to use the updating of data in real-time as taught by WONG in the system executing the method of HALDER with the motivation to adapt to changes and provide a route to a user.
Claim 14 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the computer-readable media as set forth in claim 8.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, wherein the method further comprises: receiving updated input requirements (see ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive);
generating an updated vehicle transportation itinerary based on the updated input requirements (see again ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive); and sending the updated vehicle transportation itinerary to a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses real-time updating of data to adapt to changes. It would have been obvious to use the updating of data in real-time as taught by WONG in the system executing the method of HALDER with the motivation to adapt to changes and provide a route to a user.
Claim 16 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the method as set forth in claim 15.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, further comprising: sending instructions for displaying the vehicle transportation itinerary to a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER further discloses sending further instructions for providing turn-by-turn navigation for each location on the vehicle transportation itinerary (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses a supervised learning model to provide a travel plan display the travel plan to a user. It would have been obvious to use the supervised learning and transmit the itinerary to a user as taught by WONG in the system executing the method of HALDER with the motivation to select a route and communicate the route to a user.
Claim 17 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, YE, and WONG discloses the method as set forth in claim 16.
HALDER further discloses wherein the set of input requirements further comprises a set of actions to be performed at one or more of a plurality of geographic coordinates, and wherein the vehicle transportation itinerary includes the set of actions (see ¶[0098]; in some embodiments, details of the route (e.g., turn-by-turn navigation or driving instructions) may be included in the request received in 402 so that the insurance provider system does not have to determine the route).
Claim 20 (Previously Presented)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the method as set forth in claim 15.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but WONG discloses, further comprising: receiving updated input requirements (see ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive);; generating an updated vehicle transportation itinerary based on the updated input requirements (see again ¶[0049]; in an embodiment, the travel planning module 214 may be configured to update, in a real time, the travel itinerary and hence the travel itinerary/travel plan is adaptive); and
sending the updated vehicle transportation itinerary to a mobile device (see abstract & Figs. 1, 5, and 7; & claim 1; provide a personalized travel plan and guidance to the user. The travel itinerary is further displayed on a display device of the user).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where premiums are calculated using machine learning. WONG discloses a travel planning and guidance system that uses real-time updating of data to adapt to changes. It would have been obvious to use the updating of data in real-time as taught by WONG in the system executing the method of HALDER with the motivation to adapt to changes and provide a route to a user.
Claim(s) 6, 13, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20200111169 A1 to HALDER et al. in view of SILVER and US 10133275 B1 to KOBILAROV et al., US 20190304065 A1 to BOUSMALIS et al., and US 20150170094 A1 to YE et al. as applied to claim 1 above, and further in view of US 20170316324 A1 to Barret et al. (hereinafter ‘BARRETT’).
Claim 6 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the system as set forth in claim 1.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but BARRETT discloses, wherein the supervised learning model is a neural network using a long short term memory encoder-decoder framework (see ¶[0025], [0095], [0211] and [0235]-[0236]; generate activity schedules using supervised machine learning, neural networks, and lstm networks. Model traffic congestion in an area).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where traffic conditions may be forecast to predict risk (see ¶[0053]). BARRETT discloses an event-forecasting interface that includes modeling traffic, where the model may include supervised learning, neural networks, and lstm networks). It would have been obvious to include the supervised learning, neural networks, and lstm as taught by BARRETT in the system executing the method of HALDER with the motivation to predict risk associated with a route.
Claim 13 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the computer-readable media as set forth in claim 8.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but BARRETT discloses, wherein the supervised learning model is a neural network using a long short term memory encoder-decoder framework (see ¶[0025], [0095], [0211] and [0235]-[0236]; generate activity schedules using supervised machine learning, neural networks, and lstm networks. Model traffic congestion in an area).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where traffic conditions may be forecast to predict risk (see ¶[0053]). BARRETT discloses an event-forecasting interface that includes modeling traffic, where the model may include supervised learning, neural networks, and lstm networks). It would have been obvious to include the supervised learning, neural networks, and lstm as taught by BARRETT in the system executing the method of HALDER with the motivation to predict risk associated with a route.
Claim 19 (Original)
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE discloses the method as set forth in claim 15.
The combination of HALDER, SILVER, KOBILAROV, BOUSMALIS, and YE does not specifically disclose, but BARRETT discloses, wherein the supervised learning model is a neural network using a long short term memory encoder-decoder framework (see ¶[0025], [0095], [0211] and [0235]-[0236]; generate activity schedules using supervised machine learning, neural networks, and lstm networks. Model traffic congestion in an area).
HALDER discloses autonomous vehicle premium computation using predictive models that discloses selecting a route based on the lowest premium and risk, where traffic conditions may be forecast to predict risk (see ¶[0053]). BARRETT discloses an event-forecasting interface that includes modeling traffic, where the model may include supervised learning, neural networks, and lstm networks). It would have been obvious to include the supervised learning, neural networks, and lstm as taught by BARRETT in the system executing the method of HALDER with the motivation to predict risk associated with a route.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD N SCHEUNEMANN whose telephone number is (571)270-7947. The examiner can normally be reached M-F 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia Munson can be reached at 571-270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD N SCHEUNEMANN/Primary Examiner, Art Unit 3624
Prosecution Insights