DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 11 are rejected under 35 U.S.C. 101 because the claim invention is directed
toward an abstract idea with significantly more.
Regarding claim 1,
101 Analysis – Step 1
Claim 1 is directed toward a method of training a model toward the determination of a vehicle’s driving route which includes the processes of extracting a latent vector based on trajectories corresponding to maneuver modes that the vehicle is capable of selecting in a driving situation. Subsequently, the model is trained to generate route distributions for the driving route based on the driving dataset and a latent vector in which the trajectories are generating by applying weights of indicators for maneuvers differently (according to the maneuver modes) via a route search algorithm (a process). Therefore, claim 1 is within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong I
Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to
determine whether they recite subject matter that falls within one of the follow groups of abstract
ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental
processes.
Independent claim 1 includes limitations that recite an abstract idea (emphasized below) and
will be used as a representative claim for the remainder of the 101 rejection. Claim 1 recites:
A method of training a model for determining a driving route of a vehicle, the method comprising:
extracting a latent vector based on trajectories corresponding to maneuver modes that the vehicle is capable of selecting in a driving situation;
and training the model for generating route distributions to determine the driving route of the vehicle, based on a driving dataset and the latent vector, wherein the trajectories are generated by applying, to a route search algorithm, weights of indicators for maneuvers differently according to the maneuver modes.
The examiner submits that the foregoing bolded limitation constitutes a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “extracting”, in the context of this claim encompasses a person (driver) looking at information collected and forming a simple judgment.
Accordingly, the claim recites at one abstract idea.
101 Analysis - Step 2A, Prong II
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into the practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.” In the present case, a route search algorithm is provided for the purpose of implementing the abstract idea. However, this feature is a mere computer and does not give practical application to the extraction step.
101 Analysis – Step 2B
Regarding Step 2B of the 2019 PEG, as noted above, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional limitations that amount to significantly more.
Dependent claim 5 does not recite any further limitations that cause the claim to be patent eligible. Rather, the limitations of the dependent claim are directed toward additional aspects of the judicial exception and/or well-understood, routine, and conventional additional elements that do not integrate the judicial exception into a practical application.
Claim 5 uses the limitation of “applying, to the route search algorithm, an operating range corresponding to each of the maneuver modes” which amounts to data gathering and is a form of insignificant extra-solution activity.
Regarding claim 11,
101 Analysis – Step 1
Claim 11 is directed toward an apparatus for training a model which contains a memory and processor configured to execute stored instructions in which the processor controls a series of operations which include extracting a latent vector based on trajectories corresponding to maneuver modes that the vehicle is capable of selecting in a driving situation. Subsequently, the model is trained to generate route distributions for the driving route based on the driving dataset and a latent vector in which the trajectories are generating by applying weights of indicators for maneuvers differently (according to the maneuver modes) via a route search algorithm (a process). Therefore, claim 11 is within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong I
Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to
determine whether they recite subject matter that falls within one of the follow groups of abstract
ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental
processes.
Independent claim 11 includes limitations that recite an abstract idea (emphasized below) and
will be used as a representative claim for the remainder of the 101 rejection. Claim 11 recites:
An apparatus for training a model, the apparatus comprising:
a memory configured to store instructions;
and a processor electrically connected to the memory and configured to execute the instructions, wherein, when the instructions are executed by the processor, the processor is configured to control a plurality of operations, and wherein the plurality of operations comprises:
extracting a latent vector based on trajectories corresponding to maneuver modes that a vehicle is capable of selecting in a driving situation;
and training the model for generating route distributions to determine a driving route of the vehicle, based on a driving dataset and the latent vector, wherein the trajectories are generated by applying, to a route search algorithm, weights of indicators for maneuvers differently according to the maneuver modes.
The examiner submits that the foregoing bolded limitation constitutes a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “extracting”, in the context of this claim encompasses a person (driver) looking at information collected and forming a simple judgment.
Accordingly, the claim recites at one abstract idea.
101 Analysis - Step 2A, Prong II
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into the practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.” In the present case, a memory, a processor, and a route search algorithm are provided for the purpose of implementing the abstract idea. However, these features are mere computers and do not give practical application to the extraction step.
101 Analysis – Step 2B
Regarding Step 2B of the 2019 PEG, as noted above, representative independent claim 11 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional limitations that amount to significantly more.
Dependent claims 15 and 20 do not recite any further limitations that cause the claim to be patent eligible. Rather, the limitations of the dependent claim are directed toward additional aspects of the judicial exception and/or well-understood, routine, and conventional additional elements that do not integrate the judicial exception into a practical application.
Claim 15 uses the limitation of “applying, to the route search algorithm, an operating range corresponding to each of the maneuver modes”, which amounts to data gathering and is a form of insignificant extra-solution activity.
Claim 20 uses the limitation of “obtaining a route distribution by inputting the driving dataset to the trained model”, which amounts to data gathering and is a form of insignificant extra-solution activity.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 10-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Smolyanskiy, et al. (U.S. Patent Application Publication No. 20220138568) in view of Narayanan, et al. (U.S. Patent No. 12005892).
Regarding claim 1, Smolyanskiy, et al. teaches: A method of training a model for determining a driving route of a vehicle, the method comprising: extracting a latent vector based on trajectories corresponding to maneuver modes that the vehicle is capable of selecting in a driving situation; (Paragraph [0033]: "…learn a policy that maps the state at a certain time to a certain action, such as an ego-vehicle acceleration. A task reward may be defined by a sum of manually designed reward terms such as distance to the lead car, acceleration changes, etc., as well as a sparse penalty term accounting for the rare events such as cut-ins and collisions. The trained RL policy may thus, over iterations, learn to plan so as to reduce the gap and avoid cut-ins, and to keep a slower speed and avoid future harsh braking [various maneuvers which can be selected in driving situation]." ; Paragraph [0034]: "A latent vector may be fed into three fully convolutional layers to extract features, in which each of the policy and critic networks may be a three - layer multilayer perceptron (MLP) [extraction of latent vector]." ; Block (802), Paragraph [0107]: "FIG. 8 is a flow diagram showing a method (800) for training a machine learning model to predict actor movements using actor positions predicted using a DNN, […] The method (800), at block (B802), includes determining an actor position using a simulator. […] a position of an actor may include, without limitation, one or more of the actor's location, pose, orientation, size, height, yaw, pitch, or roll, etc.) For example, the training engine (112) may determine, using a simulation corresponding to the world state (118), a first at least one position of one or more actors based at least on a first state of an environment. The one or more actors may include the vehicle and/or at least one other vehicle [method application - collecting data corresponding to potential maneuver modes in driving situation].")
wherein the trajectories are generated by applying, to a route search algorithm, weights of indicators for maneuvers differently according to the maneuver modes (Paragraph [0086]: "…because the occupancy scores (e.g., from the confidence fields (420)) are not probabilities, to avoid over-spreading trajectories, a sharpening operation may be performed. For example, a sharpening operation may be applied to the confidence fields (420) to assign higher weights to higher confidence scored points before computing the weighted average [weights are different for various trajectories related to route algorithm].").
Smolyanskiy, et al. does not teach and training the model for generating route distributions to determine the driving route of the vehicle, based on a driving dataset and the latent vector.
In a similar field of endeavor (simulation of vehicle trajectories road scenes), Narayanan, et al. teaches: and training the model for generating route distributions to determine the driving route of the vehicle, based on a driving dataset and the latent vector, (Col. 9, lines 4-16: "…The network architecture includes a latent encoder (610) [based on latent vector] and a conditional generator (640). The exemplary embodiments model the temporal information with the agents' previous locations using convolutional Long-Short-Term Memories (ConvLSTMs). […] a state pooling operation to feed agents state information at respective locations in consecutive timestep [training model]. While the exemplary embodiments provide trajectory specific labels to capture diverse predictions, the exemplary embodiments leverage conditional variational generative models to model diversity in the data for each type of label [based on route distributions and driving dataset]." ; Block (813), Col. 12, lines 43-48: "At block (813), feed the road graph and the sampled velocity profile with a desired destination to a dynamics simulator to generate a plurality of simulated diverse trajectories output on a visualization device to allow a user to select one or more of the plurality of simulated diverse trajectories for recreating a desired driving scenario [method claim - application of training concepts].").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify Smolyanskiy, et al. to include the teaching of Narayanan, et al. based on a reasonable expectation of success and motivation to improve the process of simultaneous multi-agent recurrent trajectory projection of vehicle-based road scenes (Narayanan, et al. Col. 1, lines 14-16 and Col. 1, lines 48-67).
Regarding claim 2, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, wherein the indicators comprise a first indicator for a search time, (Smolyanskiy, et al. Paragraph [0073]: "the simulator (116) may use a physics engine to determine and/or detect one or more of the events and/or attributes used to compute the value function for one or more states or time steps [indicator for search time].")
a second indicator for a reference velocity, (Smolyanskiy, et al. Paragraph [0041]: "…the prediction MLM (104) may receive as input one or more locations and/or other attributes (e.g., velocity [reference velocity], […] for one or more actors encoded in the simulation data (102).")
a third indicator for a lateral movement, (Smolyanskiy, et al. Paragraph [0099]: "The control component(s) may follow a trajectory or path (lateral […] that has been received from the behavior selector of the planning component (s) as closely as possible and within the capabilities of the vehicle (1100) [lateral movement].")
a fourth indicator for a longitudinal movement, (Smolyanskiy, et al. Paragraph [0099]: "The control component(s) may follow a trajectory or path […] and longitudinal) that has been received from the behavior selector of the planning component (s) as closely as possible and within the capabilities of the vehicle (1100) [lateral movement].")
and a fifth indicator for a heading angle (Smolyanskiy, et al. Paragraph [0064]: "…road shape, elevation, slope, and/or contour, heading information [heading angle].").
Regarding claim 3, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, wherein the driving dataset comprises a velocity of the vehicle, map information, and occupancy information of obstacles around the vehicle ("Paragraph [0030]: "…the classical mechanical motion algorithm may extend the trajectory in a direction of travel of the actor based on the velocity, acceleration, and/or other outputs predicted by the MLM(s) [velocity of vehicle]." ; Paragraph [0043]: "The training engine (112) and/or the simulator (116) may associate any of this various information with map information (e.g., after decoding is performed using the world state decoder (110)), such as lane information, as indicated by the lane lines and trajectories for various actors [map information]." ; Paragraph [0093]: "The obstacle perceiver may perform obstacle perception that may be based on where the vehicle (1100) is allowed to drive or is capable of driving (e.g., based on the location of the drivable paths defined by avoiding detected obstacles), and how fast the vehicle (1100) can drive without colliding with an obstacle (e.g., an object, such as a structure, entity, vehicle, etc.) that is sensed by the sensors of the vehicle (1100) [occupancy information of obstacles with respect to vehicle].").
Regarding claim 4, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, wherein the maneuver modes comprise a first maneuver mode for keeping a lane, (Smolyanskiy, et al. Paragraph [0094]: "The path perceiver may perform path perception, such as by perceiving nominal paths that are available in a particular situation. In some examples, the path perceiver may further consider (e.g., account for) lane changes for path perception. A lane graph (e.g., generated using, at least in part, the HD map (404)) may represent the path or paths available to the vehicle (1100), […] the lane graph may include paths to a desired lane [maneuver modes for keeping lane] and/or may indicate available changes down the highway (or other road type), or may include nearby lanes, lane changes, forks, turns, cloverleaf interchanges, merges, and/or other information.")
a second maneuver mode for changing a lane, (Smolyanskiy, et al. Paragraph [0094]: "The path perceiver may perform path perception, such as by perceiving nominal paths that are available in a particular situation. In some examples, the path perceiver may further consider (e.g., account for) lane changes for path perception. A lane graph (e.g., generated using, at least in part, the HD map (404)) may represent the path or paths available to the vehicle (1100), […] and/or may indicate available changes down the highway (or other road type), or may include nearby lanes, lane changes [maneuver mode for changing lane]")
a third maneuver mode for stopping the vehicle, (Smolyanskiy, et al. Paragraph [0095]: "Thus, the wait perceiver may be leveraged to identify potential obstacles and implement one or more controls (e.g., slowing down, coming to a stop, etc.) [stopping mode maneuver]")
a fourth maneuver mode for swerving from obstacles around the vehicle, (Smolyanskiy, et al. Paragraph [0100]: "The obstacle avoidance component (s) may aid the autonomous vehicle (1100) in avoiding collisions with objects (e.g., moving or stationary objects) [swerving away from obstacles around vehicle].")
and a fifth maneuver mode for following trajectories of obstacles around the vehicle (Smolyanskiy, et al. Paragraph [0101]: "…the drivable paths and/or object detections may be used by the obstacle avoidance component(s) in determining controls or actions to take. For example, the drivable paths may provide an indication to the obstacle avoidance component(s) of where the vehicle (1100) may maneuver without striking any objects, structures, and/or the like, or at least where no static structures may exist [following trajectories of obstacles around vehicle].").
Regarding claim 5, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, wherein the trajectories are generated by applying, to the route search algorithm, an operating range corresponding to each of the maneuver modes (Smolyanskiy, et al. Method (800), Paragraph [0112]: "…predicting one or more positions of the one or more actors using the DNN, wherein the determining the first at least one position of one or more actors includes adjusting at least one of the one or more positions based on modeling behavior of an actor to generate the first at least one position. The second at least one position of the one or more actors may correspond to a trajectory of an actor and the method includes extending the trajectory using a classical mechanical motion algorithm to generate an extended trajectory, wherein the one or more scores correspond to the extended trajectory [extended trajectory based on machine learning algorithm - generates specific trajectory based on scoring system from first to second position].").
Regarding claim 6, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 5, and in a further embodiment, teach: The method of claim 5, wherein the operating range comprises an operating range for at least one of a steering angle and a longitudinal acceleration (Smolyanskiy, et al. Paragraph [0034]: "The ACC reward function may denote the changes in acceleration [operating range for longitudinal acceleration] […] the distance to the lead car, and may be tuned to accomplish smooth following in real - world scenarios.")
Regarding claim 7, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, wherein the weights are determined based on one or more of the maneuver modes and information about the driving situation (Smolyanskiy, et al. Paragraph [0086]: "…because the occupancy scores (e.g., from the confidence fields (420)) are not probabilities, to avoid over-spreading trajectories, a sharpening operation may be performed. For example, a sharpening operation may be applied to the confidence fields (420) to assign higher weights [weights are determined to refine specific maneuver position or trajectory] to higher confidence scored points before computing the weighted average." ; Smolyanskiy, et al. Paragraph [0088]: "once the trajectories have been computed—and converted from 2D image-space coordinates to 3D world-space coordinates, in embodiments—the trajectories may be used by the autonomous vehicle (1100) in performing one or more operations (e.g., obstacle avoidance, lane keeping, lane changing, path planning, mapping, etc.) [maneuver modes - applications].").
Regarding claim 8, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 7, and in a further embodiment, teach: The method of claim 7, wherein the information about the driving situation comprises lane information and information about obstacles around the vehicle (Smolyanskiy, et al. Paragraph [0088]: "the trajectories may be used by the autonomous vehicle (1100) in performing one or more operations [driving situation] (e.g., obstacle avoidance [obstacle information], lane keeping, lane changing [lane information]").
Regarding claim 10, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 1, and in a further embodiment, teach: The method of claim 1, further comprising: obtaining a route distribution by inputting the driving dataset to the trained model; (Smolyanskiy, et al. Block (B1008), Paragraph [0118]: "The method (1000), at block (B1008), includes applying data to a neural network to predict scores of a driving policy. This may include applying second data corresponding to the predictions to a neural network (e.g., the value function MLM (108)) to generate, using the neural network, predictions of one or more scores of a value function, the one or more scores corresponding to one or more driving policies [getting route distribution by inputting driving data into trained model].")
and controlling driving of the vehicle based on the route distribution (Smolyanskiy, et al. Method (B1010), Paragraph [0119]: "The method (1000), at block (B1010), includes determining a driving policy that corresponds to the one or more scores. The method (100), at block (B1012) includes performing a vehicle action based on the driving policy. This may include transmitting data causing a vehicle to perform one or more actions based on the one or more driving policies [controlling driving of vehicle based on route distribution].").
Regarding claim 11, Smolyanskiy, et al. teaches: An apparatus for training a model, the apparatus comprising: a memory configured to store instructions; (Paragraph [0035]: "…processor executing instructions stored in memory [memory storing instructions]")
and a processor electrically connected to the memory and configured to execute the instructions, (Paragraph [0035]: "…processor executing instructions stored in memory [processor executing instructions]" ; Paragraph [0229]: "As an example, the CPU (1206) may be directly connected to the memory (1204) [processor directly connected to memory].")
wherein, when the instructions are executed by the processor, the processor is configured to control a plurality of operations, and wherein the plurality of operations comprises: (Paragraph [0035]: "For instance, various functions may be carried out by a processor executing instructions stored in memory [instructions executed by processor with plurality of operations]")
extracting a latent vector based on trajectories corresponding to maneuver modes that a vehicle is capable of selecting in a driving situation; (Paragraph [0033]: "…learn a policy that maps the state at a certain time to a certain action, such as an ego-vehicle acceleration. A task reward may be defined by a sum of manually designed reward terms such as distance to the lead car, acceleration changes, etc., as well as a sparse penalty term accounting for the rare events such as cut-ins and collisions. The trained RL policy may thus, over iterations, learn to plan so as to reduce the gap and avoid cut-ins, and to keep a slower speed and avoid future harsh braking [various maneuvers which can be selected in driving situation]." ; Paragraph [0034]: "A latent vector may be fed into three fully convolutional layers to extract features, in which each of the policy and critic networks may be a three - layer multilayer perceptron (MLP) [extraction of latent vector].")
wherein the trajectories are generated by applying, to a route search algorithm, weights of indicators for maneuvers differently according to the maneuver modes (Paragraph [0086]: "…because the occupancy scores (e.g., from the confidence fields (420)) are not probabilities, to avoid over-spreading trajectories, a sharpening operation may be performed. For example, a sharpening operation may be applied to the confidence fields (420) to assign higher weights to higher confidence scored points before computing the weighted average [weights are different for various trajectories related to route algorithm].").
Smolyanskiy, et al. does not teach and training the model for generating route distributions to determine a driving route of the vehicle, based on a driving dataset and the latent vector.
In a similar field of endeavor (simulation of vehicle trajectories road scenes), Narayanan, et al. teaches: and training the model for generating route distributions to determine a driving route of the vehicle, based on a driving dataset and the latent vector (Col. 9, lines 4-16: "…The network architecture includes a latent encoder (610) [based on latent vector] and a conditional generator (640). The exemplary embodiments model the temporal information with the agents' previous locations using convolutional Long-Short-Term Memories (ConvLSTMs). […] a state pooling operation to feed agents state information at respective locations in consecutive timestep [training model]. While the exemplary embodiments provide trajectory specific labels to capture diverse predictions, the exemplary embodiments leverage conditional variational generative models to model diversity in the data for each type of label [based on route distributions and driving dataset].").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify Smolyanskiy, et al. to include the teaching of Narayanan, et al. based on a reasonable expectation of success and motivation to improve the process of simultaneous multi-agent recurrent trajectory projection of vehicle-based road scenes (Narayanan, et al. Col. 1, lines 14-16 and Col. 1, lines 48-67).
Regarding claim 12, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the indicators comprise a first indicator for a search time, (Smolyanskiy, et al. Paragraph [0073]: "the simulator (116) may use a physics engine to determine and/or detect one or more of the events and/or attributes used to compute the value function for one or more states or time steps [indicator for search time].")
a second indicator for a reference velocity, (Smolyanskiy, et al. Paragraph [0041]: "…the prediction MLM (104) may receive as input one or more locations and/or other attributes (e.g., velocity [reference velocity], […] for one or more actors encoded in the simulation data (102).")
a third indicator for a lateral movement, (Smolyanskiy, et al. Paragraph [0099]: "The control component(s) may follow a trajectory or path (lateral […] that has been received from the behavior selector of the planning component (s) as closely as possible and within the capabilities of the vehicle (1100) [lateral movement].")
a fourth indicator for a longitudinal movement, (Smolyanskiy, et al. Paragraph [0099]: "The control component(s) may follow a trajectory or path […] and longitudinal) that has been received from the behavior selector of the planning component (s) as closely as possible and within the capabilities of the vehicle (1100) [lateral movement].")
and a fifth indicator for a heading angle (Smolyanskiy, et al. Paragraph [0064]: "…road shape, elevation, slope, and/or contour, heading information [heading angle].").
Regarding claim 13, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the driving dataset comprises a velocity of the vehicle, map information, and occupancy information of obstacles around the vehicle (Paragraph [0030]: "…the classical mechanical motion algorithm may extend the trajectory in a direction of travel of the actor based on the velocity, acceleration, and/or other outputs predicted by the MLM(s) [velocity of vehicle]." ; Paragraph [0043]: "The training engine (112) and/or the simulator (116) may associate any of this various information with map information (e.g., after decoding is performed using the world state decoder (110)), such as lane information, as indicated by the lane lines and trajectories for various actors [map information]." ; Paragraph [0093]: "The obstacle perceiver may perform obstacle perception that may be based on where the vehicle (1100) is allowed to drive or is capable of driving (e.g., based on the location of the drivable paths defined by avoiding detected obstacles), and how fast the vehicle (1100) can drive without colliding with an obstacle (e.g., an object, such as a structure, entity, vehicle, etc.) that is sensed by the sensors of the vehicle (1100) [occupancy information of obstacles with respect to vehicle].").
Regarding claim 14, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the maneuver modes comprise a first maneuver mode for keeping a lane, (Smolyanskiy, et al. Paragraph [0094]: "The path perceiver may perform path perception, such as by perceiving nominal paths that are available in a particular situation. In some examples, the path perceiver may further consider (e.g., account for) lane changes for path perception. A lane graph (e.g., generated using, at least in part, the HD map (404)) may represent the path or paths available to the vehicle (1100), […] the lane graph may include paths to a desired lane [maneuver modes for keeping lane] and/or may indicate available changes down the highway (or other road type), or may include nearby lanes, lane changes, forks, turns, cloverleaf interchanges, merges, and/or other information.")
a second maneuver mode for changing a lane, (Smolyanskiy, et al. Paragraph [0094]: "The path perceiver may perform path perception, such as by perceiving nominal paths that are available in a particular situation. In some examples, the path perceiver may further consider (e.g., account for) lane changes for path perception. A lane graph (e.g., generated using, at least in part, the HD map (404)) may represent the path or paths available to the vehicle (1100), […] and/or may indicate available changes down the highway (or other road type), or may include nearby lanes, lane changes [maneuver mode for changing lane]")
a third maneuver mode for stopping the vehicle, (Smolyanskiy, et al. Paragraph [0095]: "Thus, the wait perceiver may be leveraged to identify potential obstacles and implement one or more controls (e.g., slowing down, coming to a stop, etc.) [stopping mode maneuver]")
a fourth maneuver mode for swerving from obstacles around the vehicle, (Smolyanskiy, et al. Paragraph [0100]: "The obstacle avoidance component (s) may aid the autonomous vehicle (1100) in avoiding collisions with objects (e.g., moving or stationary objects) [swerving away from obstacles around vehicle].")
and a fifth maneuver mode for following trajectories of obstacles around the vehicle (Smolyanskiy, et al. Paragraph [0101]: "…the drivable paths and/or object detections may be used by the obstacle avoidance component(s) in determining controls or actions to take. For example, the drivable paths may provide an indication to the obstacle avoidance component(s) of where the vehicle (1100) may maneuver without striking any objects, structures, and/or the like, or at least where no static structures may exist [following trajectories of obstacles around vehicle].").
Regarding claim 15, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the trajectories are generated by applying, to the route search algorithm, an operating range corresponding to each of the maneuver modes (Smolyanskiy, et al. Paragraph [0112]: "…predicting one or more positions of the one or more actors using the DNN, wherein the determining the first at least one position of one or more actors includes adjusting at least one of the one or more positions based on modeling behavior of an actor to generate the first at least one position. The second at least one position of the one or more actors may correspond to a trajectory of an actor and the method includes extending the trajectory using a classical mechanical motion algorithm to generate an extended trajectory, wherein the one or more scores correspond to the extended trajectory [extended trajectory based on machine learning algorithm - generates specific trajectory based on scoring system from first to second position].").
Regarding claim 16, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 15, and in a further embodiment, teach: The apparatus of claim 15, wherein the operating range comprises an operating range for at least one of a steering angle and a longitudinal acceleration (Smolyanskiy, et al. Paragraph [0034]: "The ACC reward function may denote the changes in acceleration [operating range for longitudinal acceleration] […] the distance to the lead car, and may be tuned to accomplish smooth following in real - world scenarios.").
Regarding claim 17, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the weights are determined based on one or more of the maneuver modes and information about the driving situation (Smolyanskiy, et al. Paragraph [0086]: "because the occupancy scores (e.g., from the confidence fields (420)) are not probabilities, to avoid over-spreading trajectories, a sharpening operation may be performed. For example, a sharpening operation may be applied to the confidence fields (420) to assign higher weights [weights are determined to refine specific maneuver position or trajectory] to higher confidence scored points before computing the weighted average." ; Smolyanskiy, et al. Paragraph [0088]: "once the trajectories have been computed—and converted from 2D image-space coordinates to 3D world-space coordinates, in embodiments—the trajectories may be used by the autonomous vehicle (1100) in performing one or more operations (e.g., obstacle avoidance, lane keeping, lane changing, path planning, mapping, etc.) [maneuver modes - applications].").
Regarding claim 18, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 17, and in a further embodiment, teach: The apparatus of claim 17, wherein the information about the driving situation comprises lane information and information about obstacles around the vehicle (Smolyanskiy, et al. Paragraph [0088]: "the trajectories may be used by the autonomous vehicle (1100) in performing one or more operations [driving situation] (e.g., obstacle avoidance [obstacle information], lane keeping, lane changing [lane information].").
Regarding claim 20, Smolyanskiy, et al. and Narayanan, et al. remain as applied to claim 11, and in a further embodiment, teach: The apparatus of claim 11, wherein the plurality of operations further comprises: obtaining a route distribution by inputting the driving dataset to the trained model; (Smolyanskiy, et al. Paragraph [0052]: "…the simulator (116) may incrementally move forward the world state (118) (e.g., by one or more time steps), providing an updated set of the simulation data (102) and/or the goal data (114) for another iteration of the training until the policy MLM (106) and/or the value function MLM (108) converge [inputting driving dataset into trained model]." ; Smolyanskiy, et al. Paragraph [0055]: "The route planner (150) may generate a planned path for the vehicle (1100) based upon various real or simulated inputs. The planned path may include waypoints (e.g., GPS waypoints), destinations, coordinates (e.g., Cartesian, polar, or other world coordinates), or other reference points [driving dataset].")
and controlling driving of the vehicle based on the route distribution (Smolyanskiy, et al. Paragraph [0057]: "The planning manager (154) may use the action MLM (180) to predict movements for the ego-vehicle and/or other actors in vicinity of the ego-vehicle, and/or a value metric for one or more ego-vehicle actions. The planning manager (154) may include […] and an optimizer (166) [using neural network data with path planning manager]." ; Smolyanskiy, et al. Paragraph [0058]: "he controller (156) may cause control of the vehicle (1100) in accordance with a select and/or optimized path from the optimizer (166) [controlling vehicle based on route distribution calculations].").
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Smolyanskiy, et al. (U.S. Patent Application Publication No. 20220138568) and Narayanan, et al. (U.S. Patent No. 12005892) in view of Pennetier, et al. (U.S. Patent No. 12602648).
Regarding claim 9, the combination of Smolyanskiy, et al. and Narayanan, et al. does not teach the method of claim 1, wherein the route search algorithm comprises a hybrid A* algorithm.
In a similar field of endeavor (multi-agent pathfinding and optimization system for vehicles), Pennetier, et al. teaches: The method of claim 1, wherein the route search algorithm comprises a hybrid A* algorithm (Col. 2, lines 61-64: "wherein the optimal path is determined using a hybrid model that uses a Reinforcement Learning (RL) agent complemented by an A* algorithm [route search algorithm comprises a hybrid A* algorithm]").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Smolyanskiy, et al. and Narayanan, et al. to include the teaching of Pennetier, et al. based on a reasonable expectation of success and motivation to improve the process of creating an integrated vehicle logistics optimization system containing multi-agent reinforcement learning with single-agent heuristics learning (Pennetier, et al. Col. 2, lines 39-46).
Regarding claim 19, the combination of Smolyanskiy, et al. and Narayanan, et al. does not teach the method of claim 11, wherein the route search algorithm comprises a hybrid A* algorithm.
In a similar field of endeavor (multi-agent pathfinding and optimization system for vehicles), Pennetier, et al. teaches: The apparatus of claim 11, wherein the route search algorithm comprises a hybrid A* algorithm (Col. 2, lines 61-64: "wherein the optimal path is determined using a hybrid model that uses a Reinforcement Learning (RL) agent complemented by an A* algorithm [route search algorithm comprises a hybrid A* algorithm]").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Smolyanskiy, et al. and Narayanan, et al. to include the teaching of Pennetier, et al. based on a reasonable expectation of success and motivation to improve the process of creating an integrated vehicle logistics optimization system containing multi-agent reinforcement learning with single-agent heuristics learning (Pennetier, et al. Col. 2, lines 39-46).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gaidon (U.S. Patent No. 11829150) teaches systems and processes with respect to the determination of driving behaviors for controlling a vehicle using a joint feature space.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TORRENCE S MARUNDA II whose telephone number is (571)272-5172. The examiner can normally be reached Monday-Friday 8:00-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANGELA Y ORTIZ can be reached at 571-272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TORRENCE S MARUNDA II/ Examiner, Art Unit 3663
/ANGELA Y ORTIZ/ Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights