DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/03/2025 has been entered.
Examiner' s Note
Examiner has cited particular paragraphs / columns and line numbers or figures in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Applicant is reminded that the Examiner is entitled to give the broadest reasonable interpretation to the language of the claims. Furthermore, the Examiner is not limited to Applicants' definition which is not specifically set forth in the disclosure.
Response to Amendments and Arguments
Applicant’s arguments, see Pgs. 7-9, filed 12/3/2025, with respect to the rejection of the claims under 35 U.S.C. 103 over Farabet in view of Colgate have been fully considered, but are not persuasive. During the interview, additional amendments were generally discussed that had the potential to overcome the current rejection of record; however, after further consideration of the submitted amendments, Farabet in view of Colgate discloses the claim limitations as discussed in the rejection below.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-6, 8-15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Farabet et al. (US 2019/0303759 A1) hereinafter Farabet in view of Colgate et al. (US 2021/0004017 A1) hereinafter Colgate.
Regarding Claim 1, Farabet teaches:
A system comprising (“a simulation system” [0051]):
A memory device (“stored in memory” [0027]); and
A processing device (“may include one or more graphics processing unit (GPU) servers. For example, the training sub-system may include a data center with GPUs, TPUs, CPUs, and/or other processor types.” [0030]), operatively coupled to the memory device (“various functions may be carried out by a processor executing instructions stored in memory” [0027]), to:
Receive, at the system, a set of input data including a roadgraph, the roadgraph representing drivable paths and objects configurations associated with segment of road driven and imaged by a first autonomous vehicle, and the drivable paths comprising an autonomous vehicle driving path of the first autonomous vehicle ("The simulated environment may include features of a driving environment such as roads, bridges, etc. "[0058] "the simulated environment may be generated using virtual data, real-world data, and or a combination thereof."[0059], “second vehicles may use the software stack(s) once the data collected from the first vehicles” [0032], examiner finds that roads and bridges are a drivable paths);
Farabet suggests modifying a roadgraph (See at least “The server(s) may receive, over the network(s) and from the vehicles, image data representative of images showing unexpected or changed road conditions, such as recently commenced road-work… such as information regarding construction sites, potholes, detours, flooding, and/or other obstructions.” [0226]), but does not explicitly disclose wherein the autonomous vehicle driving path comprises a plurality of path connection nodes disposed on the autonomous vehicle driving path; modify the roadgraph by adjusting a trajectory of the autonomous vehicle driving path, wherein adjusting the trajectory of the autonomous vehicle driving path comprises generating between a pair of path connection nodes of the plurality of path connection nodes, a synthetic path segment to address a disabled path interval of a path of the roadgraph, the disabled path interval being bounded by the pair of path connection nodes;
However, Colgate teaches:
wherein the autonomous vehicle driving path comprises a plurality of path connection nodes disposed on the autonomous vehicle driving path; (See at least “route represented as a connected graph of navigable lanes along the input routes based on HD maps” [0073], “FIGS. 8A-8B illustrate example lane elements and relations between lane elements in an HD map.” [110], and Figures 8A and 8B)
modify the roadgraph by adjusting a trajectory of the autonomous vehicle driving path, wherein adjusting the trajectory of the autonomous vehicle driving path comprises generating between a pair of path connection nodes of the plurality of path connection nodes, a synthetic path segment to address a disabled path interval of a path of the roadgraph, the disabled path interval being bounded by the pair of path connection nodes; (See at least “to simulate lane closures” [0043], “multiple situations of lane closures may need to be tested/evaluated, for example, different numbers of lanes” [0286], “Some embodiments may involve the generation of synthetic track data” [0296], “generating synthetic track data using the world geometry…and used in the change detection validation framework” [0297], “select vehicle route, place obstacles, and identify affect LaneEIs. In some embodiments, cones may be used as obstacles.” [0300] *Examiner notes that synthetic track data is generated to close a lane, i.e., a disabled path interval. As the routes comprise nodes and segments as seen in figure 8, then the synthetic track segments would be between a pair of nodes and bounded by those nodes.*).
At the time of filing, it would have been obvious to one of ordinary skill in the art to have incorporated the teachings of Colgate into the invention of Farabet with a reasonable expectation of success with the motivation of using a known technique to improve a similar device in the same way with predictable results. Representing routes as segments and nodes provides method of representing not only the various roads, but also the various connections between the roads, as discussed in at least paragraph(s) [0110]. Shifting the segments to modify the roadgraph allows testing of scenarios that may be difficult to obtain from the real world as discussed in at least paragraph(s) [0043] and [0286].
Farabet discloses placing a set of artifacts to generate a synthetic scene ("The simulation system may generate a simulated environment that may include AI objects, HIL objects, SIL objects, PIL objects and/or other object types." [0058]), but does not explicitly disclose place a set of artifacts along one or more lane boundaries of the modified roadgraph comprising the synthetic path segment to generate a synthetic scene, wherein placing the set of artifacts comprises placing candidate artifacts along one or more lane boundaries of the disabled path interval;
However, the above feature(s) are taught by Colgate (See at least “some embodiments may synthetically add cones in various context, for example, on roads with different number of lanes, different locations with different level of traffic, highways, etc. to simulate lane closures without actual lane closure occurring in the real world” [0043], “multiple situations of lane closure may need to be tested/evaluated, for example, different numbers of lanes, different positions where one or more cones are placed” [00286], “inserting static obstacles on predefined 3D positions that can be labeled as the ground truth, and used in the change detection validation framework” [0297], and “synthetic cones may be located on roads to simulate temporary traffic redirection” [0304]). At the time of filing, it would have been obvious to one of ordinary skill in the art to have incorporated the teachings of Colgate into the invention of Farabet with a reasonable expectation of success with the motivation of using a known technique to improve a similar device in the same way with predictable results. Adding artifacts along the disabled path would provide a real-world scenario for representing lane closures and allows testing of scenarios that may be difficult to obtain from the real world as discussed in at least paragraph(s) [0043] and [0286].
Farabet discloses train a machine learning model to navigate a second autonomous vehicle based on the synthetic scene (“the training sub-system may train and/or test any number of machine learning models, including deep neural networks (DNNs)” [0031] “the process may include a training loop, whereby new data is generated by the vehicle(s), used to train, test, verify, and/or validate ore or more perception DNNs, and the trained or deployed DNNs are then used by the vehicle(s) to navigate real-world environments” [0034], “second vehicles may use the software stack(s) once the data collected from the first vehicles” [0032]); and
cause a steering mechanism of the second autonomous vehicle to perform a steering maneuver (“the Ai controller may compute desired steering, acceleration, and/or braking” [0091], “a steering system, which may include a steering wheel, may be used to steer the vehicle (e.g., along a desired path or route)” [0127]).
Regarding Claim 2, Farabet teaches the system of claim 1, wherein the synthetic scene is a synthetic construction zone, and wherein the set of artifacts comprises a set of construction zone artifacts (“the updates to the map information may include updates for the HD map, such as information regarding construction sites, potholes detours, flooding, and or/other obstructions” [0226]).
Regarding Claim 3, Farabet teaches the system of claim 2, wherein the set of construction zone artifacts comprise at least one of: a cone, a road block, a road sign, or a person (“the updates to the map information may include updates for the HD map, such as information regarding construction sites, potholes, detours, flooding, and or/other obstructions” [0226] “the simulated environment may include features of the driving environment such as street signs, etc.” [0058] “that simulates a virtual world or environment that may include artificial intelligence vehicle or other objects (e.g., pedestrians)” [0052]).
Regarding Claim 4, Farabet does not disclose wherein adjusting the trajectory of the autonomous vehicle driving path further comprises shifting path segments of the autonomous vehicle driving path between pairs of respective path connection nodes of the plurality of path connection nodes, wherein each of the path segments is shifted by a random distance.
However, the above features would be obvious in view of Colgate. Colgate teaches shifting segments of path, as discussed in the rejection of claim 1 above. Determining how much or by what standard to shift each segment would be part of routine design and testing of such a system and would be within the skill of one in the art.
Regarding Claim 5, Farabet teaches the system of claim 1, wherein the drivable paths of the roadgraph further comprises at least one of: path center location, path heading, distance to left/right boundaries, speed limit, or an indication of drivability (“DNNs may be used for detecting lanes and boundaries on driving surfaces, for detecting drivable dree-space, for detecting traffic poles or signs, for detecting objects in the environment” [0031]).
Regarding Claim 6, Farabet teaches the system of claim 1, wherein, to place the set of artifacts along one or more lane boundaries of the modified roadgraph, the processing device is to:
Place a plurality of candidate artifacts along the one or more lane boundaries of the modified roadgraph ("The simulation system may generate a simulated environment that may include AI objects, HIL objects, SIL objects, PIL objects and/or other object types." [0058]);
Farabet does not teach:
Remove, duplicate candidate artifacts of the plurality of candidate artifacts to obtain a first set of candidate artifacts; and
Remove, from the first set of candidate artifacts, candidate artifacts that interfere with drivable paths of the modified roadgraph.
However, Colgate does teach:
Remove, duplicate candidate artifacts of the plurality of candidate artifacts to obtain a first set of candidate artifacts (See at least “the user may edit the map data by adding/removing objects…” [0288], “in order to determine whether objects perceived by the vehicle sensors and optionally by the perception module represent the same object or different objects” [0132]); and
Remove, from the first set of candidate artifacts, candidate artifacts that interfere with drivable paths of the modified roadgraph (See at least “similarly the user may add/remove/move traffic cones, construction signs, barriers, etc.” [0288] *the examiner interprets this as being able to move barriers means removing an object that is blocking a drivable path.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified Farabet to incorporate the teachings of Colgate to include removing artifacts. Doing so would also map data to remain accurate and produce more realistic synthetic scenes to improve autonomous vehicle training, as recognized by Colgate in at least the background and summary sections.
Regarding Claim 8, Farabet teaches the system of claim 1, wherein the processing device is further to:
Generate a set of training input data comprising a set of data frames from the synthetic scene (“the GPU platform may receive data about the simulated environment and may create sensor inputs” [0104]); and
Obtain a set of target output data for the set of training input data, wherein the machine learning model is trained using the set of training input data and the set of target output data (See at least “the sensor data and/or encoded data may be used as inputs to one or more DNNs of the autonomous driving software stack, and the outputs of the one or more DNNs may be used for updating a state of the virtual vehicle within the simulate environment” [0070], “controlling a virtual object within a simulated environment based at least in part on the output” [0123]).
Regarding Claim 9, Farabet teaches the system of claim 8, wherein the set of target output data comprises one or more ground-truth annotations related to the synthetic scene for the set of data frames in the set of training (“the KPI API may receive CAN data, ground truth, and/or virtual object state information (e.g., from the software stack(s) 1160 from the simulator component(s) 402 and may generate and/or provide a report (in real-time) that includes KPI’s and/or commands to save state, restore state, and/or apply changes” [0102])
Regarding Claim 10, Farabet teaches:
A method comprising (“systems and methods disclosed are related to training, testing, and verifying autonomous machines or objects in simulated environments” [0025]):
Receiving, by a processing device of a system for generating and utilizing synthetic scenes (“may include one or more graphics processing unit (GPU) servers. For example, the training sub-system may include a data center with GPUs, TPUs, CPUs, and/or other processor types.” [0030]), a set of input data including a roadgraph, the roadgraph representing drivable paths and objects configurations associated with a segment of road driven and imaged by a first autonomous vehicle, and drivable paths comprising an autonomous vehicle driving path of the first autonomous vehicle ("The simulated environment may include features of a driving environment such as roads, bridges, etc. "[0058] "the simulated environment may be generated using virtual data, real-world data, and or a combination thereof."[0059], “second vehicles may use the software stack(s) once the data collected from the first vehicles” [0032]); examiner finds that roads and bridges are considered drivable paths);
Farabet suggests modifying a roadgraph (See at least “The server(s) may receive, over the network(s) and from the vehicles, image data representative of images showing unexpected or changed road conditions, such as recently commenced road-work… such as information regarding construction sites, potholes, detours, flooding, and/or other obstructions.” [0226]), but does not explicitly disclose wherein the autonomous vehicle driving path comprises a plurality of path connection nodes disposed on the autonomous vehicle driving path; modifying, by the processing device, the roadgraph by adjusting a trajectory of the autonomous vehicle driving path, wherein adjusting the trajectory of the autonomous vehicle driving path comprises generating between a pair of path connection nodes of the plurality of path connection nodes, a synthetic path segment to address a disabled path interval of a path of the roadgraph, the disabled path interval being bounded by the pair of path connection nodes;
However, Colgate teaches:
wherein the autonomous vehicle driving path comprises a plurality of path connection nodes disposed on the autonomous vehicle driving path; (See at least “route represented as a connected graph of navigable lanes along the input routes based on HD maps” [0073], “FIGS. 8A-8B illustrate example lane elements and relations between lane elements in an HD map.” [110], and Figures 8A and 8B)
modify the roadgraph by adjusting a trajectory of the autonomous vehicle driving path, wherein adjusting the trajectory of the autonomous vehicle driving path comprises generating between a pair of path connection nodes of the plurality of path connection nodes, a synthetic path segment to address a disabled path interval of a path of the roadgraph, the disabled path interval being bounded by the pair of path connection nodes; (See at least “to simulate lane closures” [0043], “multiple situations of lane closures may need to be tested/evaluated, for example, different numbers of lanes” [0286], “Some embodiments may involve the generation of synthetic track data” [0296], “generating synthetic track data using the world geometry…and used in the change detection validation framework” [0297], “select vehicle route, place obstacles, and identify affect LaneEIs. In some embodiments, cones may be used as obstacles.” [0300] *Examiner notes that synthetic track data is generated to close a lane, i.e., a disabled path interval. As the routes comprise nodes and segments as seen in figure 8, then the synthetic track segments would be between a pair of nodes and bounded by those nodes.*).
At the time of filing, it would have been obvious to one of ordinary skill in the art to have incorporated the teachings of Colgate into the invention of Farabet with a reasonable expectation of success with the motivation of using a known technique to improve a similar device in the same way with predictable results. Representing routes as segments and nodes provides method of representing not only the various roads, but also the various connections between the roads, as discussed in at least paragraph(s) [0110]. Shifting the segments to modify the roadgraph allows testing of scenarios that may be difficult to obtain from the real world as discussed in at least paragraph(s) [0043] and [0286].
Farabet discloses placing a set of artifacts to generate a synthetic scene ("The simulation system may generate a simulated environment that may include AI objects, HIL objects, SIL objects, PIL objects and/or other object types." [0058]), but does not explicitly disclose placing, by the processing device, a set of artifacts along one or more lane boundaries of the modified roadgraph comprising the synthetic path segment to generate a synthetic scene, wherein placing the set of artifacts comprises placing candidate artifacts along one or more lane boundaries of the disabled path interval;
However, the above feature(s) are taught by Colgate (See at least “some embodiments may synthetically add cones in various context, for example, on roads with different number of lanes, different locations with different level of traffic, highways, etc. to simulate lane closures without actual lane closure occurring in the real world” [0043], “multiple situations of lane closure may need to be tested/evaluated, for example, different numbers of lanes, different positions where one or more cones are placed” [00286], “inserting static obstacles on predefined 3D positions that can be labeled as the ground truth, and used in the change detection validation framework” [0297], and “synthetic cones may be located on roads to simulate temporary traffic redirection” [0304]). At the time of filing, it would have been obvious to one of ordinary skill in the art to have incorporated the teachings of Colgate into the invention of Farabet with a reasonable expectation of success with the motivation of using a known technique to improve a similar device in the same way with predictable results. Adding artifacts along the disabled path would provide a real-world scenario for representing lane closures and allows testing of scenarios that may be difficult to obtain from the real world as discussed in at least paragraph(s) [0043] and [0286]
Farabet discloses training, by the processing device, a machine learning model to navigate a second autonomous vehicle based on the synthetic scene (“the training sub-system may train and/or test any number of machine learning models, including deep neural networks (DNNs)” [0031] “the process may include a training loop, whereby new data is generated by the vehicle(s), used to train, test, verify, and/or validate ore or more perception DNNs, and the trained or deployed DNNs are then used by the vehicle(s) to navigate real-world environments” [0034], “second vehicles may use the software stack(s) once the data collected from the first vehicles” [0032]); and
causing a steering mechanism of the second autonomous vehicle to perform a steering maneuver (“the Ai controller may compute desired steering, acceleration, and/or braking” [0091], “a steering system, which may include a steering wheel, may be used to steer the vehicle (e.g., along a desired path or route)” [0127]).
Regarding Claim 11, Farabet teaches the method of claim 10, wherein the synthetic scene is a synthetic construction zone, and wherein the set of artifacts comprises a set of construction zone artifacts. (“the updates to the map information may include updates for the HD map, such as information regarding construction sites, potholes detours, flooding, and or/other obstructions” [0226]).
Regarding Claim 12, Farabet teaches the method of claim 11, wherein the set of construction zone artifacts comprises at least one of: a cone, a road block, a road sign, or a person (“the updates to the map information may include updates for the HD map, such as information regarding construction sites, potholes, detours, flooding, and or/other obstructions” [0226] “the simulated environment may include features of the driving environment such as street signs, etc.” [0058] “that simulates a virtual world or environment that may include artificial intelligence vehicle or other objects (e.g., pedestrians)” [0052]).
Regarding Claim 13, Farabet does not disclose wherein modifying the roadgraph further comprises merging a second segment of the autonomous vehicle driving path between two of the plurality of path connection nodes with a segment of one of the drivable paths.
However, the above features would be obvious in view of Colgate. Colgate teaches shifting segments of path, as discussed in the rejection of claim 10 above. Determining how much or by what standard to shift each segment would be part of routine design and testing of such a system and would be within the skill of one in the art. For example, a lane closure would involve merging a lane, represented by the segments and nodes, into another lane, represented by different segments and nodes.
Regarding Claim 14, Farabet teaches the method of claim 10 wherein the drivable paths of the roadgraph further comprises at least one of: path center location, path heading, distance to left/right boundaries, speed limit, or an indication of drivability (“DNNs may be used for detecting lanes and boundaries on driving surfaces, for detecting drivable dree-space, for detecting traffic poles or signs, for detecting objects in the environment” [0031]).
Regarding Claim 15, Farabet teaches the method of claim 10, wherein placing artifacts comprises:
Placing a plurality of candidate artifacts along the one or more lane boundaries of the modified roadgraph ("The simulation system may generate a simulated environment that may include AI objects, HIL objects, SIL objects, PIL objects and/or other object types." [0058]);
Farabet does not teach:
Removing, duplicate candidate artifacts of the plurality of candidate artifacts to obtain a first set of candidate artifacts; and
Removing, from the first set of candidate artifacts, candidate artifacts that interfere with drivable paths of the modified roadgraph.
However, Colgate does teach:
Removing, duplicate candidate artifacts of the plurality of candidate artifacts to obtain a first set of candidate artifacts (See at least “the user may edit the map data by adding/removing objects…” [0288], “in order to determine whether objects perceived by the vehicle sensors and optionally by the perception module represent the same object or different objects” [0132]); and
Removing, from the first set of candidate artifacts, candidate artifacts that interfere with drivable paths of the modified roadgraph (See at least “similarly the user may add/remove/move traffic cones, construction signs, barriers, etc.” [0288] *the examiner interprets this as being able to move barriers means removing an object that is blocking a drivable path.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified Farabet to incorporate the teachings of Colgate to include removing artifacts. Doing so would also map data to remain accurate and produce more realistic synthetic scenes to improve autonomous vehicle training, as recognized by Colgate in at least the background and summary sections.
Regarding Claim 17, Farabet teaches the method of claim 10, further comprising:
Generating, by the processing device, a set of training input data comprising a set of data frames from the synthetic scene (“the GPU platform may receive data about the simulated environment and may create sensor inputs” [0104]); and
Obtaining, by the processing device, a set of target output data for the set of training input data, wherein the machine learning model is trained using the set of training input data and the set of target output data (See at least “the sensor data and/or encoded data may be used as inputs to one or more DNNs of the autonomous driving software stack, and the outputs of the one or more DNNs may be used for updating a state of the virtual vehicle within the simulate environment” [0070], “controlling a virtual object within a simulated environment based at least in part on the output” [0123]).
Regarding Claim 18, Farabet teaches the method of claim 17, wherein the set of target output data comprises one or more ground-truth annotations related to the synthetic scene for the set of data frames in the set of training data (“the KPI API may receive CAN data, ground truth, and/or virtual object state information (e.g., from the software stack(s) 1160 from the simulator component(s) 402 and may generate and/or provide a report (in real-time) that includes KPI’s and/or commands to save state, restore state, and/or apply changes” [0102]).
Regarding Claim 19, Farabet teaches:
A method comprising:
Obtaining a trained machine learning model used to navigate a first autonomous vehicle, wherein the trained machine learning model is trained at a system for generating and utilizing synthetic scenes (“the training sub-system may train and/or test any number of machine learning models, including deep neural networks (DNNs)” [0031] “the process may include a training loop, whereby new data is generated by the vehicle(s), used to train, test, verify, and/or validate ore or more perception DNNs, and the trained or deployed DNNs are then used by the vehicle(s) to navigate real-world environments” [0034], and is trained based on a synthetic scene comprising a modified roadgraph modified from an original roadgraph representing drivable paths and object configurations associated with a segment of road driven and imaged by a second autonomous vehicle, the drivable paths comprising an autonomous vehicle driving path of the second autonomous vehicle ("The simulated environment may include features of a driving environment such as roads, bridges, etc. "[0058] "the simulated environment may be generated using virtual data, real-world data, and or a combination thereof."[0059], “second vehicles may use the software stack(s) once the data collected from the first vehicles” [0032]); examiner finds that roads and bridges are considered drivable paths)
Receiving detection results including a set of artifacts within a scene while the autonomous vehicle is proceeding along a driving path (For example, DNNs may be used for objection detection… control generation during vehicle maneuvers)” [0003];
Causing a modification of the driving path using the trained machine learning model in view of the detection results (“DNN is used for obstacle avoidance” [0004]); and
Causing a steering mechanism of the first autonomous vehicle to perform a steering maneuver based on the modified driving path (“the Ai controller may compute desired steering, acceleration, and/or braking” [0091], “a steering system, which may include a steering wheel, may be used to steer the vehicle (e.g., along a desired path or route)” [0127]).
Farabet suggests modifying a roadgraph (See at least “The server(s) may receive, over the network(s) and from the vehicles, image data representative of images showing unexpected or changed road conditions, such as recently commenced road-work… such as information regarding construction sites, potholes, detours, flooding, and/or other obstructions.” [0226]), but does not explicitly disclose the drivable paths comprising an autonomous vehicle driving path of the second autonomous vehicle that comprises a plurality of path connection nodes disposed on the autonomous driving path, the modified roadgraph having a modified autonomous vehicle driving path that comprises a synthetic path segment generated between a pair of path connection nodes of the plurality of path connection nodes to address a disabled path interval bounded by the pair or path connection nodes of a path of the roadgraph, and a set of synthetic artifacts along at least one lane boundary within the modified roadgraph, wherein the set of synthetic artifacts comprises candidate artifacts placed along one or more lane boundaries of the disables path interval;
However, Colgate teaches the drivable paths comprising an autonomous vehicle driving path of the second autonomous vehicle that comprises a plurality of path connection nodes disposed on the autonomous driving path, (See at least “route represented as a connected graph of navigable lanes along the input routes based on HD maps” [0073], “FIGS. 8A-8B illustrate example lane elements and relations between lane elements in an HD map.” [110], and Figures 8A and 8B)
the modified roadgraph having a modified autonomous vehicle driving path that comprises a synthetic path segment generated between a pair of path connection nodes of the plurality of path connection nodes to address a disabled path interval bounded by the pair or path connection nodes of a path of the roadgraph, and a set of synthetic artifacts along at least one lane boundary within the modified roadgraph, wherein the set of synthetic artifacts comprises candidate artifacts placed along one or more lane boundaries of the disables path interval; (See at least “ may synthetically add cones in various contexts, for example, on roads with different number of lanes, different locations with different level of traffic, highways, etc. to simulate lane closures” [0043], “multiple situations of lane closures may need to be tested/evaluated, for example, different numbers of lanes” [0286], “Some embodiments may involve the generation of synthetic track data” [0296], “generating synthetic track data using the world geometry, computed from real measurements, and inserting static obstacles…and used in the change detection validation framework” [0297], “select vehicle route, place obstacles, and identify affect LaneEIs. In some embodiments, cones may be used as obstacles.” [0300], and “synthetic cones may be located on roads to simulate temporary traffic redirection” [0304] *Examiner notes that synthetic track data is generated to close a lane, i.e., a disabled path interval. As the routes comprise nodes and segments as seen in figure 8, then the synthetic track segments would be between a pair of nodes and bounded by those nodes.*).
At the time of filing, it would have been obvious to one of ordinary skill in the art to have incorporated the teachings of Colgate into the invention of Farabet with a reasonable expectation of success with the motivation of using a known technique to improve a similar device in the same way with predictable results. Representing routes as segments and nodes provides method of representing not only the various roads, but also the various connections between the roads, as discussed in at least paragraph(s) [0110]. Shifting the segments to modify the roadgraph allows testing of scenarios that may be difficult to obtain from the real world as discussed in at least paragraph(s) [0043] and [0286]
Regarding Claim 20, Farabet teaches the method of claim 19, wherein the scene is a construction zone, (“the updates to the map information may include updates for the HD map, such as information regarding construction sites, potholes detours, flooding, and or/other obstructions” [0226])
Farabet does not teach and wherein the set of artifacts comprise a cone.
However, Colgate does teach and wherein the set of artifacts comprise a cone (“The system may provide a library of synthetic objects that can be added to the map, for example, various traffic signs, cones, lane lines, etc.” [0289].
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified Farabet to incorporate the teachings of Colgate to include the artifact being a cone. Doing so would also map data to remain accurate and produce more realistic synthetic scenes to improve autonomous vehicle training especially in construction zones, as recognized by Colgate in at least the background and summary sections.
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Farabet (US 2019/0303759 A1) in view of Colgate (US 2021/0004017 A1) and Towal et al. (US 2019/0384304 A1), hereinafter Towal
Regarding Claim 7 the combination of Farabet and Colgate teaches the system of claim 6. The combination does not teach wherein the duplicate candidate artifacts are removed by applying non-maximum suppression.
However, Towal does teach wherein the duplicate candidate artifacts are removed by applying non-maximum suppression. (“non-maximum suppression may be used where two or more path geometries have associated confidence values that indicate the path geometries may correspond to the same path type. In such examples, the confidence value that is the highest for the particular path type may be used to determine the path geometry for the path type and non-maximum suppression may be used to remove, or suppress, the other geometries.” [0046] * Examiner interprets that an object/artifact is anything in the synthetic scene that can be moved including a path way*)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified Farabet and Colgate to incorporate the teachings of Towal to include the use of non-maximum suppression. Doing so would create more accurate pathways for driving, as recognized by Towal in at least paragraph 46.
Regarding Claim 16, the combination of Farabet and Colgate teaches the method of claim 15.
The combination does not teach wherein removing the duplicate candidate synthetic objects comprises applying non-maximum suppression.
However, Towal does teach wherein removing the duplicate candidate artifacts comprises applying non-maximum suppression (“non-maximum suppression may be used where two or more path geometries have associated confidence values that indicate the path geometries may correspond to the same path type. In such examples, the confidence value that is the highest for the particular path type may be used to determine the path geometry for the path type and non-maximum suppression may be used to remove, or suppress, the other geometries.” [0046] * Examiner interprets that an object/artifact is anything in the synthetic scene that can be moved including a path way*).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified Farabet and Colgate to incorporate the teachings of Towal to include the use of non-maximum suppression. Doing so would create more accurate pathways for driving, as recognized by Towal in at least paragraph 46.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892. The prior art shows the state of the art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID P MERLINO whose telephone number is (571)272-8362. The examiner can normally be reached M-Th 5:30am-3:00pm F 5:30-9:00 am ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Bishop can be reached on 571-270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/David P. Merlino/Primary Examiner, Art Unit 3665