DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is the first office action on the merits, claims 1-20 are currently pending and addressed below.
Response to Arguments/Amendments
The amendment filed January 6th, 2026 has been entered. Claims 1-20 are currently pending in the Application.
Applicant’s arguments with respect to the rejection of claims under 35 U.S.C 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, and 5-20, are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 20220146277, to Lambert et al. (hereinafter Lambert), and further in view of U.S. Patent Publication No. 20220315000, to Wray et al (hereinafter Wray).
Regarding claim 1, and commensurate claims 10, and 14, Lambert teaches A computing system for simulating change detection data, the computing system comprising: a processor; (See at least paragraph [0021] ““map change detection””). and memory communicably coupled to the processor that includes instructions that, when executed by the processor, cause the processor to: (See at least paragraph [0013] “when inputting the map data from the HD map and the sensor data captured by the perception system into a neural network to identify differences, the system may generate a score that represents a probability of a change to a features in the map data.”). Further, (See at least paragraph [0022] “The perception system may include one or more processors, and computer-readable memory with programming instructions ”). Further, (See at least paragraph [0029] “The system may receive the HD map from a map generation system via a communication link, or the HD map may have previously been downloaded and stored in the AV's onboard memory prior to step 301”). Further, (See at least paragraph [0026] “The vehicle's on-board computing system 101 will be in communication with a remote server 106”).
generate modified map features based upon the HD map features; and (See at least paragraph [0043] “an automated system or human operator may generate simulated training data, comprising simulated images or modified actual images in which certain features (such as traffic signals and traffic control signs) have been added, removed or changed.”).
train a machine learning model based upon the HD map features and the modified map features, (See at least paragraph [0010] “Before inputting the sensor data captured by the perception system into the neural network, train the neural network on a set of simulated sensor data in which one or more annotated features of the area have been altered to not match corresponding features in the HD map data.”). wherein the machine learning model is configured to detect a change between data from an HD map corresponding to an environment and sensor data generated by a vehicle in the environment. (See at least paragraph [0006] “A processor will input the sensor data along with HD data for the area into a neural network (such as a convolutional neural network) to determine distances between features in the map data and corresponding features in the sensor data. For example, the network may compare differences between embeddings for each data set, or the network may directly generate scores for different categories corresponding to map-sensor agreement or disagreement. Either way, the system may convert the sensor data into a birds-eye view and/or ego view before doing this. The system will identify any distances or scores that exceed a threshold.”). Further, (See at least paragraph [0007] “The system will input the map data from the HD map and the sensor data captured by the perception system into a neural network to generate an embedding that provides differences between features in the map data and corresponding features in the sensor data. The system will identify any differences that exceed a threshold. The system will report the features for which the differences exceed the threshold as features of the HD map that require updating.”).
Lambert fails to explicitly disclose, However, Wary discloses, convert features from a standard-definition (SD) map into high-definition (HD) map features into generate a pseudo-HD map without using an existing HD map geographically corresponding to the SD map; (See at least paragraph [0041] “routes can be planned using standard definition map data (or simply, an SD map) and roads of an SD map can be mapped to obtain HD map information.”). Further, (See at least paragraph [0044] “A road or an area may be said to be unmapped. An HD map does not include information of unmapped roads and areas.”). Still further, (See at least paragraph [0046] “sensors data from one or more vehicles can be used to supplement an SD map with information (e.g., HD information) so that the SD map can be used for lane-level route planning as described herein for autonomous driving. As such, using sensor data to supplement the HD map enables a lane-level route planner to obtain routes using the SD map or a combination of an HD map and an SD map. The HD map can be used for mapped roads and lanes and the SD map can be used for roads and lanes that are unmapped in the HD map”). Still further, (See at least paragraph [0047] “an AV, or a fleet of AVs, can generate their own HD maps as roads of SD maps are traversed.”). Lastly, (See at least paragraph [0214] “The HD information can be used to construct an HD map or to augment an HD map of the AV. Via the exploration, new road and lane-segment information can be added to the navigation map. At least a partial HD map can be constructed via the exploration”)
Lambert as modified by Wary, are analogous art because they are in the same field of endeavor, mapping systems. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Lambert to incorporate the teachings of Wary because geographic scalability. Wray teachings to synthetically map available SD areas where HD maps do not exist will aid in Lambert teachings by rasterize the synthetic maps by adding/removing lanes.
Regarding claim 5, Lambert as modified by Wray disclose the claimed features of claim 1 and Lambert further discloses, wherein generate the modified map features based upon the HD map features comprises: select an area of the pseudo-HD map that includes an HD map feature; (See at least paragraph [0045] “An AV may access an HD map, rendered as rasterized images. Entities may be labeled (i.e., assigned classes) from the back of the raster to the front in an order such as: driveable area; lane segment polygons, lane boundaries; and pedestrian crossings (crosswalks)..”)
generate a rasterized image of the area, wherein the area includes the HD map feature; (See at least paragraph [0033] “the system may first convert the sensor data into a birds-eye view of the area (step 304), an ego-view of the area (step 305), or both. Then, at 306, the birds-eye view or ego-view of the sensor data may be stacked with the HD map data as input to an early-data-fusion model, or the two data streams (sensor data and HD map data) may be fed individually into separate networks in a late-data-fusion model, in which case high dimensional features would instead be concatenated for subsequent classification or regression.”)
perform a modification to the rasterized image; (See at least paragraph [0043] “an automated system or human operator may generate simulated training data, comprising simulated images or modified actual images in which certain features (such as traffic signals and traffic control signs) have been added, removed or changed.”)
and generate a modified rasterized image based upon the rasterized image with the modification. (See at least paragraph [0010] “train the neural network on a set of simulated sensor data in which one or more annotated features of the area have been altered to not match corresponding features in the HD map data.”)
Regarding claim 6, Lambert as modified by Wray disclose the claimed features of claim 5 and Lambert further discloses, wherein the modification is one of:remove the HD map feature from the rasterized image;add a second HD map feature to the rasterized image;change a location of the HD map feature within the rasterized image; or change a color of the HD map feature within the rasterized image. (See at least paragraph [0010] “train the neural network on a set of simulated sensor data in which one or more annotated features of the area have been altered to not match corresponding features in the HD map data.”)
Regarding claim 7, Lambert as modified by Wray disclose the claimed features of claim 5 and Lambert further discloses, wherein the rasterized image and the modified rasterized image are stored as a pair along with an indication as to whether the pair is a positive change example or a negative change example. (See at least paragraph [0013] “the system may generate a score that represents a probability of a change to a features in the map data. The system may identify any scores that exceed a scoring threshold, and it may report the features for which the scores exceed the scoring threshold as features of the HD map that require updating. Alternatively or in addition, the system may generate an embedding for each data set, and it may compare the embeddings to yield distances between the features in the map data and the corresponding features in the sensor data.”). Further, (See at least paragraph [0010] “train the neural network on a set of simulated sensor data in which one or more annotated features of the area have been altered to not match corresponding features in the HD map data.”)
Regarding claim 8, Lambert as modified by Wray disclose the claimed features of claim 1 and Lambert further discloses, wherein the instructions further cause the processor to:locate an area in the HD map based upon the sensor data; (See at least paragraph [0007] “The system will access an HD map of an area in which the vehicle is present.”)
add a feature to the HD map based upon the change detected by the machine learning model; (See at least paragraph [0006] “The system will report the features for which the distances or scores that exceed the threshold, subject to any applied filters, as features of the HD map that require updating to a map generation system for updating the HD map, and/or to another system.”)
and add an annotation to the feature in the HD map that is indicative of a type of the feature. (See at least paragraph [0035] “at 311 the system may first filter some of the threshold-exceeding features to report and update only those features that relate to a particular feature class (such as lane geometry and pedestrian crosswalks) as classified in at least the birds-eye view, or only those features for which threshold-exceeding distances are detected at least a minimum number of times within a time horizon or within a number of vehicle runs.”).
Regarding claim 9, Lambert as modified by Wray disclose the claimed features of claim 1 and Lambert further discloses, wherein the instructions further cause the processor to:control vehicles based upon an updated HD map that is generated based upon the change detected by the machine learning model. (See at least paragraph [0048] “control operations of various vehicle components to move the vehicle along the trajectory. For example, the on-board computing device 520 may control braking via a brake controller 522;”)
Regarding claim 11, Lambert as modified by Wray disclose the claimed features of claim 10 and Lambert further discloses, wherein the instructions further cause the processor to: obtain a first rasterized image based upon the sensor data, wherein the first rasterized image is indicative of the environment of the vehicle; determine a location of the vehicle within the HD map based upon the sensor data; generate a second rasterized image based upon the location of the vehicle within the HD map;provide the first rasterized image and the second rasterized image as input to the machine learning model; (See at least paragraph [0008] “the sensor data captured by the perception system into the neural network, the system may convert the sensor data into a birds-eye-view of the area and when inputting the sensor data into the neural network it may input the birds-eye view. To convert the sensor data into a birds-eye view, the system may accumulate multiple frames of sensor data that is LiDAR data, generate a local ground surface mesh of the area, and trace rays from the LiDAR data to the local ground surface mesh.”). Further, (See at least paragraph [] “At 409 the system will form the bird's-eye view image by projecting the 3D points and their color values onto a 2D grid. Finally, at 410 the system may feed the birds-eye view into the network that was pre-trained at 400.”). Further, (See at least paragraph [0033] “the birds-eye view or ego-view of the sensor data may be stacked with the HD map data as input to an early-data-fusion model, or the two data streams (sensor data and HD map data) may be fed individually into separate networks in a late-data-fusion model, in which case high dimensional features would instead be concatenated for subsequent classification or regression..”).
and determine whether the HD map reflects a current state of the environment based upon an output of the machine learning model. (See at least paragraph [0013] “generate a score that represents a probability of a change to a features in the map data. The system may identify any scores that exceed a scoring threshold, and it may report the features for which the scores exceed the scoring threshold as features of the HD map that require updating.”).
Regarding claim 12, Lambert as modified by Wray disclose the claimed features of claim 11 and Lambert further discloses, wherein the instructions further cause the processor to:modify the HD map based upon the output of the machine learning model. (See at least paragraph [0006] “The system will report the features for which the distances or scores that exceed the threshold, subject to any applied filters, as features of the HD map that require updating to a map generation system for updating the HD map, and/or to another system.”)
Regarding claim 13, Lambert as modified by Wray disclose the claimed features of claim 11 and Lambert further discloses, wherein the first rasterized image and the second rasterized image are birds-eye-view images. (See at least paragraph [Abstract] “convert the sensor data into a birds-eye view ”).
Regarding claim 15, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, wherein training the machine learning model comprises training a binary classifier that outputs an indication as to whether the change has occurred in the environment based upon a first rasterized image and a second rasterized image, wherein the first rasterized image is generated based upon the sensor data, and wherein the second rasterized image is generated based upon the data from the HD map. (See at least paragraph [0033] “the birds-eye view or ego-view of the sensor data may be stacked with the HD map data as input to an early-data-fusion model, or the two data streams (sensor data and HD map data) may be fed individually into separate networks in a late-data-fusion model, in which case high dimensional features would instead be concatenated for subsequent classification or regression.”).Further, (See at least paragraph [] “system may generate a score that represents a probability of a change to a features in the map data. The system may identify any scores that exceed a scoring threshold, and it may report the features for which the scores exceed the scoring threshold as features of the HD map that require updating.”). Further, (See at least paragraph [0043] “The training data set may be, for example, in the form of triplets {x, x*, y} in which x is a local region of the map, x* is an online sensor sweep, and y is a binary label indicating whether a significant map change occurred (i.e., whether at least a threshold difference in distance between map and sensor data is present). In this example, {x, x*} should correspond to the same geographic location.”)
Regarding claim 16, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, wherein the change is selected from a group including:an addition of a road marking to the environment; a removal of the road marking from the environment; and the road marking moving from a first location to a second location in the environment. (See at least paragraph [0028] “ a road construction sign such as traffic control sign 222 may be installed or removed depending on when road crews are working in a particular location. ”).
Regarding claim 17, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, further comprising: locating a feature in the HD map based upon the sensor data; and removing the feature from the HD map based upon change detected by the machine learning model. (See at least paragraph [0043] “ modified actual images in which certain features (such as traffic signals and traffic control signs) have been added, removed or changed. ”).
Regarding claim 18, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, wherein training the machine learning model includes training a convolutional neural network (CNN) to detect the change. (See at least paragraph [0006] “A processor will input the sensor data along with HD data for the area into a neural network (such as a convolutional neural network). ”).
Regarding claim 19, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, generating modified map features based upon the HD map features comprises: (See at least paragraph [0043] “an automated system or human operator may generate simulated training data, comprising simulated images or modified actual images”).
selecting an area of the pseudo-HD map that includes an HD map feature; (See at least paragraph [0043] “ The training data set may be, for example, in the form of triplets {x, x*, y} in which x is a local region of the map, x* is an online sensor sweep”).
generating a rasterized image of the area, wherein the area includes the HD map feature; (See at least paragraph [0045] “ An AV may access an HD map, rendered as rasterized images. Entities may be labeled (i.e., assigned classes) from the back of the raster to the front in an order such as: driveable area; lane segment polygons, lane boundaries; and pedestrian crossings (crosswalks)”).
performing a modification to the rasterized image; and (See at least paragraph [0043] “an automated system or human operator may generate simulated training data, comprising simulated images or modified actual images in which certain features (such as traffic signals and traffic control signs) have been added, removed or changed”).
generating a modified rasterized image based upon the rasterized image with the modification (See at least paragraph [0043] “modified actual images in which certain features (such as traffic signals and traffic control signs) have been added, removed or changed. Annotations (labels) will be included for each of these features in the training data”).
Regarding claim 20, Lambert as modified by Wray disclose the claimed features of claim 14 and Lambert further discloses, wherein the HD map features include lane markings. (See at least paragraph [0027] “FIG. 2 illustrates an example illustration of HD map data for an intersection in which a first road 201 intersects with a second road 202. The geometry of each lane within each street may be represented as polygons such as lane polygons 203 and 204. Crosswalks 208a-208d and other road markings (such as double centerlines 213) may be represented as polylines or pairs of parallel polylines, while stop lines such as 209a-209b may be represented as polygons.”).
Claims 2-4, is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 20220146277, to Lambert et al. (hereinafter Lambert), and further in view of U.S. Patent Publication No. 20220315000, to Wray et al (hereinafter Wray), and further in view of U.S. Patent No. 10969232 B1, to Johnson et al (hereinafter Johnson).
Regarding claim 2, Lambert as modified by Wray disclose the claimed features of claim 1 and further disclose, Lambert fails to explicitly disclose, However, Johnson discloses, wherein the features from the SD map comprise a graph that includes nodes and edges connecting the nodes, wherein the edges represent roads and the nodes represent junctions connecting the roads. (See [Column 4, Lines 58-60] “an HD segment connectivity graph (a graph being comprised of nodes and edges), provide input to a block 130”). Further, (See [Column 3, Lines 25-26] “generating a graph, comprising nodes and edges, from a high definition (HD) map,”). Further, (See [Column 11, Lines 41-42] “In a graph, an edge will connect two adjacent nodes.”). Further, (See [Column 2, Lines 1-2] “this sequence may be a relatively sparse set of points, defined by junctions.”). Further, (See [Column 4, Lines 56-63] “wherein the features from the SD map comprise a graph that includes nodes and edges connecting the nodes, wherein the edges represent roads and the nodes represent junctions connecting the roads.”). Further, (See [Column 11, Lines 40-46] “FIG. 3 depicts a highway interchange with a number of nodes 310-1 to 310-n marked thereon. In a graph, an edge will connect two adjacent nodes. In developing a route, a route segment may comprise one or more edges. Depending on the severity of curves, bends, or other changes in the route portion, more nodes may be provided, meaning more edges, and possibly more segments.”). Further, (See [Column 12, Lines 7-15] “FIG. 6 depicts a route in which red lines 610 signify an HD graph network. Pins 620-0 to 620-4 signify SD waypoints. Pins 630-1 to 630-n signify HD nodes. Green line 640 is a generated path using the various nodes and waypoints. It should be noted that there are several red lines in the HD graph network 610 that do not follow or track the green line 640. The waypoints 620-0 to 620-4 which help to define the route do not match those red lines, making those red lines extraneous to the route.”)
Lambert as modified by Johnson, are analogous art because they are in the same field of endeavor, mapping systems. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Lambert as modified by Wray to incorporate the teachings of Johnson because the conversion of Johnson will maintain a one-to-one relationship between their SD road segments and their HD road segments.
Regarding claim 3, Lambert as modified by Wray, and Johnson disclose the claimed features of claim 2 and further disclose, Lambert fails to explicitly disclose, However, Johnson discloses, wherein the edges are assigned criteria that is indicative of attributes of the roads. (See [Column 1, Lines 38-42] “Information to define such things as lane size and location on a road; lane type, such as regular, shoulder, passing, and/or express lanes, as well as HOV lanes and/or bike lanes. There may be information about speeds within a lane; striping or solid lines; and/or line color.”).Further, (See [Column 4, Lines 11-19] “the technique described herein applies an incremental spatial join between SD lines and their HD counterparts. In creating this spatial join, a first task is to build a common spatial network with which line segments can be matched based on road type (motorway, ramp etc.), stacking/Z order (e.g. intersections), and proximity. Aspects of the inventive technique enable grouping of line segments and join based on this network and report out unmatched records. In one aspect, output tables can be customized.”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Lambert as modified by Wray to incorporate the teachings of Johnson for the same motivation reasons in claim 2.
Regarding claim 4, Lambert as modified by Wray, and Johnson disclose the claimed features of claim 3 and further disclose, Lambert fails to explicitly disclose, However, Johnson discloses, wherein the attributes of the roads include at least one of:numbers of lanes on the roads;road markings on the roads;types of the roads; or speed limits of the roads. (See [Column 1, Lines 33-42] “Information similar to the information found in an SD map, for example, a map based on road curves, elevation changes, and/or location coordinates. There also may be local information about road-specific speed limits. Information to define such things as lane size and location on a road; lane type, such as regular, shoulder, passing, and/or express lanes, as well as HOV lanes and/or bike lanes. There may be information about speeds within a lane; striping or solid lines; and/or line color. ”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Lambert as modified by Wray to incorporate the teachings of Johnson for the same motivation reasons in claim 2.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Wesam Almadhrhi whose telephone number is (571) 270-3844. The examiner can normally be reached on 7:30 AM - 5PM Mon-Fri Eastern Alt Fri.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached on (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESAM NMN ALMADHRHI/Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights