DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 02/16/2026 is being entered. Claims 1, 4, 6, 8, 11-12, 14, 17, and 19 are amended. Claims 1-20 are pending, and rejected as detailed below. This action is final as necessitated by amendment.
35 U.S.C. 112(a) Claim Rejections
Amendment to claims 1, 8, and 14 are entered. Therefore the 35 U.S.C. 112(a) claim rejection for claims 1, 8, and 14 have been withdrawn.
35 U.S.C. 112(b) Claim Rejections
Amendment to claims 1, 8, and 14 are entered. Therefore the 35 U.S.C. 112(b) claim rejection for claims 1, 8, and 14 have been withdrawn.
Response to Arguments
Claim Rejections under 35 U.S.C. §102
Arguments:
Applicant argues that Ben-Shachar does not teach "transforming, to edge- relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks, wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map and an offset from that edge," as recited in Independent Claims 1, 8, and 14.
The Office Action alleges that Ben-Shachar teaches the distance-along-an-edge coordinate of edge-relative coordinates in [0324] and the offset-from-that-edge coordinate in [0336] and [0366]. This is incorrect because those paragraphs are taken out of context and misapplied to the claims, as demonstrated below.
Regarding [0324], that paragraph pertains to a vehicle correcting its localization through "image observations of landmarks." That is, the vehicle compares its location to the location of a known landmark and adjusts or corrects its localization (its estimate of where it is located) accordingly. Ben-Shachar states, "The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800." This has nothing to do with transforming a set of estimated locations for a landmark to edge-relative coordinates. Ben-Shachar's example for a landmark's location is "GPS data"-latitude and longitude, neither of which corresponds to edge-relative coordinates. The phrase "along a target trajectory" is simply stating that the landmark has a known location somewhere along the vehicle's route. There is nothing in [0324] about measuring or determining a distance along an edge of the base zone map, which the Office Action alleges corresponds to Ben-Shachar's "map skeleton" in FIG. 14 of Ben-Shachar. Note that there is no mention of the "map skeleton 1420" in [0324].
Response:
Applicant’s arguments with respect to para. 0324 and the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive. More specifically, Ben-Shachar [203-204] teaches how image processing is utilized to enhance to sparse map in relation to the detected landmarks so that the vehicle navigation can be optimized. Then, Ben-Shachar [0324; “Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800”] teaches how the received data is processed in relation to the target trajectory of the vehicle so that the distance along an edge (target trajectory) can be calculated. Furthermore, Ben-Shachar [0293; FIG. 15 illustrates an example by which additional detail may be generated for a sparse map within a segment of a map skeleton (e.g., segment A to B within skeleton 1420)”] teaches that the sparse map is a part of the map skeleton 1420. Therefore, Ben-Shachar teaches about measuring or determining a distance along an edge of the base zone map.
Arguments:
Applicant argues that, Regarding [0336], this paragraph says nothing about transforming the spatial coordinates of a set of estimated locations for a landmark. Instead, it describes sampling point locations "to represent particular lane marks" and "to create a mapped lane mark in the sparse map." The paragraph goes on to say that the vehicle 200 "may be configured to detect a plurality of edge location points 2411 along the lane mark." There is nothing in this paragraph about an offset from an edge of the base zone map, which the Office Action alleges corresponds to Ben- Shachar's "map skeleton" in FIG. 14. Again, there is no mention of the “map skeleton 1420” in [0336].
Response:
Applicant’s arguments with respect to para. 0336 and the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been considered but are moot. More specifically, Examiner mistakenly states that Ben-Shachar [0336] teaches the offset from an edge in “Response to Arguments” section of the previous office action. This was typo as the correct paragraph that teaches the offset from an edge is Ben-Shachar [0366]. However, Examiner uses the correct reference paragraph, Ben-Shachar [0366], in claim 1, 8 and 14 to reject the corresponding claimed limitation in “Claim Rejections - 35 USC § 102” section of the previous office action. Examiner apologies for the confusion.
Arguments:
Applicant argues that, Regarding [0366], this paragraph simply teaches that a vehicle determines a lateral distance from the vehicle to a lane mark through, e.g., image recognition techniques or algorithms. The claims recite that the "offset" coordinate is from an edge of the base zone map (recall that the claims also recite that the edges of the base zone map represent roadways). If the lane mark is treated as a "landmark," the vehicle itself would have to be the "edge," but a vehicle cannot be an "edge," as that term is used throughout the specification and claims. A vehicle is not a roadway represented as the edge of a graph data structure. If the lane mark is treated as the "edge," the vehicle would have to be the "landmark," but a vehicle is not a landmark, as that term is consistently used throughout the specification and claims. Moreover, there is nothing in this paragraph about transforming the spatial coordinates of a set of estimated locations for a landmark. The "map skeleton 1420," which the Office Action alleges corresponds to the recited "base zone map," is also not mentioned in this paragraph.
Response:
Applicant’s arguments with respect to para. 0366 and the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive. More specifically, Ben-Shachar [0293; FIG. 15 illustrates an example by which additional detail may be generated for a sparse map within a segment of a map skeleton (e.g., segment A to B within skeleton 1420)”] teaches that the sparse map is a part of the map skeleton 1420. Examiner also provides the following explanation in relation to figure 25A to provide clarity for the following terms “edge”, “landmark”, and “offset”, and to show that Ben-Shachar [0336 and FIG. 25A] teaches the calculation of offset distance.
PNG
media_image1.png
411
511
media_image1.png
Greyscale
Arguments:
Applicant argues that Ben-Shachar does not teach or suggest "transforming, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks," Ben-Shachar clearly does not teach or suggest performing data association in an edge-relative coordinate frame, identifying a cluster of candidate locations for each landmark that edge-relative coordinate frame, and computing, for each landmark, a final estimated location based on the identified cluster for that landmark, as recited in Independent Claims 1, 8, and 14.
Response:
Applicant’s arguments, as amended herein, with respect to the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive. More specifically, Ben-Shachar [0419]. In particular, the amendments to claims 1, 8, and 14 are addressed in the instant office action.
Arguments:
Applicant argues that The claims, as currently amended, are allowable. In light of the current claim amendments and the foregoing arguments, Applicant believes Independent Claims 1, 8, and 14 to be allowable. Each of Claims 2-7, 9-13, and 15-20 is thus also allowable at least by virtue of it’s depending from an allowable claim. Withdrawal of the rejections, under § 102(a)(2), is respectfully requested.
Response:
Applicant’s arguments, as amended herein, with respect to the rejections of dependent claims of the independent claims 1, 8, and 14 have been fully considered and not persuasive as the independent claims 1, 8, and 14 are rejected based on Ben-Shachar.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ben-Shachar (US 20250174124 A1).
Regarding claim 1, Ben-Shachar teaches (Currently Amended) A system for generating map data (Ben-Shachar, at least one para. 0002; “The present disclosure relates generally to vehicle navigation and, more specifically, to systems and methods for detecting and/or classifying various objects in an environment of a vehicle”), the system comprising:
a processor (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor”); and
a memory storing machine-readable instructions that, when executed by the processor, cause the processor to (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor comprising circuitry and a memory. The memory may include instructions executable by the circuitry to cause the at least one processor to perform operations comprising receiving map data corresponding to a road segment on which the host vehicle is navigating or will navigate, wherein the map data comprises a landmark orientation for a landmark positioned relative to the road segment.”):
receive, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generate a base zone map of the region as a graph that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”, wherein the map skeleton 1420 shows the graph representation of the road network as shown below);
PNG
media_image2.png
340
449
media_image2.png
Greyscale
transform, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”, wherein FIG. 25A, as shown below, provide clarity for the following terms “edge”, “landmark”, and “offset”.);
PNG
media_image1.png
411
511
media_image1.png
Greyscale
perform data association using the edge-relative coordinates to identify, from the sets of estimated locations for the landmarks in the plurality of landmarks, a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0419; “By way of example, referring to FIG. 27B, drive identifiers V1, V2, and V3 may be determined as corresponding to objects 2721A, 2721B, and 2721C that may each represent road sign 2721 in a cluster of objects in region 2762. Further, drive identifiers V1 and V3 may be determined as corresponding to objects 2723A and 2723C that may each represent road sign 2723 in the cluster of objects in region 2762.”);
compute, for each landmark in the plurality of landmarks, a final estimated location based on the identified cluster for that landmark (Ben-Shachar, at least one para. 0419; “each of drive identifiers V1 and V3 may be included twice in the collection of drive identifiers associated with the landmark cluster in region 2762. Because at least one of the drive identifiers V1 or V3 is included more than once, the landmark cluster associated with region 2762 may be determined as including two actual landmarks, namely the two road signs 2721 and 2723.”); and
output a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for at least one of localization, navigation, or path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 2, Ben-Shachar teaches (Original) The system of claim 1, wherein the machine-readable instructions include further instructions that, when executed by the processor, cause the processor to compare the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). In this example, server 3045 may increase a confidence score for the indicator. As another example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is different from an earlier-determined status and/or that the detection information has a confidence score below a threshold (e.g., where the detection information indicates the sign is blocked or where the detection information is associated with at least one image with an image attribute below a threshold).”).
Regarding claim 3, Ben-Shachar teaches (Original) The system of claim 1, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 4, Ben-Shachar teaches (Currently Amended) The system of claim 1, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the landmarks in the plurality of landmarks include at least one of traffic signs, traffic signal lights, or roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 5, Ben-Shachar teaches (Original) The system of claim 1, wherein the spatial coordinates include latitude, longitude, (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and height above a ground level (Ben-Shachar, at least one para. 0234; “a landmark size may be stored using 8 bytes of data. A distance to a previous landmark, a lateral offset, and height may be specified using 12 bytes of data.”).
Regarding claim 6, Ben-Shachar teaches (Currently Amended) The system of claim 1, wherein the edge-relative coordinates improve the data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 7, Ben-Shachar teaches (Original) The system of claim 6, wherein the machine-readable instructions cause the processor to compute the final estimated location for each landmark in the plurality of landmarks as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Regarding claim 8, Ben-Shachar teaches (Currently Amended) A non-transitory computer-readable medium for generating map data and storing instructions that, when executed by a processor, cause the processor to (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor comprising circuitry and a memory. The memory may include instructions executable by the circuitry to cause the at least one processor to perform operations comprising receiving map data corresponding to a road segment on which the host vehicle is navigating or will navigate, wherein the map data comprises a landmark orientation for a landmark positioned relative to the road segment.”):
receive, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generate a base zone map of the region as a graph that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”, wherein the map skeleton 1420 shows the graph representation of the road network as shown below);
PNG
media_image2.png
340
449
media_image2.png
Greyscale
transform, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”, wherein FIG. 25A, as shown below, provide clarity for the following terms “edge”, “landmark”, and “offset”.);
PNG
media_image1.png
411
511
media_image1.png
Greyscale
perform data association using the edge-relative coordinates to identify, from the sets of estimated locations for the landmarks in the plurality of landmarks, a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0419; “By way of example, referring to FIG. 27B, drive identifiers V1, V2, and V3 may be determined as corresponding to objects 2721A, 2721B, and 2721C that may each represent road sign 2721 in a cluster of objects in region 2762. Further, drive identifiers V1 and V3 may be determined as corresponding to objects 2723A and 2723C that may each represent road sign 2723 in the cluster of objects in region 2762.”);
compute, for each landmark in the plurality of landmarks, a final estimated location based on the identified cluster for that landmark (Ben-Shachar, at least one para. 0419; “each of drive identifiers V1 and V3 may be included twice in the collection of drive identifiers associated with the landmark cluster in region 2762. Because at least one of the drive identifiers V1 or V3 is included more than once, the landmark cluster associated with region 2762 may be determined as including two actual landmarks, namely the two road signs 2721 and 2723.”); and
output a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for at least one of localization, navigation, or path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 9, Ben-Shachar teaches (Original) The non-transitory computer-readable medium of claim 8, wherein the instructions include further instructions that, when executed by the processor, cause the processor to compare the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). In this example, server 3045 may increase a confidence score for the indicator. As another example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is different from an earlier-determined status and/or that the detection information has a confidence score below a threshold (e.g., where the detection information indicates the sign is blocked or where the detection information is associated with at least one image with an image attribute below a threshold).”).
Regarding claim 10, Ben-Shachar teaches (Original) The non-transitory computer-readable medium of claim 8, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 11, Ben-Shachar teaches (Currently Amended) The non-transitory computer-readable medium of claim 8, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the landmarks in the plurality of landmarks include at least one of traffic signs, traffic signal lights, or roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 12, Ben-Shachar teaches (Currently Amended) The non-transitory computer-readable medium of claim 8, wherein the edge-relative coordinates improve the data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 13, Ben-Shachar teaches (Original) The non-transitory computer-readable medium of claim 12, wherein the instructions cause the processor to compute the final estimated location for each landmark in the plurality of landmarks as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Regarding claim 14, Ben-Shachar teaches (Currently Amended) A method (Ben-Shachar, at least one para. 0002; “The present disclosure relates generally to vehicle navigation and, more specifically, to systems and methods for detecting and/or classifying various objects in an environment of a vehicle”), comprising:
receiving, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generating a base zone map of the region as a graph that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”, wherein the map skeleton 1420 shows the graph representation of the road network as shown below);
PNG
media_image2.png
340
449
media_image2.png
Greyscale
transforming, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”, wherein FIG. 25A, as shown below, provide clarity for the following terms “edge”, “landmark”, and “offset”.);
PNG
media_image1.png
411
511
media_image1.png
Greyscale
performing data association using the edge-relative coordinates to identify, from the sets of estimated locations for the landmarks in the plurality of landmarks, a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0419; “By way of example, referring to FIG. 27B, drive identifiers V1, V2, and V3 may be determined as corresponding to objects 2721A, 2721B, and 2721C that may each represent road sign 2721 in a cluster of objects in region 2762. Further, drive identifiers V1 and V3 may be determined as corresponding to objects 2723A and 2723C that may each represent road sign 2723 in the cluster of objects in region 2762.”);
computing, for each landmark in the plurality of landmarks, a final estimated location based on the identified cluster for that landmark (Ben-Shachar, at least one para. 0419; “each of drive identifiers V1 and V3 may be included twice in the collection of drive identifiers associated with the landmark cluster in region 2762. Because at least one of the drive identifiers V1 or V3 is included more than once, the landmark cluster associated with region 2762 may be determined as including two actual landmarks, namely the two road signs 2721 and 2723.”); and
outputting a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for one or more of localization, navigation, and path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 15, Ben-Shachar teaches (Original) The method of claim 14, further comprising comparing the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). ”).
Regarding claim 16, Ben-Shachar teaches (Original) The method of claim 14, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 17, Ben-Shachar teaches (Currently Amended) The method of claim 14, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the plurality of landmarks include at least one of traffic signs, traffic signal lights, or roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 18, Ben-Shachar teaches (Original) The method of claim 14, wherein the spatial coordinates include latitude, longitude, (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and height above a ground level (Ben-Shachar, at least one para. 0234; “a landmark size may be stored using 8 bytes of data. A distance to a previous landmark, a lateral offset, and height may be specified using 12 bytes of data.”).
Regarding claim 19, Ben-Shachar teaches (Currently Amended) The method of claim 14, wherein the edge-relative coordinates improve the data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 20, Ben-Shachar teaches (Original) The method of claim 19, wherein the final estimated location for each landmark in the plurality of landmarks is computed as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UPUL P CHANDRASIRI whose telephone number is (703)756-5823. The examiner can normally be reached M-F 8.30 am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at 571-272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/U.P.C./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights