DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Application status
This office action is in response to remarks filed on 09/04/2025. Claims 1-20 are pending. Claims 1-20 are rejected.
Response to Arguments
Claim Rejections under 35 U.S.C. §102
Applicant argues that Ben-Shachar of representing a path as points having coordinates separated by a predetermined distance and of using polynomials to represent right and left roadway boundaries, there is nothing in Ben-Shachar about generating a base zone map of a region in which roadways are represented as the edges of a graph and intersections are represented as the junctions of a graph, in accordance with graph theory. Therefore, Ben-Shachar does not teach or suggest the above-quoted feature recited in the independent claims.
Applicant’s arguments of Ben-Shachar not teaching a generation of a base zone map of a region in which roadways are represented as the edges of a graph and intersections are represented as the junctions of a graph with respect to the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive as Ben-Shachar teaches the generation of a map skeleton in para. [0292].
Applicant also argues that Ben-Shachar does not teach "[transform/transforming, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks, wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map and an offset from that edge to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks," as recited in Independent Claims 1, 8, and 14.
Applicant’s arguments of Ben-Shachar not teaching edge-relative coordinates with respect to the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive as Ben-Shachar teaches a distance along an edge of the base zone map and an offset from that edge in para. [0324] and [0366].
Applicant also argues Ben-Shachar, at [0366], discusses "determining an actual lateral distance to the at least one lane mark based on analysis of at least one image," that does not meet the recited "edge-relative coordinates" for several reasons. First, the "at least one lane mark" in Ben-Shachar is not an "edge" as that term is used in the claims, as explained above; it is not an "edge" of a graph (the base zone map). Second, even if Ben-Shachar did teach the concept of edges and junctions (it does not), the other component of the edge-relative coordinates-the distance along an edge of the base zone map-would still be missing from the reference. Third, Ben-Shachar does not discuss edge-relative coordinates or anything that could arguably be equivalent to edge-relative coordinates.
Applicant’s arguments of Ben-Shachar not teaching edge-relative coordinates with respect to the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive. In reference to para [0346] Ben-Shachar teaches that the road edge 2510 and road sign 2521 (yield sign that indicates a junction) are both consider as “landmarks”. Furthermore, Ben-Shachar teaches concept of edges and junctions in para [0292] and shown the corresponding map in FIG. 14. Furthermore, Ben-Shachar teaches the distance along an edge in para [0324] and offset from that edge in para [0336].
Applicant also argues Ben-Shachar is silent regarding both a Global Nearest Neighbor algorithm and using edge-relative coordinates to improve the performance of data association performed by that algorithm. Therefore, Ben-Shachar does not teach or suggest the above-quoted feature recited in the independent claims.
Applicant’s arguments of Ben-Shachar not teaching a Global Nearest Neighbor algorithm and using edge-relative coordinates to improve the performance of data association performed by that algorithm with respect to the rejections of claims 1, 8, and 14 under 35 U.S.C. §102 have been fully considered and not persuasive. More specifically, Ben-Shachar teaches the use of any form of machine learning model or algorithm to determine an orientation of landmark in para. [0486]. Therefore, it would be obvious to one of ordinary skill in the art to use any form of machine learning model or algorithm of Ben-Shachar to determine an exact orientation of landmark taught by claimed invention. Furthermore, applicant argument with respect the Global Nearest Neighbor (GNN) indicates “intended use” and does not further illustrate any structural differences between the Global Nearest Neighbor (GNN) algorithm of the claimed invention and any form of machine learning model or algorithm of Ben-Shachar (MPEP at ¶ 7.37.09; a recitation of the intended use of the claimed invention must result in a structural difference between the claimed invention and the prior art in order to patentably distinguish the claimed invention from the prior art. If the prior art structure is capable of performing the intended use, then it meets the claim.).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 8, and 14 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Applicant has mentioned “to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks” in claims 1, 8, and 14. However, Applicant fails to provide any additional information such as How the Global Nearest Neighbor (GNN) algorithm improves? What is performing data association? What type of data is used for the process of performing data association?
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 8, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks” in claims 1, 8, and 14 is a relative term which renders the claim indefinite. The term “to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It is unclear how the Global Nearest Neighbor (GNN) algorithm improves? What is performing data association? What type of data is used for the process of performing data association?
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ben-Shachar (US 20250174124 A1).
Regarding claim 1, Ben-Shachar teaches A system for generating map data (Ben-Shachar, at least one para. 0002; “The present disclosure relates generally to vehicle navigation and, more specifically, to systems and methods for detecting and/or classifying various objects in an environment of a vehicle”), the system comprising:
a processor (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor”); and
a memory storing machine-readable instructions that, when executed by the processor, cause the processor to (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor comprising circuitry and a memory. The memory may include instructions executable by the circuitry to cause the at least one processor to perform operations comprising receiving map data corresponding to a road segment on which the host vehicle is navigating or will navigate, wherein the map data comprises a landmark orientation for a landmark positioned relative to the road segment.”):
receive, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generate a base zone map of the region that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”);
PNG
media_image1.png
340
449
media_image1.png
Greyscale
transform, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”) to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0338; “vehicle 200 may be configured to detect these center points using various image recognition techniques, such as convolutional neural networks (CNN), scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG) features, or other techniques..”) and (Ben-Shachar, at least one para. 0486; “As a result of the training process, the neural network may be configured to generate an orientation based on various location indicators input to the model. Various other training or machine learning algorithms may be used, including a logistic regression, a linear regression, a regression, a random forest, a K-Nearest Neighbor (KNN) model, a K-Means model, a decision tree, a cox proportional hazards regression model, a Naïve Bayes model, a Support Vector Machines (SVM) model, a gradient boosting algorithm, or any other form of machine learning model or algorithm.”, wherein it would be obvious to one of ordinary skill in the art to use any form of machine learning model or algorithm of Ben-Shachar to determine an exact orientation of landmark taught by claimed invention.); and
output a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for one or more of localization, navigation, and path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 2, Ben-Shachar teaches The system of claim 1, wherein the machine-readable instructions include further instructions that, when executed by the processor, cause the processor to compare the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). In this example, server 3045 may increase a confidence score for the indicator. As another example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is different from an earlier-determined status and/or that the detection information has a confidence score below a threshold (e.g., where the detection information indicates the sign is blocked or where the detection information is associated with at least one image with an image attribute below a threshold).”).
Regarding claim 3, Ben-Shachar teaches The system of claim 1, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 4, Ben-Shachar teaches The system of claim 1, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the landmarks in the plurality of landmarks include one or more of traffic signs, traffic signal lights, and roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 5, Ben-Shachar teaches The system of claim 1, wherein the spatial coordinates include latitude, longitude, (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and height above a ground level (Ben-Shachar, at least one para. 0234; “a landmark size may be stored using 8 bytes of data. A distance to a previous landmark, a lateral offset, and height may be specified using 12 bytes of data.”).
Regarding claim 6, Ben-Shachar teaches The system of claim 1, wherein the edge-relative coordinates improve the GNN algorithm in performing data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist the GNN algorithm in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 7, Ben-Shachar teaches The system of claim 6, wherein the machine-readable instructions cause the processor to compute the final estimated location for each landmark in the plurality of landmarks as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Regarding claim 8, Ben-Shachar teaches A non-transitory computer-readable medium for generating map data and storing instructions that, when executed by a processor, cause the processor to (Ben-Shachar, at least one para. 0011; “In an embodiment, a navigation system for a host vehicle may include at least one processor comprising circuitry and a memory. The memory may include instructions executable by the circuitry to cause the at least one processor to perform operations comprising receiving map data corresponding to a road segment on which the host vehicle is navigating or will navigate, wherein the map data comprises a landmark orientation for a landmark positioned relative to the road segment.”):
receive, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generate a base zone map of the region that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”);
PNG
media_image1.png
340
449
media_image1.png
Greyscale
transform, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”) to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0338; “vehicle 200 may be configured to detect these center points using various image recognition techniques, such as convolutional neural networks (CNN), scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG) features, or other techniques..”) and (Ben-Shachar, at least one para. 0486; “As a result of the training process, the neural network may be configured to generate an orientation based on various location indicators input to the model. Various other training or machine learning algorithms may be used, including a logistic regression, a linear regression, a regression, a random forest, a K-Nearest Neighbor (KNN) model, a K-Means model, a decision tree, a cox proportional hazards regression model, a Naïve Bayes model, a Support Vector Machines (SVM) model, a gradient boosting algorithm, or any other form of machine learning model or algorithm.”, wherein it would be obvious to one of ordinary skill in the art to use any form of machine learning model or algorithm of Ben-Shachar to determine an exact orientation of landmark taught by claimed invention.); and
output a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for one or more of localization, navigation, and path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 9, Ben-Shachar teaches The non-transitory computer-readable medium of claim 8, wherein the instructions include further instructions that, when executed by the processor, cause the processor to compare the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). In this example, server 3045 may increase a confidence score for the indicator. As another example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is different from an earlier-determined status and/or that the detection information has a confidence score below a threshold (e.g., where the detection information indicates the sign is blocked or where the detection information is associated with at least one image with an image attribute below a threshold).”).
Regarding claim 10, Ben-Shachar teaches The non-transitory computer-readable medium of claim 8, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 11, Ben-Shachar teaches The non-transitory computer-readable medium of claim 8, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the landmarks in the plurality of landmarks include one or more of traffic signs, traffic signal lights, and roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 12, Ben-Shachar teaches The non-transitory computer-readable medium of claim 8, wherein the edge-relative coordinates improve the GNN algorithm in performing data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist the GNN algorithm in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 13, Ben-Shachar teaches The non-transitory computer-readable medium of claim 12, wherein the instructions cause the processor to compute the final estimated location for each landmark in the plurality of landmarks as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Regarding claim 14, Ben-Shachar teaches A method (Ben-Shachar, at least one para. 0002; “The present disclosure relates generally to vehicle navigation and, more specifically, to systems and methods for detecting and/or classifying various objects in an environment of a vehicle”), comprising:
receiving, from one or more vehicles that traveled within a region, a set of estimated locations for each landmark in a plurality of landmarks within the region (Ben-Shachar, at least one para. 0147; “In a three camera system, a first processing device may receive images from both the main camera and the narrow field of view camera, and perform vision processing of the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects.”);
generating a base zone map of the region that represents roadways as edges and intersections as junctions (Ben-Shachar, at least one para. 0292 and FIG. 14 as shown below; “server 1230 may generate a map skeleton 1420 using one or more statistical techniques to determine whether variations in the raw location data 1410 represent actual divergences or statistical errors. Each path within skeleton 1420 may be linked back to the raw data 1410 that formed the path. For example, the path between A and B within skeleton 1420 is linked to raw data 1410 from drives 2, 3, 4, and 5 but not from drive 1. Skeleton 1420 may not be detailed enough to be used to navigate a vehicle (e.g., because it combines drives from multiple lanes on the same road unlike the splines described above) but may provide useful topological information and may be used to define intersections.”);
PNG
media_image1.png
340
449
media_image1.png
Greyscale
transforming, to edge-relative coordinates, spatial coordinates of the set of estimated locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0203-0204; “Semantic objects may also include other recognized object or feature types that are not associated with certain standardized characteristics. Such objects or features may include potholes, tar seams, light poles, non-standardized signs, curbs, trees, tree branches, or any other type of recognized object type with one or more variable characteristics (e.g., variable dimensions). In such cases, in addition to transmitting to a server an indication of the detected object or feature type (e.g., pothole, pole, etc.) and position information for the detected object or feature, a harvesting vehicle may also transmit an indication of a size of the object or feature. The size may be expressed in 2D image dimensions (e.g., with a bounding box or one or more dimension values) or real-world dimensions (determined through structure in motion calculations, based on LIDAR or RADAR system outputs, based on trained neural network outputs, etc.). In some cases, such non-semantic features may include a detected corner of a building or a corner of a detected window of a building, a unique stone or object near a roadway, a concrete splatter in a roadway shoulder, or any other detectable object or feature. Upon detecting such an object or feature one or more harvesting vehicles may transmit to a map generation server a location of one or more points (2D image points or 3D real world points) associated with the detected object/feature. Additionally, a compressed or simplified image segment (e.g., an image hash) may be generated for a region of the captured image including the detected object or feature. This image hash may be calculated based on a predetermined image processing algorithm and may form an effective signature for the detected non-semantic object or feature.”), wherein the edge-relative coordinates are defined in terms of a distance along an edge of the base zone map (Ben-Shachar, at least one para. 0324; “when vehicle detects a landmark within an image captured by the camera, the landmark may be compared to a known landmark stored within the road model or sparse map 800. The known landmark may have a known location (e.g., GPS data) along a target trajectory stored in the road model and/or sparse map 800. Based on the current speed and images of the landmark, the distance from the vehicle to the landmark may be estimated. The location of the vehicle along a target trajectory may be adjusted based on the distance to the landmark and the landmark's known location (stored in the road model or sparse map 800). The landmark's position/location data (e.g., mean values from multiple drives) stored in the road model and/or sparse map 800 may be presumed to be accurate.”) and an offset from that edge (Ben-Shachar, at least one para. 0366; “At step 2625, process 2600B may include determining an actual lateral distance to the at least one lane mark based on analysis of the at least one image. For example, the vehicle may determine a distance 2530, as shown in FIG. 25A, representing the actual distance between the vehicle and lane mark 2510.”) to improve a Global Nearest Neighbor (GNN) algorithm in performing data association to generate a final estimated location for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0338; “vehicle 200 may be configured to detect these center points using various image recognition techniques, such as convolutional neural networks (CNN), scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG) features, or other techniques..”) and (Ben-Shachar, at least one para. 0486; “As a result of the training process, the neural network may be configured to generate an orientation based on various location indicators input to the model. Various other training or machine learning algorithms may be used, including a logistic regression, a linear regression, a regression, a random forest, a K-Nearest Neighbor (KNN) model, a K-Means model, a decision tree, a cox proportional hazards regression model, a Naïve Bayes model, a Support Vector Machines (SVM) model, a gradient boosting algorithm, or any other form of machine learning model or algorithm.”, wherein it would be obvious to one of ordinary skill in the art to use any form of machine learning model or algorithm of Ben-Shachar to determine an exact orientation of landmark taught by claimed invention.); and
outputting a final zone map that includes the final estimated location for at least one landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0211; “In general, sparse map 800 may be generated based on data (e.g., drive information) collected from one or more vehicles as they travel along roadways. For example, using sensors aboard the one or more vehicles (e.g., cameras, speedometers, GPS, accelerometers, etc.), the trajectories that the one or more vehicles travel along a roadway may be recorded, and the polynomial representation of a preferred trajectory for vehicles making subsequent trips along the roadway may be determined based on the collected trajectories travelled by the one or more vehicles. Similarly, data collected by the one or more vehicles may aid in identifying potential landmarks along a particular roadway. Data collected from traversing vehicles may also be used to identify road profile information, such as road width profiles, road roughness profiles, traffic line spacing profiles, road conditions, etc.”);
wherein the final zone map is used for one or more of localization, navigation, and path planning to control an autonomous vehicle (Ben-Shachar, at least one para. 0211; “Using the collected information, sparse map 800 may be generated and distributed (e.g., for local storage or via on-the-fly data transmission) for use in navigating one or more autonomous vehicles.”).
Regarding claim 15, Ben-Shachar teaches The method of claim 14, further comprising comparing the final zone map with an earlier version of the final zone map to identify and output changes in landmarks within the region (Ben-Shachar, at least one para. 0464; “Modifying a confidence score for the indicator may be based on received or generated detection information. For example, server 3045 may generate detection information based on at least one first image received from at least one vehicle, and the detection information may indicate that a status of a landmark is the same as an earlier-determined status (e.g., based on detection information based on at least one second image captured prior to the first image). ”).
Regarding claim 16, Ben-Shachar teaches The method of claim 14, wherein the set of estimated locations for each landmark in the plurality of landmarks is derived from perception systems in the one or more vehicles that process raw sensor data output by sensors in the one or more vehicles (Ben-Shachar, at least one para. 0381; “In some embodiments, the landmark detection information comprises sensor data obtained by at least one sensor of one or more of the plurality of vehicles. As discussed elsewhere in this disclosure, the one or more vehicles (autonomous or non-autonomous) may include one or more sensors (e.g., image capture devices 122, 124, 126, position sensor 130, radar sensor, LIDAR sensor). In some embodiments, the at least one sensor includes a camera, a radar, or a lidar. In some embodiments, the sensor data includes at least one of images captured by an image capture device, radar data, or lidar data. ”).
Regarding claim 17, Ben-Shachar teaches The method of claim 14, wherein the spatial coordinates include latitude and longitude (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and the plurality of landmarks include one or more of traffic signs, traffic signal lights, and roadway features (Ben-Shachar, at least one para. 0395; “In some embodiments, the one or more actual landmarks includes at least a traffic sign, a traffic light, a road marking, a pole, or a construction indicator.”).
Regarding claim 18, Ben-Shachar teaches The method of claim 14, wherein the spatial coordinates include latitude, longitude, (Ben-Shachar, at least one para. 0378; “In some embodiments, the landmark detection information comprises one or more three-dimensional real-world coordinates corresponding to a surface of a landmark of the one or more landmarks. As discussed elsewhere in this disclosure, three-dimensional real-world coordinates may include, for example, latitude/longitude coordinates.”) and height above a ground level (Ben-Shachar, at least one para. 0234; “a landmark size may be stored using 8 bytes of data. A distance to a previous landmark, a lateral offset, and height may be specified using 12 bytes of data.”).
Regarding claim 19, Ben-Shachar teaches The method of claim 14, wherein the edge-relative coordinates improve the GNN algorithm in performing data association by clarifying spatial relationships among the plurality of landmarks with respect to one or more edges in the base zone map to assist the GNN algorithm in identifying, from the sets of estimated locations for the landmarks in the plurality of landmarks (Ben-Shachar, at least one para. 0418; “In some embodiments, a count of the one or more actual landmarks positioned along the road segment for one of the at least two landmark clusters is equal to one when a same drive identifier is not included in the distribution of the drive identifiers for the one of the at least two landmark clusters.”), a cluster of candidate locations for each landmark in the plurality of landmarks (Ben-Shachar, at least one para. 0420; “In contrast, drive identifiers V1, V2, V3, and V4 may be determined as corresponding to objects 2724A, 2724B, 2724C, and 2724D, respectively, that may each represent landmark 2724 in a cluster of objects in region 2762. Because none of the drive identifiers V1, V2, V3, or V4 is included more than once relative to the landmark cluster associated with region 2762, that cluster may be determined as including only one actual landmark, namely landmark 2724.”).
Regarding claim 20, Ben-Shachar teaches The method of claim 19, wherein the final estimated location for each landmark in the plurality of landmarks is computed as a centroid of the cluster of candidate locations for that landmark (Ben-Shachar, at least one para. 0421; “image processor 190 may aggregate landmark detection information included in the drive information and may identify at least two landmark clusters as discussed elsewhere in this disclosure. One or both of application processor 180 and image processor 190 may determine a distribution of drive identifiers relative to the identified landmark clusters as discussed above. One or both of application processor 180 and image processor 190 may also determine a location identifier for one or more actual landmarks based on the distribution of drive identifiers.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UPUL P CHANDRASIRI whose telephone number is (703)756-5823. The examiner can normally be reached M-F 8.30 am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at 571-272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/U.P.C./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights