DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/16/2024, 9/20/2024, and 12/20/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Status of Claims
This Office action is in reply to filing by applicant on April 16, 2024.
Claims 1 – 20 are currently pending and have been examined.
This action is made non-final.
Examiner Note
The invention utilizes various sensors on an autonomous vehicle to relate local surroundings (road name, object, etc.) to known map data, thereby pinpointing its location respecting the said map data, then using such location to automatically navigate the vehicle, depending upon various levels of confidence in the sensed data. There is no “Alice” sort of 35 USC 101 subject matter eligibility rejection.
Claim Rejections – 35 USC 103
In the event the determination of the status of the application as subject to AIA 35 USC 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office Action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 USC 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
Claims 1 – 20 are rejected pursuant to 35 USC 103 as being unpatentable over Hayat (JP2022553491A, an English copy is attached hereto) in view of Zhang (US20220197286A1).
Regarding claims 1, 16, and 20:
Hayat discloses:
receiving a sensor dataset acquired by a sensor subsystem, wherein: the sensor dataset includes information about a road, (“In some embodiments of the present disclosure, an autonomous vehicle may use information obtained while navigating (eg, from cameras, GPS devices, accelerometers, speed sensors, suspension sensors, etc.).”, [3d paragraph below “Overview”]) and (“A processor receives from the image capture device at least one image an autonomous vehicle may use information obtained while and analyzes the at least one image to detect the presence of one or more objects represented in the at least one image, determining location information associated with the one or more detected objects based on analysis of the at least one image,”, [3d para below “Overview of self-driving cars]);
the sensor subsystem comprises multiple different types of sensors including at least one of a camera, a light detection and ranging (LiDAR) sensor, a positioning sensor, a radar sensor, or a mapping sensor, and (“The disclosed embodiments may use cameras to provide autonomous vehicle navigation functionality. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, or more cameras that monitor the environment of the vehicle. The disclosed system may provide navigational responses based, for example, on analysis of images captured by one or more of these cameras.”, [4th para below “Description”])
the sensor dataset has a first spatial accuracy level; and (“In some embodiments, the landmarks included in sparse map 800 may be represented by data objects of predetermined size. Data representing landmarks may include any parameter suitable for identifying a particular landmark. For example, in some embodiments, the landmarks stored in the sparse map 800 include the physical size of the landmark (e.g., a size to aid in estimating the distance to the landmark based on its known size/scale). distance to previous landmark, lateral offset, height, type code (e.g. type of landmark, i.e. what type of directional sign, traffic sign, etc.), GPS coordinates (e.g. to assist in accurate localization) and any other suitable parameters may be included.”);
determining, by at least one processor, a confidence level by comparing the sensor dataset and the map that includes prior information about the road, (“As the vehicle 200 approaches the intersection, at step 564, the processing unit 110 may update confidences associated with the analyzed intersection geometry and the detected traffic lights. For example, the estimated number of traffic lights that appear at an intersection can affect confidence when compared to the number that actually appears at the intersection. Therefore, based on confidence, the processing unit 110 may delegate control to the driver of the vehicle 200 in order to improve safety conditions. Performing steps 560, 562, and 564, processing unit 110 may identify traffic lights that appear in the set of captured images and analyze intersection geometry information. Based on this identification and analysis, processing unit 110 can generate one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.”) and (“In some embodiments, the landmarks included in sparse map 800 may be represented by data objects of predetermined size. Data representing landmarks may include any parameter suitable for identifying a particular landmark. For example, in some embodiments, the landmarks stored in the sparse map 800 include the physical size of the landmark (e.g., a size to aid in estimating the distance to the landmark based on its known size/scale). distance to previous landmark, lateral offset, height, type code (e.g. type of landmark, i.e. what type of directional sign, traffic sign, etc.), GPS coordinates (e.g. to assist in accurate localization) and any other suitable parameters may be included.”);
wherein the map has a second spatial accuracy level; (“Estimate the position of the vehicle relative to the target trajectory. Errors can be accumulated in dead-reckoning navigation, and over time the position determination relative to the target trajectory can become less and less accurate. The vehicle can use the landmarks (and their known positions) present in the sparse map 800 to remove dead-reckoning errors in position determination. In this manner, the identified landmarks included in the sparse map 800 can serve as navigational pivots that can determine the precise position of the vehicle relative to the target trajectory. The identified landmarks do not necessarily have to be available to the autonomous vehicle, as a certain amount of error can be tolerated in location finding. Rather, preferred navigation may be possible based on landmark intervals of 10 meters, 20 meters, 50 meters, 100 meters, 500 meters, 1 kilometer, 2 kilometers, or longer, as described above. In some embodiments, a density of one identified landmark per kilometer of road may be sufficient to maintain longitudinal position determination accuracy to within one meter.”), the system is capable of determining spatial error / accuracy levels;
Hayat does not expressly disclose, but Zhang teaches:
in response to determining that the confidence level exceeds a confidence threshold, processing the map by the at least one processor; and (“For example, the vehicle 105 may control itself while in the autonomous mode, and may be operable to determine a current state of the vehicle and its environment, determine a predicted behavior of at least one other vehicle in the environment, determine a confidence level that may correspond to a likelihood of the at least one other vehicle to perform the predicted behavior, and control the vehicle 105 based on the determined information. While in autonomous mode, the vehicle 105 may be configured to operate without human interaction.”, [042]) and (“For example, HD maps can define properties of the lanes for all routes in the mapped area with high enough accuracy to be used for lane level navigation.”, [062]);
storing the processed map as an electronic file, (“The in-vehicle control system 150 can include a data processor 171 configured to execute the image processing module 200 for processing image data received from one or more of the vehicle subsystems 140. The data processor 171 can be combined with a data storage device 172 as part of a computing system 170 in the in-vehicle control system 150. The data storage device 172 can be used to store data, processing parameters, and data processing instructions”, [036]);
wherein the processed map is configured to guide an autonomous vehicle to operate on the road. (“receive output from at least one sensor located on the 171 configured to execute the image processing module 200 for processing image of the autonomous vehicle, retrieve a navigational map used for driving the autonomous vehicle, … trigger mapping of the driving environment based on the output of the at least one sensor, update the navigational map based on the mapped driving environment, and drive the autonomous vehicle using the updated navigational map.”, [003], and see Abstract, published 6/23/2022.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Hayat to incorporate the teachings of Zhang because Hayat would be more efficient and versatile if it could navigate using its correlated map as done in Zhang. (“in response to detecting the one or more inconsistencies, trigger mapping of the driving environment based on the output of the at least one sensor; update the navigational map based on the mapped driving environment; and drive the autonomous vehicle using the updated navigational map.”, Zhang at [013]).
Regarding claim 2:
The combination of Hayat and Zhang have the limitations of claim 1:
Hayat further teaches:
in response to determining that the confidence level exceeds the confidence threshold, causing a notification to be transmitted to an operator; and receiving an input from the operator indicating at least one of: maintaining the map, or updating the map based on the sensor dataset; and processing the map comprises processing the map according to the input. (“As the vehicle 200 approaches the intersection, at step 564, the processing unit 110 may update confidences associated with the analyzed intersection geometry and the detected traffic lights. For example, the estimated number of traffic lights that appear at an intersection can affect confidence when compared to the number that actually appears at the intersection. Therefore, based on confidence, the processing unit 110 may delegate control to the driver of the vehicle 200 in order to improve safety conditions. Performing steps 560, 562, and 564, processing unit 110 may identify traffic lights that appear in the set of captured images and analyze intersection geometry information. Based on this identification and analysis, processing unit 110 can generate one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.”).
Regarding claim 3:
The combination of Hayat and Zhang have the limitations of claim 1:
Hayat further teaches:
wherein processing the map comprises updating the map based on the sensor dataset. (“As discussed above, wireless transceiver 172 may transmit and/or receive data over one or more networks (eg, cellular networks, the Internet, etc.). For example, wireless transceiver 172 may upload data collected by system 100 to one or more servers and download data from one or more servers. Via wireless transceiver 172, system 100 may receive periodic or on-demand updates to data stored in map database 160, memory 140, and/or memory 150, for example. Similarly, the wireless transceiver 172 may receive any data from the system 100 (eg, images captured by the image acquisition unit 120, position sensor 130 or other sensors, vehicle control system, etc.) and/or processing. Any data processed by unit 110 may be uploaded to one or more servers.”).
Regarding claim 4:
The combination of Hayat and Zhang have the limitations of claim 1:
Hayat further teaches:
wherein: the road comprises a plurality of road units; the sensor dataset comprises a set of data frames, each of the set of data frames corresponding to a section of the road represented in the data frame; and each of the plurality of road units corresponds to multiple data frames of the sensor dataset. (“At step 582 , processing unit 110 may analyze the navigational information identified at step 580 . In one embodiment, processing unit 110 may compute the distance (eg, along the trail) between the snail trail and the road polynomial. The variance of this distance along the trail meets a predetermined threshold (eg, 0.1-0.2 meters for straight roads, 0.3-0.4 meters for roads with gentle curves, and 0 for roads with sharp curves). .5-0.6 meters), the processing unit 110 may determine that the preceding vehicle is probably changing lanes.”), a predetermined threshold (confidence level), is had.
Regarding claim 5:
The combination of Hayat and Zhang have the limitations of claim 4:
Hayat further teaches:
wherein comparing the sensor dataset and the map comprises: for each of the plurality of road units, determining a unit confidence level. (“At step 582, processing unit 110 may analyze the navigational information identified at step 580 . In one embodiment, processing unit 110 may compute the distance (eg, along the trail) between the snail trail and the road polynomial. The variance of this distance along the trail meets a predetermined threshold (eg, 0.1-0.2 meters for straight roads, 0.3-0.4 meters for roads with gentle curves, and 0 for roads with sharp curves). .5-0.6 meters), the processing unit 110 may determine that the preceding vehicle is probably changing lanes.”, [qwqwqw]), a predetermined threshold (confidence level), is had.
Regarding claim 6:
The combination of Hayat and Zhang have the limitations of claim 5:
Hayat further teaches:
wherein: determining that the confidence level exceeds a threshold comprises determining that at least one unit confidence level of the plurality of road units exceeds the confidence threshold, and processing the map comprises: for each of the plurality of road units that has a corresponding unit confidence level exceeding the confidence threshold, updating, based on multiple data frames of the sensor dataset of the road unit, a portion of the map that corresponds to the road unit. (“At stage 556, processing unit 110 may perform multi-frame analysis, for example, by tracking detected segments through successive image frames and accumulating frame-by-frame data associated with the detected segments. . As the processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 will become more reliable and will be associated with progressively higher degrees of confidence. Thus, performing steps 550, 552, 554, and 556 allows processing unit 110 to identify road markings that appear in the set of captured images and obtain lane geometry information. Based on this identification and obtained information, processing unit 110 can generate one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.”).
Regarding claim 7:
The combination of Hayat and Zhang have the limitations of claim 5:
Hayat further teaches:
wherein for a first road unit from the plurality of road units, determining the unit confidence level comprises: obtaining multiple frame confidence levels for multiple data frames corresponding to the first road unit by comparing the multiple data frames with corresponding portions of the map, respectively; and determining the unit confidence level of the first road unit based on the multiple frame confidence levels. (“At stage 556, processing unit 110 may perform multi-frame analysis, for example, by tracking detected segments through successive image frames and accumulating frame-by-frame data associated with the detected segments. As the processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 will become more reliable and will be associated with progressively higher degrees of confidence. Thus, performing steps 550, 552, 554, and 556 allows processing unit 110 to identify road markings that appear in the set of captured images and obtain lane geometry information. Based on this identification and obtained information, processing unit 110 can generate one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.”).
Regarding claim 8:
The combination of Hayat and Zhang have the limitations of claim 5:
Hayat further teaches:
identifying at least two neighboring road units along the road that satisfy a merger condition, and obtaining a road segment by merging the at least two neighboring road units. Examiner broadly interprets this limitation to include that vehicles may merge from neighboring to common trajectories … (“The at least one navigation map segment may include multiple target trajectories associated with lanes of travel along the roadway represented by the at least one navigation map segment. For example, at least one navigation map segment may include target trajectories 2910, 2912, and 2914, as shown in FIG. Multiple target trajectories may be represented as three-dimensional splines on at least one navigation map segment, as described above with respect to FIG. In some embodiments, multiple target trajectories may be determined based on driving information obtained from multiple vehicles during previous passages of the roadway by multiple vehicles.”).
Regarding claim 9:
The combination of Hayat and Zhang have the limitations of claim 8:
Hayat further teaches:
determining a segment confidence level of the road segment based on unit confidence levels of the at least two neighboring road units. Examiner broadly interprets this limitation to include that neighboring paths can affect confidence levels, … (“At stage 556, processing unit 110 may perform multi-frame analysis, for example, by tracking detected segments through successive image frames and accumulating frame-by-frame data associated with the detected segments. . As the processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 will become more reliable and will be associated with progressively higher degrees of confidence. Thus, performing steps 550, 552, 554, and 556 allows processing unit 110 to identify road markings that appear in the set of captured images and obtain lane geometry information. Based on this identification and obtained information, processing unit 110 can generate one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.”).
Regarding claim 10:
The combination of Hayat and Zhang have the limitations of claim 5:
Hayat further teaches:
obtaining a plurality of road segments, wherein each of the plurality of road segments is obtained by merging at least two neighboring road units that satisfy a merger condition. (“FIG. 29 shows exemplary positions that may be determined for a vehicle associated with navigation map segment 2900 consistent with the disclosed embodiments. Navigation map segment 2900 may be part of a road navigation model, such as sparse map 800 or various other types of navigation models. Navigation map segment 2900 may include various representations of objects or features along the road segment. For example, navigation map segment 2900 may correspond to the portion of the roadway represented in image 2800 . Thus, navigation map segment 2900 may include representations of road signs 2810 and directional arrows 2820 as shown. Navigation map segment 2900 may include representations of other features such as road edges 2840 and 2842 and lane markings 2844 and 2846 (and/or 2830), but these representations are made clear in FIG. excluded because Navigation map segment 2900 may include one or more target trajectories 2910, 2912, and 2914, which may correspond to lanes of travel within the road segment. Consistent with the exemplary image 2800 shown in FIG. 28, the host vehicle may be traveling substantially along the target trajectory 2912 (although the exact location of the host vehicle may not necessarily be along the target trajectory 2912). good).”).
Regarding claim 11:
The combination of Hayat and Zhang have the limitations of claim 10:
Hayat further teaches:
obtaining a causing a road representation of the road to be output to a display, wherein the road representation comprises a plurality of segment representations each of which corresponds to a road segment of the plurality of road segments and relates to a segment confidence level of the road segment. (“In some embodiments, user interface 170 may include a display, speakers, tactile devices, and/or any other device for providing output information to a user.”) and (“As the processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 will become more reliable and will be associated with progressively higher degrees of confidence. Thus, performing steps 550, 552, 554, and 556 allows processing unit 110 to identify road markings that appear in the set of captured images and obtain lane geometry information. Based on this identification and obtained information, processing unit 110 can generate one or more navigation responses”).
Regarding claim 12:
The combination of Hayat and Zhang have the limitations of claim 10:
Hayat further teaches:
wherein a difference between segment confidence levels of any two neighboring road segments along the road fails to satisfy the merger condition. (“For example, the estimated number of traffic lights that appear at an intersection can affect confidence when compared to the number that actually appears at the intersection. Therefore, based on confidence, the processing unit 110 may delegate control to the driver of the vehicle 200 in order to improve safety conditions”).
Regarding claim 13:
The combination of Hayat and Zhang have the limitations of claim 1:
Hayat further teaches:
obtaining trajectories of a plurality of candidate users; identifying, based on the trajectories, a target user from the plurality of candidate users; and transmitting the processed map or a notification regarding the processed map to the target user before the target user reaches the road. Examiner broadly interprets this limitation to include that user may receive map info prior to his/her destination, … (“The host vehicle may determine its position and orientation relative to the map in order to use the map for navigation.’).
Regarding claim 14:
The combination of Hayat and Zhang have the limitations of claim 13:
Hayat further teaches:
wherein the plurality of candidate users comprise the autonomous vehicle. (“For example, autonomous vehicles may need to process and interpret visual information (e.g., information captured from cameras) and other sources of information (e.g., GPS devices, speed sensors, accelerometers, Information obtained from a suspension sensor, etc.) may also be used. At the same time, in order to navigate to the destination, the autonomous vehicle will identify its position on a particular roadway (e.g., a particular lane on a multi-lane road) and navigate side-by-side with other vehicles. , avoid obstacles and pedestrians, obey traffic signals and signs, and travel from one road to another at appropriate intersections or interchanges. The use and interpretation of the vast amount of information that an autonomous vehicle collects as it drives to its destination presents many design challenges.”).
Regarding claim 15:
The combination of Hayat and Zhang have the limitations of claim 13:
Hayat further teaches:
the processed map comprises an updated map that is generated based on the sensor dataset, and the notification comprises a prompt inviting an acceptance of the processed map. (“The server may deliver updated models or updates to the vehicle to provide autonomous vehicle navigation.”).
Regarding claim 17:
The combination of Hayat and Zhang have the limitations of claim 16:
Hayat further teaches:
a transmitter configured to transmit a processed map to an autonomous vehicle. (“Further, in some embodiments, the redundancy and availability of received data is based on information received from one or more sensors (e.g., radar, LiDAR, acoustic sensors, one or more transmitters and receivers external to the vehicle). information received from the aircraft).”).
Regarding claim 18:
The combination of Hayat and Zhang have the limitations of claim 17:
Hayat further teaches:
wherein the at least one processor is located outside the autonomous vehicle. (“A processor (eg, processing unit 110) provided in vehicle 200 may receive the data contained in sparse map 800 from a remote server and may execute this data to guide automated driving of vehicle 200.”).
Regarding claim 19:
The combination of Hayat and Zhang have the limitations of claim 16:
Hayat further teaches:
wherein the at least one processor is configured to receive a sensor dataset of a road represented in the map, the sensor dataset being acquired by a sensor subsystem that is located at a different location than at least one of the at least one processor. Examiner broadly interprets this limitation to include that the processor and sensor may be located separately, … (“However, sparse map 800 need not be stored locally to the vehicle. In some embodiments, sparse map 800 may be stored in a storage device or computer readable medium provided on a remote server in communication with vehicle 200 or a device associated with vehicle 200 . A processor (eg, processing unit 110) provided in vehicle 200 may receive the data contained in sparse map 800 from a remote server and may execute this data to guide automated driving of vehicle 200. . In such embodiments, a remote server may store all of sparse map 800 or only a portion thereof. Accordingly, storage devices or computer readable media onboard vehicle 200 and/or onboard one or more additional vehicles may store the remaining portions of sparse map 800.”).
CONCLUSION
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see attached form 892.
Brown (US20220081003A1) - A lead autonomous vehicle (AV) includes a sensor configured to observe a field of view in front of the lead AV. Following AVs are on the same road behind the lead AV. A processor of the lead AV is configured to detect a construction zone. The processor sends a first message to an operation server, indicating that the construction zone is detected. The processor updates driving instructions of the lead AV to navigate around the construction zone. While navigating around the construction zone, the processor sends sensor data associated with the construction zone to the operation server. The operation server determines an extent of the construction zone. If the operation server determines that the construction zone is extensive, it sends re-routing instructions to the AVs. If the operation server determines that the construction zone is not extensive, it sends instructions to navigate around the construction zone to the AVs.
Luo (US10223807B) - A method of localization for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform utilizing one or more autonomous vehicle driving modules that execute processing of images from a camera and data from a LiDAR the following steps comprising: aligning a 3D submap with a global map; extracting features from the 3D submap and the global map; classifying the extracted features in classes; and establishing correspondence of features in a same class between the 3D submap and the global map.
Yan (US10762635B2) – A system and method for actively selecting and labeling images for semantic segmentation are disclosed. A particular embodiment includes: receiving image data from an image generating device; performing semantic segmentation or other object detection on the received image data to identify and label objects in the image data and produce semantic label image data; determining the quality of the semantic label image data based on prediction probabilities associated with regions or portions of the image; and identifying a region or portion of the image for manual labeling if an associated prediction probability is below a pre-determined threshold.
Malach (JP2022535351A) - A Systems and methods for vehicle navigation are provided. In one embodiment, the at least one processor may receive at least one captured image of the environment of the vehicle from the vehicle's camera. The processor may analyze the at least one image to identify road topological features within the vehicle's environment represented by the at least one image and at least one point associated with the at least one image. Based on the identified road topological features, the processor may determine an estimated route within the environment of the vehicle associated with the at least one point. The processor may further cause the vehicle to perform navigation operations based on the estimated route.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW COBB whose telephone number is (571) 272-3850. The examiner can normally be reached 9 - 5, M - F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to call examiner Cobb as above, or to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Nolan, can be reached at (571) 270-7016. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at (866) 217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/MATTHEW COBB/Examiner, Art Unit 3661
/PETER D NOLAN/Supervisory Patent Examiner, Art Unit 3661