DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments to the specification and claim 20 have overcame all previously held objections. Claims 1-20 remain pending. Claims 1, 8, 15, and 20 have been amended.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-9, 12-16 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cohen (KR 102359806 B1) in view of Throsby (JP 2022539245 A) and Hosoya (JP 2022182094 A).
With respect to claim 1, Cohen teaches a method for detecting a portion of an environment of a vehicle, comprising: generating, using one or more sensors coupled to a vehicle (“…sensing block included in the vehicle” page 7 paragraph 7 lines 1-2 and “Various other sensors may be included in the sensing block, and may rely on some or all of the sensors to build an understanding of the vehicle's driving condition. In addition to cameras (front, side, rear, etc.), other sensors such as radar, lidar, and acoustic sensors may be included in the detection block.” Page 8 paragraph 1 lines 1-3), environmental data from an environment of the vehicle (“Additionally, the sensing block may include one or more components configured to communicate and transmit/receive information regarding the vehicle's surrounding environment.” Page 8 paragraph 1 lines 3-5), wherein the environmental data comprises one or more of the following: ground LiDAR data from the environment (“lidar” page 8 paragraph 1 line 3), camera data from the environment (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), and path data corresponding to a change in position of one or more other vehicles within the environment (“In operation 580 , the processing unit 110 may determine driving information about the lead vehicle (eg, the vehicle driving ahead of the vehicle 200 ). For example, the processing unit 110 may determine the position, speed (eg, direction and speed), and/or acceleration of the lead vehicle by using the method described with reference to FIGS. 5A and 5B above. Processing unit 110 may also utilize one or more road polynomials, lookout points (lookpoints associated with vehicle 200), and/or a snail trail utilizing the method described with reference to FIG. 5E above.” Page 22 paragraph 5 lines 2-7);
Cohen does not teach inputting the environmental data into a machine learning model trained to generate a heatmap; and using a processor: based on the environmental data, determining a portion of the environment, wherein the portion of the environment comprises an area having a likelihood, greater than a minimum threshold, of being adjacent to one or more pavement markings, and generating the heatmap, wherein the heatmap corresponds to the portion of the environment.
Throsby teaches inputting the environmental data into a machine learning model trained to generate a heatmap (“In some examples, a machine learning model can output a heatmap associated with predicted probabilities” page 4 paragraph 6 lines 7-8); and using a processor (“One or more processors” page 9 paragraph 5 line 1): based on the environmental data (“receiving sensor data of an environment captured by sensors of an autonomous vehicle into the system when instructions are executed… and generating a multi-channel image representing a top-down view of the environment based at least in part on the sensor data… inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 1-10), determining a portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5), being adjacent to one or more pavement markings (“The action data is used to correspond to one or more channels of a multi-channel image representing the first vehicle's target lane, trajectory, etc. ) can be generated.” Page 2 paragraph 4 lines 9-10 And “…inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 8-10), and generating the heatmap, wherein the heatmap corresponds to the portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5).
Hosoya teaches wherein the portion of the environment comprises an area having a likelihood, greater than a minimum threshold (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold…” page 8 paragraph 2 lines 1-2), of being adjacent to one or more pavement markings (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold, the lane marking recognition unit 132 recognizes the line segment as a lane marking.” page 8 paragraph 2 lines 1-3).
Throsby is analogous art in the same field of endeavor as the claimed invention. Throsby is directed towards sensor coupled vehicles that gather environmental data (“Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of the environment, which can include a second vehicle or objects away from the vehicle such as pedestrians.” Page 2 paragraph 4 lines 2-4). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, with the expectation that doing so would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6).
Hosoya is analogous art in the same field of endeavor as the claimed invention. Hosoya is directed towards control program and system for a vehicle with external sensors (“The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , a navigation device 50 , an MPU (Map Positioning Unit) 60, a driving operator 80 , an automatic driving control device 100 , a driving force output device 200, a braking device 210 , and a steering device 220 .” page 2 paragraph 5 lines 2-6). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps, with the expectation that doing so would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17).
With respect for claim 2, Cohen, Throsby, and Hosoya teach the method of claim 1. Cohen further teaches the method of claim 1, wherein the one or more sensors comprise one or more of the following: one or more LiDAR systems (“…sensor system 206 includes lidar sensors…” page 13 paragraph 3 line 1), one or more cameras (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), and one or more RADAR systems (“…sensor system 206 includes … radar sensors…” page 13 paragraph 3 line 1).
With respect to claim 5, Cohen Throsby, and Hosoya teach the method of claim 1. Cohen further teaches the method of claim 1, wherein: the one or more sensors comprise one or more cameras (“For example, a camera cluster…a variable number of cameras…” page 15 paragraph 2 line 11) configured to generate one or more images (“…each cluster capable of capturing images from a specific area around the vehicle towards a specific direction.” Page 15 paragraph 2 lines 14-15), and the camera data comprises one or more images (“…each cluster capable of capturing images from a specific area around the vehicle towards a specific direction.” Page 15 paragraph 2 lines 14-15).
With respect to claim 6, Cohen, Throsby, and Hosoya teach the method of claim 1. Throsby further teaches the method of claim 1, wherein: the processor is configured to run the machine learning model (“In some examples, the machine learning model can output coordinates associated with the object and probability information associated with each coordinate.” Page 4 paragraph 6 lines 3-5), and the machine learning model comprises a neural network (“In some examples, the machine learning model may include a convolutional neural network (CNN), which may include one or more recurrent neural network (RNN) layers, such as long short-term memory (LSTM) layers.” Page 4 paragraph 6 lines 5-7).
With respect to claim 7, Cohen, Throsby and Hosoya teach the method of claim 1. Throsby further teaches the method of claim 1, wherein generating path data corresponding to a change in position of the one or more other vehicle in the environment comprises: using the processor (“…processor…” page 9 paragraph 5 line 1): identifying, using image recognition (“object detection, segmentation, and/or classification. In some examples, the perceptual component 222 detects the presence of entities in proximity to the vehicle 202 and/or entity types (e.g., cars, pedestrians, bicycles, animals, buildings, trees, road surfaces, curbs, sidewalks, unknown, etc.)” page 9 paragraph 7 lines 1-4), a first position of one of the one or more other vehicles at a first time (“Example 126 represents a first multi-channel image …first channel 132 may represent a bounding box, position, extent (eg, length and width), etc. of autonomous vehicle 106 and/or object 108 in the environment.” Page 7 paragraph 3 lines 1-4); identifying, using image recognition, a second position of the one of the one or more other vehicles at a second time (“Example 128 represents a second multi-channel image associated with second candidate action 118 . In some examples, some aspects of example 128 may be equal to some aspects of example 126. For example, example 128 can include first channel 132” page 7 paragraph 4 lines 1-3), wherein the second time is after the first time (“As can be appreciated, examples 126 and 128 can include multiple multi-channel images representing the environment at various points in time within the environment. For example, examples 126 and/or 128 may represent the history of autonomous vehicle 106 and object 108 (and other objects such as pedestrians and vehicles) over the past 4 seconds at 0.5 second intervals, although any Instances of numbers and periods can be used to represent the environment” page 8 paragraph 2); determining a change in position between the first position and the second position (“For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 6-10, motion information); and generating a visual representation of the change in position (“Sensor data and any data based on sensor data can be represented in a top-down view of the environment. For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 5-10).
With respect to claim 8, Cohen teaches a vehicle (“…sensing block included in the vehicle” page 7 paragraph 7 line 2), and an imaging module, coupled to the vehicle (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), the imaging module comprising: one or more cameras (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), configured to capture an image depicting an environment within view of the one or more cameras (“For example, a camera cluster…a variable number of cameras…” page 15 paragraph 2 line 11 and “…each cluster capable of capturing images from a specific area around the vehicle towards a specific direction.” Page 15 paragraph 2 lines 14-15), and a processor (“The processing unit 110 may include one or more processing devices. In some embodiments, the processing unit 110 may include an application processor 180 , an image processor 190 , or other suitable processing device” page 4 paragraph 5 lines 5-7), configured to: generate, using one or more sensors coupled to a vehicle (“…sensing block included in the vehicle” page 7 paragraph 7 lines 1-2 and “Various other sensors may be included in the sensing block, and may rely on some or all of the sensors to build an understanding of the vehicle's driving condition. In addition to cameras (front, side, rear, etc.), other sensors such as radar, lidar, and acoustic sensors may be included in the detection block.” Page 8 paragraph 1 lines 1-3), environmental data from an environment of the vehicle (“Additionally, the sensing block may include one or more components configured to communicate and transmit/receive information regarding the vehicle's surrounding environment.” Page 8 paragraph 1 lines 3-5), wherein the environmental data comprises one or more of the following: ground LiDAR data from the environment (“lidar” page 8 paragraph 1 line 3), camera data from the environment (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), and path data corresponding to a change in position of one or more other vehicles within the environment (“In operation 580 , the processing unit 110 may determine driving information about the lead vehicle (eg, the vehicle driving ahead of the vehicle 200 ). For example, the processing unit 110 may determine the position, speed (eg, direction and speed), and/or acceleration of the lead vehicle by using the method described with reference to FIGS. 5A and 5B above. Processing unit 110 may also utilize one or more road polynomials, lookout points (lookpoints associated with vehicle 200), and/or a snail trail utilizing the method described with reference to FIG. 5E above.” Page 22 paragraph 5 lines 2-7);
Throsby teaches a processor (“One or more processors” page 9 paragraph 5 line 1), configured to: input the environmental data into a machine learning model trained to generate a heatmap (“In some examples, a machine learning model can output a heatmap associated with predicted probabilities” page 4 paragraph 6 lines 7-8), based on the environmental data (“receiving sensor data of an environment captured by sensors of an autonomous vehicle into the system when instructions are executed… and generating a multi-channel image representing a top-down view of the environment based at least in part on the sensor data… inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 1-10), determining a portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5), being adjacent to one or more pavement markings (“The action data is used to correspond to one or more channels of a multi-channel image representing the first vehicle's target lane, trajectory, etc. ) can be generated.” Page 2 paragraph 4 lines 9-10 And “…inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 8-10), and generating the heatmap, wherein the heatmap corresponds to the portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5).
Hosoya teaches wherein the portion of the environment comprises an area having a likelihood, greater than a minimum threshold (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold…” page 8 paragraph 2 lines 1-2), of being adjacent to one or more pavement markings (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold, the lane marking recognition unit 132 recognizes the line segment as a lane marking.” page 8 paragraph 2 lines 1-3).
Throsby is analogous art in the same field of endeavor as the claimed invention. Throsby is directed towards sensor coupled vehicles that gather environmental data (“Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of the environment, which can include a second vehicle or objects away from the vehicle such as pedestrians.” Page 2 paragraph 4 lines 2-4). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, with the expectation that doing so would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6).
Hosoya is analogous art in the same field of endeavor as the claimed invention. Hosoya is directed towards control program and system for a vehicle with external sensors (“The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , a navigation device 50 , an MPU (Map Positioning Unit) 60, a driving operator 80 , an automatic driving control device 100 , a driving force output device 200, a braking device 210 , and a steering device 220 .” page 2 paragraph 5 lines 2-6). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps, with the expectation that doing so would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17).
With respect to claim 9, Cohen, Throsby, and Hosoya teach the system of claim 8. Cohen teaches the system of claim 8, wherein the one or more sensors comprise one or more of the following: one or more LiDAR systems (“…sensor system 206 includes lidar sensors…” page 13 paragraph 3 line 1), one or more cameras (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), and one or more RADAR systems (“…sensor system 206 includes … radar sensors…” page 13 paragraph 3 line 1)
With respect to claim 12, Cohen, Throsby, and Hosoyo teach the system of claim 8. Throsby further teaches the system of claim 8, wherein: the one or more sensors comprise one or more cameras (“Sensor system 206 may include multiple instances of each of these or other types of sensors. For example, lidar sensors may include individual lidar sensors located at the corners, front, rear, sides, and/or top of vehicle 202 . As another example, camera sensors may include multiple cameras positioned at various locations around the exterior and/or interior of vehicle 202 .” page 13 paragraph 3 lines 5-9) configured to generate one or more birds-eye-view images (“Multi-channel images (which may be referred to simply as images throughout) that encode various parameters of objects and/or environments in a top-down view can be generated based on sensor data, map data, and/or action data.” Page 2 paragraph 4 lines 4-6), and the camera data comprises one or more birds-eye-view images (“Sensor system 206 may include multiple instances of each of these or other types of sensors. For example, lidar sensors may include individual lidar sensors located at the corners, front, rear, sides, and/or top of vehicle 202 . As another example, camera sensors may include multiple cameras positioned at various locations around the exterior and/or interior of vehicle 202 .” page 13 paragraph 3 lines 5-9 and “Sensor data and any data based on sensor data can be represented in a top-down view of the environment” page 3 paragraph 5 lines 5-6).
With respect to claim 13, Cohen, Throsby and Hosoya teach the system of claim 8. Throsby further teaches the system of claim 8, wherein: the processor is configured to run the machine learning model (“In some examples, the machine learning model can output coordinates associated with the object and probability information associated with each coordinate.” Page 4 paragraph 6 lines 3-5), and the machine learning model comprises a neural network (“In some examples, the machine learning model may include a convolutional neural network (CNN), which may include one or more recurrent neural network (RNN) layers, such as long short-term memory (LSTM) layers.” Page 4 paragraph 6 lines 5-7).
With respect to claim 14, Cohen, Throsby and Hosoya teach the system of claim 8. Throsby further teaches the system of claim 8, wherein generating path data corresponding to a change in position of the one or more other vehicle in the environment comprises: using the processor (“…processor…” page 9 paragraph 5 line 1): identifying, using image recognition (“object detection, segmentation, and/or classification. In some examples, the perceptual component 222 detects the presence of entities in proximity to the vehicle 202 and/or entity types (e.g., cars, pedestrians, bicycles, animals, buildings, trees, road surfaces, curbs, sidewalks, unknown, etc.)” page 9 paragraph 7 lines 1-4), a first position of one of the one or more other vehicles at a first time (“Example 126 represents a first multi-channel image …first channel 132 may represent a bounding box, position, extent (eg, length and width), etc. of autonomous vehicle 106 and/or object 108 in the environment.” Page 7 paragraph 3 lines 1-4); identifying, using image recognition, a second position of the one of the one or more other vehicles at a second time (“Example 128 represents a second multi-channel image associated with second candidate action 118 . In some examples, some aspects of example 128 may be equal to some aspects of example 126. For example, example 128 can include first channel 132” page 7 paragraph 4 lines 1-3), wherein the second time is after the first time (“As can be appreciated, examples 126 and 128 can include multiple multi-channel images representing the environment at various points in time within the environment. For example, examples 126 and/or 128 may represent the history of autonomous vehicle 106 and object 108 (and other objects such as pedestrians and vehicles) over the past 4 seconds at 0.5 second intervals, although any Instances of numbers and periods can be used to represent the environment” page 8 paragraph 2); determining a change in position between the first position and the second position (“For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 6-10, motion information); and generating a visual representation of the change in position (“Sensor data and any data based on sensor data can be represented in a top-down view of the environment. For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 5-10).
With respect to claim 15, Cohen teaches an imaging device comprising one or more cameras (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), the imaging device coupled to a vehicle (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), wherein the one or more cameras are configured to capture an image depicting an environment within view of the one or more cameras (“…image capturing apparatuses 122 , 124 , and 126 may include cameras having different fields of view and focal lengths. The field of view of the image capturing apparatuses 122 , 124 , and 126 may include a necessary area related to the surroundings of the vehicle 200” page 13 paragraph 4 lines 1-4); and a computing device (“System 100…” page 4 paragraph 5 line 1), including a processor (“System 100 may include various components depending on the requirements of its implementation. In some embodiments, the system 100 includes a processing unit…” page 4 paragraph 5 lines 1-3) and a memory (“System 100 may include various components depending on the requirements of its implementation. In some embodiments, the system 100 includes … one or more memory units 140 …” page 4 paragraph 5 lines 1-4), coupled to the vehicle (“system 100 may be included in vehicle 200 , as illustrated in FIG. 1A” page 8 paragraph 2 line 2), configured to store programming instructions (“configuring the processing processing device may include storing executable instructions in a memory accessible to the processing device during operation” page 6 paragraph 1 lines 1-2) that, when executed by the processor, cause the processor to: generate, using one or more sensors coupled to a vehicle (“…sensing block included in the vehicle” page 7 paragraph 7 lines 1-2 and “Various other sensors may be included in the sensing block, and may rely on some or all of the sensors to build an understanding of the vehicle's driving condition. In addition to cameras (front, side, rear, etc.), other sensors such as radar, lidar, and acoustic sensors may be included in the detection block.” Page 8 paragraph 1 lines 1-3), environmental data from an environment of the vehicle (“Additionally, the sensing block may include one or more components configured to communicate and transmit/receive information regarding the vehicle's surrounding environment.” Page 8 paragraph 1 lines 3-5), wherein the environmental data comprises one or more of the following: ground LiDAR data from the environment (“lidar” page 8 paragraph 1 line 3), camera data from the environment (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2), and path data corresponding to a change in position of one or more other vehicles within the environment (“In operation 580 , the processing unit 110 may determine driving information about the lead vehicle (eg, the vehicle driving ahead of the vehicle 200 ). For example, the processing unit 110 may determine the position, speed (eg, direction and speed), and/or acceleration of the lead vehicle by using the method described with reference to FIGS. 5A and 5B above. Processing unit 110 may also utilize one or more road polynomials, lookout points (lookpoints associated with vehicle 200), and/or a snail trail utilizing the method described with reference to FIG. 5E above.” Page 22 paragraph 5 lines 2-7);
Cohen does not teach programming instructions that, when executed by the processor, cause the processor to: input the environmental data into a machine learning model trained to generate a heatmap; based on the environmental data, determine a portion of the environment, wherein the portion of the environment comprises an area having a likelihood, greater than a minimum threshold, of being adjacent to one or more pavement markings; and generate the heatmap, wherein the heatmap corresponds to the portion of the environment.
Throsby teaches programming instructions (“Processor 216 of vehicle 202 and processor 244 of computing device 242 may be any suitable processor capable of executing instructions to process data and perform operations as described herein.” Page 15 paragraph 5 lines 1-2) that, when executed by a processor (“One or more processors” page 9 paragraph 5 line 1), cause the processor to: input the environmental data into a machine learning model trained to generate a heatmap (“In some examples, a machine learning model can output a heatmap associated with predicted probabilities” page 4 paragraph 6 lines 7-8); based on the environmental data (“receiving sensor data of an environment captured by sensors of an autonomous vehicle into the system when instructions are executed… and generating a multi-channel image representing a top-down view of the environment based at least in part on the sensor data… inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 1-10), determining a portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5), being adjacent to one or more pavement markings (“The action data is used to correspond to one or more channels of a multi-channel image representing the first vehicle's target lane, trajectory, etc. ) can be generated.” Page 2 paragraph 4 lines 9-10 And “…inputting a channel image into a machine learning model trained to generate a heatmap containing predicted probabilities of possible locations associated with the vehicle” page 22 paragraph 6 lines 8-10), and generating the heatmap, wherein the heatmap corresponds to the portion of the environment (“A heatmap can represent an individualized region of the environment proximate to the autonomous vehicle. For example, the heatmap may represent a 64x64 grid (or a JxK size grid) representing a 100 meter by 100 meter area around the autonomous vehicle. Of course, the heatmap can represent any size region and can represent any number of discrete portions of the region. In some cases, a portion of the heatmap can be referred to as a heatmap cell” page 4 paragraph 7 lines 1-5).
Hosoya teaches wherein a portion of a environment comprises an area having a likelihood, greater than a minimum threshold (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold…” page 8 paragraph 2 lines 1-2), of being adjacent to one or more pavement markings (“Further, based on the edge extraction result, if there is an edge line segment (for example, a straight line or a curved line) whose length is equal to or greater than the first threshold, the lane marking recognition unit 132 recognizes the line segment as a lane marking.” page 8 paragraph 2 lines 1-3).
Throsby is analogous art in the same field of endeavor as the claimed invention. Throsby is directed towards sensor coupled vehicles that gather environmental data (“Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of the environment, which can include a second vehicle or objects away from the vehicle such as pedestrians.” Page 2 paragraph 4 lines 2-4). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen and Throsby by using the environmental data gather by Cohen, as input into the machine learning heatmap generator of Throsby, with the expectation that doing so would lead to improvements in decision making (pathing, navigation, etc.) based on enhanced complex reasoning capabilities created by understanding the behavior and intent of objects in the environment around the sensing vehicle (“Autonomous driving in dense urban environments is challenging because of the complex reasoning often used to resolve multidirectional interactions between objects. This reasoning can be time sensitive and constantly evolving. The techniques described herein are intended for driving scenarios, which may include, but are not limited to, urban intersections without traffic lights. At these junctions, multiple objects (vehicles, pedestrians, cyclists, etc.) often compete for the same shared space, so predicting object intent is useful for successfully navigating intersections.” Throsby Description of Embodiments paragraph 2 lines 1-6).
Hosoya is analogous art in the same field of endeavor as the claimed invention. Hosoya is directed towards control program and system for a vehicle with external sensors (“The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , a navigation device 50 , an MPU (Map Positioning Unit) 60, a driving operator 80 , an automatic driving control device 100 , a driving force output device 200, a braking device 210 , and a steering device 220 .” page 2 paragraph 5 lines 2-6). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby and Hosoya by using Hosoya’s lane detection strategy to determine the probability of lane markings within Throsby’s heatmaps, with the expectation that doing so would lead to improved lane detection by using recognition processes that can “…prevent lane markings from being erroneously recognized” (page 11 paragraph 2 lines 16-17).
With respect to claim 16, Cohen, Throsby, and Hosoya teach the system of claim 15. Cohen further teaches the system of claim 15, wherein the one or more sensors comprise one or more of the following: one or more LiDAR systems (“…sensor system 206 includes lidar sensors…” page 13 paragraph 3 line 1); one or more cameras (“One or more cameras (eg, the image capture devices 122, 124, 126) may be part of a sensing block included in the vehicle.” Page 7 paragraph 7 lines 1-2); and one or more RADAR systems (“…sensor system 206 includes … radar sensors…” page 13 paragraph 3 line 1).
With respect to claim 19, Cohen, Throsby, and Hosoya teach the system of claim 15. Throsby further teaches the system of claim 15, wherein: the processor is configured to run the machine learning model (“In some examples, the machine learning model can output coordinates associated with the object and probability information associated with each coordinate.” Page 4 paragraph 6 lines 3-5), and the machine learning model comprises a neural network (“In some examples, the machine learning model may include a convolutional neural network (CNN), which may include one or more recurrent neural network (RNN) layers, such as long short-term memory (LSTM) layers.” Page 4 paragraph 6 lines 5-7)
With respect to claim 20, Cohen, Throsby and Hosoya teach the system of claim 15. Throsby further teaches the system of claim 15, wherein, in the generating path data corresponding to a change in position of the one or more other vehicle in the environment, the programming instructions, when executed by the processor, are further configured to cause the processor (“…processor…” page 9 paragraph 5 line 1) to: identify, using image recognition (“object detection, segmentation, and/or classification. In some examples, the perceptual component 222 detects the presence of entities in proximity to the vehicle 202 and/or entity types (e.g., cars, pedestrians, bicycles, animals, buildings, trees, road surfaces, curbs, sidewalks, unknown, etc.)” page 9 paragraph 7 lines 1-4), a first position of one of the one or more other vehicles at a first time (“Example 126 represents a first multi-channel image …first channel 132 may represent a bounding box, position, extent (eg, length and width), etc. of autonomous vehicle 106 and/or object 108 in the environment.” Page 7 paragraph 3 lines 1-4); identify, using image recognition, a second position of the one of the one or more other vehicles at a second time (“Example 128 represents a second multi-channel image associated with second candidate action 118 . In some examples, some aspects of example 128 may be equal to some aspects of example 126. For example, example 128 can include first channel 132” page 7 paragraph 4 lines 1-3), wherein the second time is after the first time (“As can be appreciated, examples 126 and 128 can include multiple multi-channel images representing the environment at various points in time within the environment. For example, examples 126 and/or 128 may represent the history of autonomous vehicle 106 and object 108 (and other objects such as pedestrians and vehicles) over the past 4 seconds at 0.5 second intervals, although any Instances of numbers and periods can be used to represent the environment” page 8 paragraph 2); determine a change in position between the first position and the second position (“For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 6-10, motion information); and generate a visual representation of the change in position (“Sensor data and any data based on sensor data can be represented in a top-down view of the environment. For example, an image may represent an object as a two-dimensional bounding box representing the position of the object within the environment, as well as the object's extent (e.g., object length and width) and object classification (e.g., vehicle, pedestrian, etc.). can be represented. Motion information, such as velocity information, can be represented as a velocity vector associated with a bounding box, although other representations are envisioned.” Page 3 paragraph 5 lines 5-10).
Claims 3-4, 10-11, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Cohen, Throsby and Hosoya as applied to claims 1, 8, and 15 above, respectfully, and further in view of Migishima (JP 2021528628 A).
With respect to claim 3, Cohen, Throsby, and Hosoya teach the method of claim 1, but do not teach the method of claim 1, wherein the ground LiDAR data comprises a 2-dimensional grouping of data points within the environment.
Migishima teaches wherein ground LiDAR data comprises a 2-dimensional grouping of data points (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4) within the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
With respect to claim 4, Cohen, Throsby, and Hosoya teach the method of claim 1, but do not teach the method of claim 1, wherein generating ground LiDAR data comprises: capturing, using one or more LiDAR systems, 3-dimensional LiDAR data from the environment; and distilling the 3-dimensional LiDAR data to the ground lidar data.
Migishima teaches wherein generating ground LiDAR data comprises: capturing, using one or more LiDAR systems (“Further, although drawn as a single sensor for exemplary purposes, any number of image capture devices (s) 102, lidar sensors (s) 104, and / or other sensors 106 (s) are expected.” Page 5 paragraph 1 lines 16-18), 3-dimensional LiDAR data (“The measured value of the lidar sensor 104 can be represented as three-dimensional lidar sensor data having coordinates (eg, Cartesian coordinates, polar coordinates, etc.)…” page 5 paragraph 4 lines 10-11) from the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5); and distilling the 3-dimensional LiDAR data to the ground lidar data (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
With respect to claim 10, Cohen, Throsby, and Hosoya teach the system of claim 8, but do not teach the system of claim 8, wherein the ground LiDAR data comprises a 2-dimensional grouping of data points within the environment.
Migishima teaches wherein ground LiDAR data comprises a 2-dimensional grouping of data points (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4) within the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
With respect to claim 11, Cohen, Throsby, and Hosoya teach the system of claim 8, but do not teach the system of claim 8, wherein generating ground LiDAR data comprises: capturing, using one or more LiDAR systems, 3-dimensional LiDAR data from the environment; and distilling the 3-dimensional LiDAR data to the ground lidar data.
Migishima teaches wherein generating ground LiDAR data comprises: capturing, using one or more LiDAR systems (“Further, although drawn as a single sensor for exemplary purposes, any number of image capture devices (s) 102, lidar sensors (s) 104, and / or other sensors 106 (s) are expected.” Page 5 paragraph 1 lines 16-18), 3-dimensional LiDAR data (“The measured value of the lidar sensor 104 can be represented as three-dimensional lidar sensor data having coordinates (eg, Cartesian coordinates, polar coordinates, etc.)…” page 5 paragraph 4 lines 10-11) from the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5); and distilling the 3-dimensional LiDAR data to the ground lidar data (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
With respect to claim 17, Cohen, Throsby, and Hosoya teach the system of claim 15, but do not teach the system of claim 15, wherein the ground LiDAR data comprises a 2-dimensional grouping of data points within the environment.
Migishima teaches wherein ground LiDAR data comprises a 2-dimensional grouping of data points (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4) within the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
With respect to claim 18, Cohen, Throsby, and Hosoya teach the system of claim 15. Cohen further teaches programming instructions executed by the processor (“configuring the processing processing device may include storing executable instructions in a memory accessible to the processing device during operation” page 6 paragraph 1 lines 1-2), but does not teach the system of claim 15, wherein, in the generating ground LiDAR data, the programming instructions, when executed by the processor, are further configured to cause the processor to: capture, using one or more LiDAR systems, 3-dimensional LiDAR data from the environment; and distill the 3-dimensional LiDAR data to the ground lidar data.
Migishima teaches wherein generating ground LiDAR data comprises: capturing, using one or more LiDAR systems (“Further, although drawn as a single sensor for exemplary purposes, any number of image capture devices (s) 102, lidar sensors (s) 104, and / or other sensors 106 (s) are expected.” Page 5 paragraph 1 lines 16-18), 3-dimensional LiDAR data (“The measured value of the lidar sensor 104 can be represented as three-dimensional lidar sensor data having coordinates (eg, Cartesian coordinates, polar coordinates, etc.)…” page 5 paragraph 4 lines 10-11) from the environment (“The lidar sensor 104 is mounted such that one or more lasers rotate (eg, around a substantially vertical axis), thereby capturing, for example, the lidar sensor data 112 associated with the environment 108.” Page 5 paragraph 4 lines 3-5); and distilling the 3-dimensional LiDAR data to the ground lidar data (“In some examples, the conversion operation can be used to convert 3D lidar sensor data into multi-channel 2D data” page 6 paragraph 1 lines 3-4).
Migishima is analogous art in the same field of endeavor as the claimed invention. Migishima is directed towards sensor data gathered by vehicles for decision making (“As mentioned above, autonomous vehicles and other machines rely on multiple sensors that provide input to cognitive systems that detect objects in the environment surrounding the autonomous vehicle or machine.” Page 2 paragraph 6 lines 1-2 And “This application uses data generated by individual sensor modalities (eg, image data, lidar (light detection and ranging) sensor data, lidar (light detection and ranging) data, SONAR (sound navigation and ranging) data, etc.). Then identify the error associated with the individual sensor modality by identifying the group of each object and compare each group of objects (eg processed sensor data) with the output of the cognitive system.” Page 2 paragraph 7 lines 1-5). A person of ordinary skill in the art before the effective filing date of the claimed invention could have reasoned that combining Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27). Therefore, it would have been obvious for a person of ordinary skill before the effective filing date of the claimed invention to combine Cohen, Throsby, Hosoya and Migishima by utilizing Migishima’s sensing technique and accompanying ML model with Cohen’s sensors, with the expectation that doing so would lead to improvements in accuracy and performance of the system and behavior of the vehicle (“Providing the above information to the machine learning system helps to improve the accuracy of the training data utilized by the above system, thereby further enhancing the performance of the system and / or the behavior of the vehicle in general.” Page 4 paragraph 3 lines 24-27).
Response to Arguments
Applicant's arguments filed 10/14/2025 have been fully considered but they are not persuasive.
Applicant argues on page 12, “While analyzing markings, Hosoya is silent as to any method of analyzing portions of a vehicle environment of being adjacent to lane markings, let alone determining a position of a vehicle environment comprising "an area having a likelihood, greater than a minimum threshold, of being adjacent to one or more pavement markings," as recited in the present claims.”. The examiner disagrees. Hosoya analyses the area around a car spatially, identifies edges, and then based on a threshold value determines if these edges are lane markings. This is well within the BRI of the current claim language. As mapped (see above claim mapping) the limitation is taught by Hosoya, with the edges being viewed as adjacent to the lane markings themselves. On pages 11 and 12 the applicant argues that both Cohen and Throsby, taken singularly, or in combination with Hosoya , do not teach the limitation “determining a portion of the environment, wherein the portion of the environment comprises an area having a likelihood, greater than a minimum threshold, of being adjacent to one or more pavement markings” the examiner agrees that when taken singularly, Cohen and Throsby do not teach this limitation, but finds this argument moot primarily because these references were never used to teach this limitation. Additionally, Hosoya does teach the limitation (see above), therefore the examiner cannot agree that Cohen and Throsby taken in combination with Hosoya does not teach the stated limitation. Therefore, the examiner maintains this rejection.
Because the rejection has been maintained applicant’s argument that the dependent claims 2-7, 9-14, and 16-19 should be allowable due to their dependance is also moot and associated rejections have been maintained.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REBECCA C WILLIAMS whose telephone number is (571)272-7074. The examiner can normally be reached M-F 7:30am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew W Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REBECCA COLETTE WILLIAMS/Examiner, Art Unit 2677
/ANDREW W BEE/ Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights