Response to Amendment
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 11, 13, 15-16, 18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Efland et al. (US 20210406559 A1) in view of Kareev et al. (US 20180240336 A1).
In regard to claim 11, Efland teaches a method of managing traffic rules related to traffic enforcement (Efland, Fig. 2, the map 200 corresponds to given area of the real world and is made up of several different layers that each contain different information about the real-world environment), comprising: generating or updating a map layer (Efland, Fig. 2, Para. 60, the geometric map layer 202 may provide a representation of the real-world environment that is orders of magnitude more precise than the representation provided by the base map layer 201. For example, while the base map layer 201 may represent the location of the road network within the real-world environment at an approximately meter-level of precision, which is generally not sufficient to support autonomous vehicle operation, the geometric map layer 202 may be able to represent the location of the road network within the real-world environment at a centimeter-level of precision), using one or more processors of a server (Efland, Fig. 5, Para. 131, computing platform 500 may generally comprise any one or more computer systems (e.g., an on-board vehicle computing system and/or one or more off-board servers) that collectively include at least a processor 502, data storage 504, and a communication interface 506), based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices (Efland, Para. 68, beginning with the collection of sensor data for a given real-world environment, which could take various forms (e.g., image data, LiDAR data, GPS data, IMU data, etc.). The collected sensor data is fused together and processed in order to generate the geometric data for the high-resolution map), Wherein the map layer comprises semantic roadway geometries generated from the videos captured by the one or more edge devices (Efland, Para. 68, the processed sensor data may be analyzed using computer-vision and/or machine-learning techniques in order to automatically generate an initial set of semantic data for the high-resolution map, which may involve the use of one or more object detection models. Para. 74, the collected sensor data may include an indication of how well the captured sensor data in a given area compares to the semantic map layer. For instance, semantic elements identified from captured sensor data during operation may be analyzed for their correlation to the semantic map layer as a reference. This type of snapping may attempt to align lane boundaries, painted road markings, traffic signals, and the like with corresponding elements in the semantic map layer); generating or updating, using the one or more processors of the server, a traffic enforcement layer on top of the map layer, wherein a plurality of traffic rules associated with at least one semantic roadway geometry of the map layer are saved as part of the traffic enforcement layer (Efland, Para. 61, Building from the geometric map layer 202, the map 200 may further include a semantic map layer 203 that includes data objects for semantic elements that are found within the real-world environment (i.e., “semantic objects”), which may be embedded with semantic metadata indicating information about such semantic elements. For example, the semantic map layer 203 may include semantic objects for lane boundaries, crosswalks, parking spots, stop signs, traffic lights and the like, each of which includes semantic metadata that provides information about the classification of the semantic element, the location of the semantic element, and perhaps also additional contextual information about the semantic element that can be used by a vehicle to drive safely and effectively); generating or updating, using the one or more processors of the server, a traffic insight layer (Efland, Fig. 2); wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one of the traffic rule of the plurality of traffic rules based on a change in a traffic throughput or flow determined by the traffic insight layer (Efland, Para. 110, The computing system may also flag the given area of real-world environment 100 for re-evaluation and, at a later time, the given area may be re-evaluated. For example, vehicle 101 or another vehicle may capture sensor data including new 2D images of the given area that indicate that the previously detected traffic control elements are no longer present), and wherein adjusting or providing the suggestion to adjust the at least one of the traffic rule of the plurality of traffic rules further comprises not enforcing or providing a suggestion to not enforce the at least one traffic rule of the plurality traffic rules (Efland, Para. 110, In response, the computing platform may revert the previous updates that were made to the real-time layer 205 (e.g., by pushing a command to revert the previous updates) such that the map may return to its original state; enforcing turn restriction with the turn restriction signs 103, 104 are removed).
Efland also teaches transmitting the traffic enforcement layer from the server to at least one of the one or more edge devices; and evaluating, by the at least one edge device, subsequently captured videos using the traffic enforcement layer (Efland, Para. 76, the collected sensor data may be evaluated and, based on the evaluation, a change to a given area of the real-world environment may be detected. In this regard, block 302 may involve the evaluation of both raw sensor data as well as derived data that is based on the raw sensor data (e.g., a vectorized representation derived from the raw sensor data). Depending on the nature of the detected change, these operations may be performed on-vehicle, off-vehicle by a back-end computing platform that collects captured sensor data from a plurality of vehicles, or some combination of these. Further, the operations of evaluating the collected sensor data and detecting a change may take various forms, which may depend on the type of sensor data being evaluated and the type of evaluation being performed; Para. 77, these operations may involve detecting a new semantic element (e.g., a road barricade, a traffic signal, road sign, etc.) at a given area within the real-world environment that was not previously located at the given area).
Efland does not teach to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry.
However, Kareev teaches to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry (Kareev, Para. 21, The motoring and enforcement of traffic violations is based on video streams captured by the camera 110, by cameras installed in other traffic monitoring systems (not shown in FIG. 1) that are communicatively connected to the traffic monitoring system 100, or by a combination thereof. In an embodiment, the traffic monitoring system 100 is configured to analyze the data streams and, in particular, video streams, to detect complex traffic violations; Para. 55, includes comparing the identified driving pattern to a database providing illegal driving patterns. The illegal driving patterns are determined based on local traffic rules and the location of the vehicle. S460 may further include querying a database using the identified license plate to determine at least a classification of the vehicle and an eligibility of the vehicle to drive on the road).
Efland and Kareev are analogous art because they both pertain to traffic monitoring system.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to use updated map layer with local traffic rules (as taught by Efland) in determining traffic violation (as taught by Kareev) in order to monitor and detect occurrence of traffic violations.
In regard to claim 13, Combination of Efland and Kareev teach the method of claim 11, wherein each of the edge devices is coupled to a carrier vehicle and wherein at least part of the videos are captured while the carrier vehicle is in motion (Efland, Para. 70, the collected sensor data discussed herein may generally refer to sensor data captured by one or more sensor-equipped vehicles operating in the real-world environment and may take various forms).
In regard to claim 15, Combination of Efland and Kareev teach the method of claim 11, wherein the map layer is generated or updated by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network (Efland, Fig. 6, Para. 156, deriving the representation of the surrounding environment perceived by vehicle 600 using the raw data may involve detecting objects within the vehicle's surrounding environment, which may result in the determination of class labels, bounding boxes, or the like for each detected object. In this respect, the particular classes of objects that are detected by perception subsystem 602a (which may be referred to as “agents”) may take various forms, including both (i) “dynamic” objects that have the potential to move, such as vehicles, cyclists, pedestrians, and animals, among other examples, and (ii) “static” objects that generally do not have the potential to move, such as streets, curbs, lane markings, traffic lights, stop signs, and buildings, among other examples. Further, in practice, perception subsystem 602a may be configured to detect objects within the vehicle's surrounding environment using any type of object detection model now known or later developed, including but not limited object detection models based on convolutional neural networks (CNN)).
In regard to claim 16, Efland teaches a method of managing traffic rules related to traffic enforcement (Efland, Fig. 2, the map 200 corresponds to given area of the real world and is made up of several different layers that each contain different information about the real-world environment), comprising: generating or updating a map layer (Efland, Fig. 2, Para. 60, the geometric map layer 202 may provide a representation of the real-world environment that is orders of magnitude more precise than the representation provided by the base map layer 201. For example, while the base map layer 201 may represent the location of the road network within the real-world environment at an approximately meter-level of precision, which is generally not sufficient to support autonomous vehicle operation, the geometric map layer 202 may be able to represent the location of the road network within the real-world environment at a centimeter-level of precision), using one or more processors of a server (Efland, Fig. 5, Para. 131, computing platform 500 may generally comprise any one or more computer systems (e.g., an on-board vehicle computing system and/or one or more off-board servers) that collectively include at least a processor 502, data storage 504, and a communication interface 506), based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices (Efland, Para. 68, beginning with the collection of sensor data for a given real-world environment, which could take various forms (e.g., image data, LiDAR data, GPS data, IMU data, etc.). The collected sensor data is fused together and processed in order to generate the geometric data for the high-resolution map), Wherein the map layer comprises semantic roadway geometries generated from the videos captured by the one or more edge devices (Efland, Para. 68, the processed sensor data may be analyzed using computer-vision and/or machine-learning techniques in order to automatically generate an initial set of semantic data for the high-resolution map, which may involve the use of one or more object detection models. Para. 74, the collected sensor data may include an indication of how well the captured sensor data in a given area compares to the semantic map layer. For instance, semantic elements identified from captured sensor data during operation may be analyzed for their correlation to the semantic map layer as a reference. This type of snapping may attempt to align lane boundaries, painted road markings, traffic signals, and the like with corresponding elements in the semantic map layer), and wherein the map layer is generated or updated by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network (Efland, Fig. 6, Para. 156, deriving the representation of the surrounding environment perceived by vehicle 600 using the raw data may involve detecting objects within the vehicle's surrounding environment, which may result in the determination of class labels, bounding boxes, or the like for each detected object. In this respect, the particular classes of objects that are detected by perception subsystem 602a (which may be referred to as “agents”) may take various forms, including both (i) “dynamic” objects that have the potential to move, such as vehicles, cyclists, pedestrians, and animals, among other examples, and (ii) “static” objects that generally do not have the potential to move, such as streets, curbs, lane markings, traffic lights, stop signs, and buildings, among other examples. Further, in practice, perception subsystem 602a may be configured to detect objects within the vehicle's surrounding environment using any type of object detection model now known or later developed, including but not limited object detection models based on convolutional neural networks (CNN)); generating or updating, using the one or more processors of the server, a traffic enforcement layer on top of the map layer, wherein a plurality of traffic rules associated with at least one semantic roadway geometry of the map layer are saved as part of the traffic enforcement layer (Efland, Para. 61, Building from the geometric map layer 202, the map 200 may further include a semantic map layer 203 that includes data objects for semantic elements that are found within the real-world environment (i.e., “semantic objects”), which may be embedded with semantic metadata indicating information about such semantic elements. For example, the semantic map layer 203 may include semantic objects for lane boundaries, crosswalks, parking spots, stop signs, traffic lights and the like, each of which includes semantic metadata that provides information about the classification of the semantic element, the location of the semantic element, and perhaps also additional contextual information about the semantic element that can be used by a vehicle to drive safely and effectively); generating or updating, using the one or more processors of the server, a traffic insight layer, wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one traffic rule of the plurality of traffic rules of the traffic enforcement layer (Efland, Fig. 4A; Para. 109, the computing platform may effect updates to the real-time layer 203 by adding information for new semantic elements 103, 104, and 105, as depicted in the top-down view 115 showing a visualization of the updated map 115. In addition, the computing platform may update the real-time layer by adding an indication of the construction zone that is blocking traffic in the given lane, shown in the top-down view 115 as polygon 106) based in part on traffic violations or traffic conditions determined by the one or more edge devices or the server (Efland, Para. 106, In FIG. 4A, vehicle 101 may be a human-driven vehicle equipped with a sensor that captures image data, such as a monocular camera (e.g., a dashboard camera) that captures 2D image data. The 2D image data may include indications of the signs 103 and 104 and the barricade 105. Vehicle 101 may also capture GPS sensor data that may provide an approximation of the location of vehicle 101 within the given area of real-world environment 100).
Efland also teaches transmitting the traffic enforcement layer from the server to at least one of the one or more edge devices; and evaluating, by the at least one edge device, subsequently captured videos using the traffic enforcement layer (Efland, Para. 76, the collected sensor data may be evaluated and, based on the evaluation, a change to a given area of the real-world environment may be detected. In this regard, block 302 may involve the evaluation of both raw sensor data as well as derived data that is based on the raw sensor data (e.g., a vectorized representation derived from the raw sensor data). Depending on the nature of the detected change, these operations may be performed on-vehicle, off-vehicle by a back-end computing platform that collects captured sensor data from a plurality of vehicles, or some combination of these. Further, the operations of evaluating the collected sensor data and detecting a change may take various forms, which may depend on the type of sensor data being evaluated and the type of evaluation being performed; Para. 77, these operations may involve detecting a new semantic element (e.g., a road barricade, a traffic signal, road sign, etc.) at a given area within the real-world environment that was not previously located at the given area).
Efland does not teach to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry.
However, Kareev teaches to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry (Kareev, Para. 21, The motoring and enforcement of traffic violations is based on video streams captured by the camera 110, by cameras installed in other traffic monitoring systems (not shown in FIG. 1) that are communicatively connected to the traffic monitoring system 100, or by a combination thereof. In an embodiment, the traffic monitoring system 100 is configured to analyze the data streams and, in particular, video streams, to detect complex traffic violations; Para. 55, includes comparing the identified driving pattern to a database providing illegal driving patterns. The illegal driving patterns are determined based on local traffic rules and the location of the vehicle. S460 may further include querying a database using the identified license plate to determine at least a classification of the vehicle and an eligibility of the vehicle to drive on the road).
Efland and Kareev are analogous art because they both pertain to traffic monitoring system.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to use updated map layer with local traffic rules (as taught by Efland) in determining traffic violation (as taught by Kareev) in order to monitor and detect occurrence of traffic violations.
In regard to claim 18, the claim is interpreted and rejected for the same reasons as stated in the rejection of claim 13 as stated above.
In regard to claim 20, the claim is interpreted and rejected for the same reasons as stated in the rejection of claim 15 as stated above.
Claim(s) 1-5, 7-10, 12, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Efland et al. (US 20210406559 A1) in view of Kareev et al. (US 20180240336 A1) and further in view of Monaci et al. (US 20210264339 A1).
In regard to claim 1, Efland teaches a method of managing traffic rules related to traffic enforcement (Efland, Fig. 2, the map 200 corresponds to given area of the real world and is made up of several different layers that each contain different information about the real-world environment), comprising: generating or updating a map layer (Efland, Fig. 2, Para. 60, the geometric map layer 202 may provide a representation of the real-world environment that is orders of magnitude more precise than the representation provided by the base map layer 201. For example, while the base map layer 201 may represent the location of the road network within the real-world environment at an approximately meter-level of precision, which is generally not sufficient to support autonomous vehicle operation, the geometric map layer 202 may be able to represent the location of the road network within the real-world environment at a centimeter-level of precision), using one or more processors of a server (Efland, Fig. 5, Para. 131, computing platform 500 may generally comprise any one or more computer systems (e.g., an on-board vehicle computing system and/or one or more off-board servers) that collectively include at least a processor 502, data storage 504, and a communication interface 506), based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices (Efland, Para. 68, beginning with the collection of sensor data for a given real-world environment, which could take various forms (e.g., image data, LiDAR data, GPS data, IMU data, etc.). The collected sensor data is fused together and processed in order to generate the geometric data for the high-resolution map), Wherein the map layer comprises semantic roadway geometries generated from the videos captured by the one or more edge devices (Efland, Para. 68, the processed sensor data may be analyzed using computer-vision and/or machine-learning techniques in order to automatically generate an initial set of semantic data for the high-resolution map, which may involve the use of one or more object detection models. Para. 74, the collected sensor data may include an indication of how well the captured sensor data in a given area compares to the semantic map layer. For instance, semantic elements identified from captured sensor data during operation may be analyzed for their correlation to the semantic map layer as a reference. This type of snapping may attempt to align lane boundaries, painted road markings, traffic signals, and the like with corresponding elements in the semantic map layer); wherein the traffic rule is associated with at least one semantic roadway geometry of the map layer (Efland, Para. 61, (Efland, Para. 61, Building from the geometric map layer 202, the map 200 may further include a semantic map layer 203 that includes data objects for semantic elements that are found within the real-world environment (i.e., “semantic objects”), which may be embedded with semantic metadata indicating information about such semantic elements. For example, the semantic map layer 203 may include semantic objects for lane boundaries, crosswalks, parking spots, stop signs, traffic lights and the like); generating or updating, using the one or more processors of the server, a traffic enforcement layer on top of the map layer, wherein the traffic rule is saved as part of the traffic enforcement layer (Efland, Para. 61, Building from the geometric map layer 202, the map 200 may further include a semantic map layer 203 that includes data objects for semantic elements that are found within the real-world environment (i.e., “semantic objects”), which may be embedded with semantic metadata indicating information about such semantic elements. For example, the semantic map layer 203 may include semantic objects for lane boundaries, crosswalks, parking spots, stop signs, traffic lights and the like, each of which includes semantic metadata that provides information about the classification of the semantic element, the location of the semantic element, and perhaps also additional contextual information about the semantic element that can be used by a vehicle to drive safely and effectively).
Efland also teaches transmitting the traffic enforcement layer from the server to at least one of the one or more edge devices; and evaluating, by the at least one edge device, subsequently captured videos using the traffic enforcement layer (Efland, Para. 76, the collected sensor data may be evaluated and, based on the evaluation, a change to a given area of the real-world environment may be detected. In this regard, block 302 may involve the evaluation of both raw sensor data as well as derived data that is based on the raw sensor data (e.g., a vectorized representation derived from the raw sensor data). Depending on the nature of the detected change, these operations may be performed on-vehicle, off-vehicle by a back-end computing platform that collects captured sensor data from a plurality of vehicles, or some combination of these. Further, the operations of evaluating the collected sensor data and detecting a change may take various forms, which may depend on the type of sensor data being evaluated and the type of evaluation being performed; Para. 77, these operations may involve detecting a new semantic element (e.g., a road barricade, a traffic signal, road sign, etc.) at a given area within the real-world environment that was not previously located at the given area).
Efland does not teach to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry.
However, Kareev teaches to determine whether one or more vehicles violate the traffic rule associated with the at least one semantic roadway geometry (Kareev, Para. 21, The motoring and enforcement of traffic violations is based on video streams captured by the camera 110, by cameras installed in other traffic monitoring systems (not shown in FIG. 1) that are communicatively connected to the traffic monitoring system 100, or by a combination thereof. In an embodiment, the traffic monitoring system 100 is configured to analyze the data streams and, in particular, video streams, to detect complex traffic violations; Para. 55, includes comparing the identified driving pattern to a database providing illegal driving patterns. The illegal driving patterns are determined based on local traffic rules and the location of the vehicle. S460 may further include querying a database using the identified license plate to determine at least a classification of the vehicle and an eligibility of the vehicle to drive on the road).
Efland and Kareev are analogous art because they both pertain to traffic monitoring system.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to use updated map layer with local traffic rules (as taught by Efland) in determining traffic violation (as taught by Kareev) in order to monitor and detect occurrence of traffic violations.
Efland teaches this initial set of semantic data may then undergo a human curation/validation stage during which human curators review and update the initial set of semantic data in order to ensure that it has a sufficient level of accuracy for use in a high-definition map (e.g., position information that is accurate at a centimeter-level) (Para. 68).
Combination of Efland and Kareev do not specifically teach receiving, at the server, a traffic rule via a user dragging and dropping the traffic rule onto a roadway shown on an interactive map editor user interface.
However, Monaci teaches receiving, at the server, a traffic rule via a user dragging and dropping the traffic rule onto a roadway shown on an interactive map editor user interface (Monaci, Para. 89, Interactive editing for the schematic maps 140, 142 can include, for instance, receiving edit commands (selection, dragging, controls, or any other suitable interface command) (not shown) to select, move, enlarge, shrink, add, delete, customize (e.g., change line or shape color or configuration), label, etc. one or more components of the schematic map 140, 142).
Efland, Kareev and Monaci are analogous art because they all pertain to generating map based on learned traffic situations.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have editing capabilities to generate and modify maps in real time (as taught by Monaci) in order to ensure that map data has a sufficient level of accuracy for use in a semantic map.
In regard to claim 2, Combination of Efland, Kareev and Monaci teach the method of claim 1, wherein the traffic rules comprises at least one of a rule type, a rule attribute, and a rule logic (Monaci, Para. 89, Interactive editing for the schematic maps 140, 142 can include, for instance, receiving edit commands (selection, dragging, controls, or any other suitable interface command) (not shown) to select, move, enlarge, shrink, add, delete, customize (e.g., change line or shape color or configuration), label, etc. one or more components of the schematic map 140, 142).
In regard to claim 3, Combination of Efland, Kareev and Monaci teach the method of claim 2, further comprising receiving the traffic rules in response to the user dragging and dropping at least one of the rule type, the rule attribute, and the rule logic onto a route point displayed over the roadway (Monaci, Fig. 4A; Para. 90; Para. 106, a user such as a transportation network operator can use the schedule editor module to edit spatial information, such as by creating, configuring, or updating one or more routes along transportation lines and their transport stops. Further, the user (or a different user) can edit the temporal information to reflect updated information. Spatial and temporal information can be edited in some embodiments using different interfaces, and the edited information provided via one interface can synchronized with the information viewed on the other interface).
In regard to claim 4, Combination of Efland, Kareev and Monaci teach the method of claim 1, further comprising generating or updating a traffic insight layer, wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one of the traffic rules of the traffic enforcement layer (Efland, Fig. 4A; Para. 109, the computing platform may effect updates to the real-time layer 203 by adding information for new semantic elements 103, 104, and 105, as depicted in the top-down view 115 showing a visualization of the updated map 115. In addition, the computing platform may update the real-time layer by adding an indication of the construction zone that is blocking traffic in the given lane, shown in the top-down view 115 as polygon 106) based in part on traffic violations or traffic conditions determined by the one or more edge devices or the server (Efland, Para. 106, In FIG. 4A, vehicle 101 may be a human-driven vehicle equipped with a sensor that captures image data, such as a monocular camera (e.g., a dashboard camera) that captures 2D image data. The 2D image data may include indications of the signs 103 and 104 and the barricade 105. Vehicle 101 may also capture GPS sensor data that may provide an approximation of the location of vehicle 101 within the given area of real-world environment 100).
In regard to claim 5, Combination of Efland, Kareev, and Monaci teach the method of claim 4, wherein the traffic insight layer is further configured to adjust or provide the suggestion to adjust one of the traffic rules based on a change in a traffic throughput or flow determined by the traffic insight layer (Efland, Para. 110, The computing system may also flag the given area of real-world environment 100 for re-evaluation and, at a later time, the given area may be re-evaluated. For example, vehicle 101 or another vehicle may capture sensor data including new 2D images of the given area that indicate that the previously detected traffic control elements are no longer present), and wherein adjusting or providing the suggestion to adjust one of the traffic rules further comprises not enforcing or providing a suggestion to not enforce one of the traffic rules based on the change in the traffic throughput or flow (Efland, Para. 110, In response, the computing platform may revert the previous updates that were made to the real-time layer 205 (e.g., by pushing a command to revert the previous updates) such that the map may return to its original state; enforcing turn restriction with the turn restriction signs 103, 104 are removed).
In regard to claim 7, Combination of Efland, Kareev, and Monaci teach the method of claim 1, wherein updating the map layer further comprises receiving a semantic annotation via user inputs applied to the interactive map editor user interface (Monaci, Para. 135, Different and more granular ways of dynamically rendering the map 230 at different time instances can additionally or alternatively be used. Nonlimiting examples include: modifying colors; modifying line thickness or configuration (e.g., dashed, dotted, stippled, etc.); modifying one or more shapes; modifying transport stop markers; modifying labels for transport stops and/or transportation lines; creating movement in one or more indications (e.g., causing one or more lines or transport stops to repeatedly blink, shrink, enlarge, or fade); and/or extracting and regenerating completely different network visualizations for different times of the day).
In regard to claim 8, Combination of Efland, Kareev, and Monaci teach the method of claim 1, wherein generating or updating the traffic enforcement layer further comprises converting raw traffic rule data into the traffic rule (Efland, Para. 76-77, In this regard, block 302 may involve the evaluation of both raw sensor data as well as derived data that is based on the raw sensor data (e.g., a vectorized representation derived from the raw sensor data). Depending on the nature of the detected change, these operations may be performed on-vehicle, off-vehicle by a back-end computing platform that collects captured sensor data from a plurality of vehicles, or some combination of these. Further, the operations of evaluating the collected sensor data and detecting a change may take various forms, which may depend on the type of sensor data being evaluated and the type of evaluation being performed. these operations may involve detecting a new semantic element (e.g., a road barricade, a traffic signal, road sign, etc.) at a given area within the real-world environment that was not previously located at the given area).
In regard to claim 9, Combination of Efland, Kareev, and Monaci teach the method of claim 1, wherein each of the edge devices is coupled to a carrier vehicle and wherein at least part of the videos are captured while the carrier vehicle is in motion (Efland, Para. 70, the collected sensor data discussed herein may generally refer to sensor data captured by one or more sensor-equipped vehicles operating in the real-world environment and may take various forms).
In regard to claim 10, Combination of Efland, Kareev, and Monaci teach the method of claim 1, wherein the map layer is generated or updated by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network (Efland, Fig. 6, Para. 156, deriving the representation of the surrounding environment perceived by vehicle 600 using the raw data may involve detecting objects within the vehicle's surrounding environment, which may result in the determination of class labels, bounding boxes, or the like for each detected object. In this respect, the particular classes of objects that are detected by perception subsystem 602a (which may be referred to as “agents”) may take various forms, including both (i) “dynamic” objects that have the potential to move, such as vehicles, cyclists, pedestrians, and animals, among other examples, and (ii) “static” objects that generally do not have the potential to move, such as streets, curbs, lane markings, traffic lights, stop signs, and buildings, among other examples. Further, in practice, perception subsystem 602a may be configured to detect objects within the vehicle's surrounding environment using any type of object detection model now known or later developed, including but not limited object detection models based on convolutional neural networks (CNN)).
In regard to claim 12, Combination of Efland and Kareev do not teach the method of claim 11, wherein generating or updating the traffic enforcement layer further comprises the server receiving at least some of the traffic rules via user inputs applied to an interactive map editor user interface.
However, Monaci teaches wherein generating or updating the traffic enforcement layer further comprises the server receiving at least some of the traffic rules via user inputs applied to an interactive map editor user interface (Monaci, Para. 135, Different and more granular ways of dynamically rendering the map 230 at different time instances can additionally or alternatively be used. Nonlimiting examples include: modifying colors; modifying line thickness or configuration (e.g., dashed, dotted, stippled, etc.); modifying one or more shapes; modifying transport stop markers; modifying labels for transport stops and/or transportation lines; creating movement in one or more indications (e.g., causing one or more lines or transport stops to repeatedly blink, shrink, enlarge, or fade); and/or extracting and regenerating completely different network visualizations for different times of the day).
Efland, Kareev, and Monaci are analogous art because they all pertain to generating map based on learned traffic situations.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have editing capabilities to generate and modify maps in real time (as taught by Monaci) in order to ensure that map data has a sufficient level of accuracy for use in a semantic map.
In regard to claim 17, the claim is interpreted and rejected for the same reasons as stated in the rejection of claim 12 as stated above.
Claim 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Efland et al. (US 20210406559 A1) in view of Kareev et al. (US 20180240336 A1) and Monaci et al. (US 20210264339 A1) and further in view of Dorne et a. (US 20190385453 A1).
In regard to claim 6, Combination of Efland, Kareev, and Monaci do not teach the method of claim 4, wherein generating or updating the traffic insight layer further comprises generating a heatmap of traffic violations detected by the one or more edge devices.
However, Dorne teaches comprises generating a heatmap of traffic violations detected by the one or more edge devices (Dorne, Para. 64, a user may view locations of events associated with law enforcement activities, traffic incidents (such as parking tickets, moving violations, and the like), parking information (such as in public parking lot, meter information), License Plate Number camera locations, and/or the like (such as emergency calls, and the like); Para. 71, the user may select and/or hover a cursor over any grouping area on the map interface to cause the map system to update the interactive heatmap to reflect the number of events within the selected grouping area. Also, the user may select and/or hover a cursor over an intersection in the interactive heatmap to cause the map system to update the grouping numbers in the map interface. For example, when the user selects the grouping 250 (indicating a grouping of 16 events) in the search area 118 in the map interface, the map system will update the interactive heatmap to reflect only the License Plate reads associated with the 16 instances and the search result list 202 will be updated as well. When the user selects the intersection 510 in the interactive heatmap, the map system will update the map interface to reflect only the groupings having events that are part of the selected time period).
Efland, Kareev, Monaci, and Dorne are analogous art because they all pertain to interactive traffic information mapping system.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have information regarding law enforcement events on the interactive vehicle information mapping system (as taught by Dorne) in order to allow for rapid and deep searching, retrieval, and/or analysis of various vehicle-related data, objects, features, and/or metadata by the user.
Claim 14 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Efland et al. (US 20210406559 A1) in view of Kareev et al. (US 20180240336 A1) and further in view of Dorne et a. (US 20190385453 A1).
In regard to claim 14, Combination of Efland and Kareev do not teach the method of claim 11, wherein generating or updating the traffic insight layer further comprises generating a heatmap of traffic violations detected by the one or more edge devices.
However, Dorne teaches comprises generating a heatmap of traffic violations detected by the one or more edge devices (Dorne, Para. 64, a user may view locations of events associated with law enforcement activities, traffic incidents (such as parking tickets, moving violations, and the like), parking information (such as in public parking lot, meter information), License Plate Number camera locations, and/or the like (such as emergency calls, and the like); Para. 71, the user may select and/or hover a cursor over any grouping area on the map interface to cause the map system to update the interactive heatmap to reflect the number of events within the selected grouping area. Also, the user may select and/or hover a cursor over an intersection in the interactive heatmap to cause the map system to update the grouping numbers in the map interface. For example, when the user selects the grouping 250 (indicating a grouping of 16 events) in the search area 118 in the map interface, the map system will update the interactive heatmap to reflect only the License Plate reads associated with the 16 instances and the search result list 202 will be updated as well. When the user selects the intersection 510 in the interactive heatmap, the map system will update the map interface to reflect only the groupings having events that are part of the selected time period).
Efland, Kareev, and Dorne are analogous art because they all pertain to interactive traffic information mapping system.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have information regarding law enforcement events on the interactive vehicle information mapping system (as taught by Dorne) in order to allow for rapid and deep searching, retrieval, and/or analysis of various vehicle-related data, objects, features, and/or metadata by the user.
In regard to claim 19, the claim is interpreted and rejected for the same reasons as stated in the rejection of claim 14 as stated above.
Response to Arguments
Response to amended claims is considered above in claim Rejections.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHARMIN AKHTER whose telephone number is (571)272-9365. The examiner can normally be reached on Monday - Thursday 8:00am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Davetta W Goins can be reached on (571) 272.2957. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHARMIN AKHTER/
Examiner, Art Unit 2689
Prosecution Insights