DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is in response to Applicant’s case, no. 18/930,828, with an effective filing date of 10/29/2024. Claims 1-3 and 5-21 are currently pending. Claim 4 has been canceled. Claim 21 has been added by the Applicant.
Response to Arguments
Examiner acknowledges that the necessary changes were made regarding the Specification regarding the minor informalities and the abstract in Applicant’s arguments, see pg. 12, and subsequently withdraws those respective objections. However, the Applicant did not address the numerous improper trade marks and names used throughout the disclosure. Therefore, this objection is hereby maintained.
Examiner acknowledges the cancellation of claim 4 in Applicant’s arguments, see pp.12-13, which renders the objection moot.
Regarding the 35 USC § 103 rejection of claims 1-3, 5, 8, 10, 12-13, 15, 17, and 19-20 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1) [hereinafter referred to as Chen] in view of Weiland et al. (US Pat. No. 8,892,356 B1) [hereinafter referred to as Weiland], the Applicant has elected to amend the aforementioned independent claims. Therefore, the Examiner’s rejection in the previous Office Action based on 35 USC § 103 is rendered moot. However, due to said amendments, new references Han et al. (US Pat. No. 11,738,770 B2) [hereinafter referred to as Han] and Moustafa et al. (US Pat. Pub. No. 2022/0126864 A1) [hereinafter referred to as Moustafa] have been necessitated, which upon closer examination fully replaces Tang et al. (US Pat. Pub. No. 2022/0343917 A1) [hereinafter referred to as Tang]. As such, Tang is no longer required to address any limitation of the claims. Therefore, a new rejection based on 35 USC § 103 has been made and is discussed in detail below.
Regarding claim 1, the Applicant argues, pp.13-16, that Chen, as modified by Weiland, does not disclose or teach the limitation obtaining one or more machine learning models comprising ground-truth associations between traffic control signals and lane segments, the ground-truth associations generated from non-visual data based at least on one or more rules associated with positioning of traffic control devices relative to corresponding lane segments. However, Han teaches in column (col) 28 line(s) 51-67 that the HD map system explicitly generating a lane element graph in which individual lane elements are semantically associated with the traffic restrictions and controls that apply to them (i.e., traffic signals and signs, speed limits, etc.). These associations are not derived from visual perception alone; they are “ground-truth” semantic links that are encoded in the map’s lane element graph structure itself based upon legal navigability required for path planning.
Therefore, this argument is moot.
Furthermore, the Applicant argues that Chen, as modified by Weiland, does not disclose or teach the limitation one or more machine learning models trained at least partially using one or more synthetically generated training datasets. However, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets.
Therefore, this argument is moot.
Regarding independent claims 8 and 18, Applicant argues, while differing in scope, these claims recite similar features to claim 1 and their rejections should likewise be withdrawn.
However, this argument is unpersuasive for the same reasons as given above.
Applicant argues the dependent claims are patentable by virtue of their dependency.
This argument is unpersuasive as each independent claim has been fully rejected for the reasons as given above.
Specification
The following terms, which are trade names or marks used in commerce, have been noted in this application repeatedly in numerous locations. The terms are:
NVIDIA trade names such as NVLink and NV Switch (e.g.,[00187] ln 10), PhysX SDK (e.g., [0038] ln 11), or DriveSIM (e.g.,[0037] ln 2);
Bluetooth, Bluetooth Low Energy, Z-Wave, Zigbe, LoRaWAN, Sigfox, Wi-Fi, (e.g.,[0093] ln 8-9);
Foveon X3 (e.g.,[0095] ln 9);
FlexRay (e.g.,[00105] ln 2);
PyTorch and TensorFlow (e.g.,[00215] ln 6-7); and
PCIe (e.g.,[00187] ln 6).
These terms should be accompanied by their generic terminology; furthermore the term should be capitalized wherever they appear (emphasis added) or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1-3, 5-6, 9, 17, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, Moustafa et al. (US Pat. Pub. No. 2022/0126864 A1), hereinafter referred to as Moustafa, and Weiland et al. (US Pat. No. 8,892,356 B1), hereinafter referred to as Weiland.
Regarding claim 1, Chen discloses:
A method comprising:
applying, to the one or more machine learning models, input data ([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks) including at least:
first feature data representative of at least one of first geometric information or first semantic information corresponding to a plurality of lane segments ([0016] s.5-7, autonomous system’s perception system may detect its current lane parameters, construed as geometric information, and that of adjacent lanes (e.g., oncoming traffic) including that of direction to assess risk/cost and perform vehicle operations based on object avoidance); and
second feature data representative of at least one of second geometric information or second semantic information corresponding to a plurality of traffic control signals ([0019] object avoidance decisions are based on semantic classes of information including traffic signal information and geometric characteristics of the class, the position of the object (e.g., height of the object, distance of the object from the ground surface, shape of the object, etc.), [0045] machine learning models and/or neural networks perform classification of the semantic information, and [0084] communication connection(s) may allow the vehicle to communicate with other nearby computing device(s) (e.g., traffic signals)); and
causing a machine to perform one or more operations based at least on the associating ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle),
but Chen does not disclose:
obtaining one or more machine learning models comprising ground-truth associations between traffic control signals and lane segments, the ground-truth associations generated from non-visual data based at least on one or more rules associated with positioning of traffic control devices relative to corresponding lane segments.
However, Han teaches in column (col) 28 line(s) 51-67 that the HD map system may generate a lane element graph that represents a network of lanes to allow a vehicle to plan a legal path between a source and a destination. A lane element graph may allow navigation of autonomous vehicles through a mapped area. Each lane element may be associated with the traffic restrictions that apply to it such as speed limit, speed bump, and traffic signs and signals. A lane element graph may represent the navigable road surface that is divided into lane elements, and may include connectivity among lane elements (e.g., where vehicles can go from a current lane element) as well as semantic association between lane elements and features (e.g., a speed limit in a current lane element) to assist in on-vehicle routing and planning needs. The lane elements may be topologically connected, and each lane element may be known to its successors, predecessors, and left and right neighbors. This is construed as explicitly generating a lane element graph in which individual lane elements are semantically associated with the traffic restrictions and controls that apply to them (i.e., traffic signals and signs, speed limits, etc.). These associations are not derived from visual perception alone; they are “ground-truth” semantic links that are encoded in the map’s lane element graph structure itself based upon legal navigability required for path planning. Furthermore, see Fig. 22 below which discloses derived attributes that are integrated into the lane element graph.
[AltContent: rect]
PNG
media_image1.png
613
852
media_image1.png
Greyscale
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, by incorporating the semantic lane associating teachings of Han, such that the combination would provide for the predictable result of improving autonomous decision making based upon associated traffic parameters.
However, Chen, as modified by Han, do not explicitly disclose:
one or more machine learning models trained at least partially using one or more synthetically generated training datasets.
However, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the synthetic data trained machine learning model teachings of Moustafa, such that the combination would provide for the predictable result of, as acknowledged by Moustafa in [0439] s.9, improving training accessibility by intelligently controlling the synthetic data creation, such that the system may create images (for training) that would otherwise require a very long time for a vehicle to encounter in real life.
However, Chen, as modified by Han and Moustafa, do not explicitly disclose:
computing, using the one or more machine learning models and based at least on the input data, a plurality of confidence scores indicative of whether one or more active signals of the plurality of traffic control signals correspond to one or more lane segments of the plurality of lane segments; and
associating, based at least on the plurality of confidence scores, the one or more active signals with the one or more lane segments.
However, Weiland teaches in column (col) 8 lines (ln) 50-62 of confidence indications for vehicles entering an intersection in a specific lane corresponding to traffic signals that indicates the likelihood that the geometry of the maneuver accurately predicts or represents a vehicle’s path. Furthermore, in claim 6 of Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection. This is construed as a computing model for predicting based on a plurality of confidence scores, an association between one or more lane segments and one or more active traffic signals. Lastly, see Fig 18 below regarding active signal for one or more lane segments.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the lane associating based on confidence level teachings of Weiland, such that the combination would provide for the predictable result of improving autonomous decision making without the intervention of a human driver.
PNG
media_image2.png
294
388
media_image2.png
Greyscale
Figure 18
Regarding claim 2, Chen, as modified by Han, Moustafa, and Weiland, discloses:
The method of claim 1, wherein the first semantic information corresponding to the plurality of lane segments includes at least one or more directions associated with the plurality of lane segments (see claim 1 [0016] s.5-7 where direction of traffic is discerned by the system in its cost analysis of object avoidance).
Regarding claim 3, Chen, as modified by Han, Moustafa, and Weiland, discloses:
The method of claim 1, wherein the one or more machine learning models include one or more deep neural networks (see claim 1 regarding machine learning and/or neural networks, [0045] s.7 the semantic class estimation may include segmenting and/or classifying extracted deep convolutional features into semantic data (e.g., rigidity, hardness, safety risk, risk of position change, class or type, potential direction of travel, etc.), and [0091] deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN)).
Regarding claim 4, Applicant has elected to cancel the claim and, therefore, the limitations are no longer under consideration.
Regarding claim 5, Chen, as modified by Han, Moustafa, and Weiland, discloses:
The method of claim 1, wherein the associating of the one or more active signals with the one or more lane segments comprises, at least:
associating one or more first active signals with one or more first lane segments having one or more first directions (see claim 1 regarding oncoming traffic which differentiates one direction from another); and
associating one or more second active signals with one or more second lane segments having one or more second directions (see claim 1 as stated above and regarding different entry lanes having different exit lanes, which necessarily could be a different direction (e.g., a turning lane)).
Regarding claim 6, Chen, as modified by Han, Moustafa, and Weiland, discloses:
The method of claim 1, wherein the first feature data and the second feature data include non-image features (Han teaches, as discussed in claim 1, in col 28 ln 51-67 non-visual, rule-based ground truth associations).
Regarding claim 9, Chen, as modified by Han and Weiland, discloses The system of claim 8 as discussed above, they do not explicitly disclose:
synthetically generate the ground truth data from non-visual data using the one or more rules.
However, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the synthetic data trained machine learning model teachings of Moustafa, such that the combination would provide for the predictable result of, as acknowledged by Moustafa in [0439] s.9, improving training accessibility by intelligently controlling the synthetic data creation, such that the system may create images (for training) that would otherwise require a very long time for a vehicle to encounter in real life.
Regarding claim 17, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle and [0002] use in autonomous vehicles);
a perception system for an autonomous or semi-autonomous machine ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle and [0017] s.4, perception pipelines and sensor systems);
a system for performing one or more digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets ([0050] determine a bounding box or region associated with the object and determine if the bounding box intersects with a corridor (e.g., a three-dimensional representation of the planned path));
a system for performing one or more deep learning operations (([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks; [0045] s.7 the semantic class estimation may include segmenting and/or classifying extracted deep convolutional features into semantic data (e.g., rigidity, hardness, safety risk, risk of position change, class or type, potential direction of travel, etc.); and [0091] deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN)));
a system implemented using an edge device;
a system implemented using a robot ([0078] s.2, use within a robotic system);
a system for performing one or more generative AI operations ([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks which machine learning is a known subset of AI operations);
a system for performing operations using a large language model;
a system for performing operations using one or more vision language models (VLMs);
a system for performing operations using one or more multi-modal language models;
a system for using or deploying one or more inference microservices;
a system for performing one or more conversational AI operations;
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center.
However, Chen does not explicitly disclose:
a system for performing one or more simulation operations; and
a system implemented at least partially using cloud computing resources.
However, Han is further relied upon to teach in col 5 ln 34-67 cloud-based service that may allow clients such as a vehicle computing system 120 ( e.g., vehicle computing systems 120a-d) to make requests for information and services. Furthermore, Han teaches in col 47 ln 3-5 can also be applied for displaying maps for purposes of computer simulation.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by further incorporating the simulation and cloud teachings of Han, such that as the semantic lane associating teachings of Han are considered within Chen, the simulation and cloud teachings are also considered.
However, Chen, as already modified by Han and Weiland, do not explicitly disclose:
a system for generating synthetic data.
However, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the synthetic data trained machine learning model teachings of Moustafa, such that the combination would provide for the predictable result of, as acknowledged by Moustafa in [0439] s.9, improving training accessibility by intelligently controlling the synthetic data creation, such that the system may create images (for training) that would otherwise require a very long time for a vehicle to encounter in real life.
Regarding claim 21, although Chen, as modified by Han, Moustafa, and Weiland, discloses the method of claim 1, Chen does not explicitly disclose:
wherein the one or more rules include a priority order for assigning active signals to lane segments based at least on signal state and lane-segment direction.
However, Han is further relied upon to teach in col 38 ln 28-33 intersection related work using the tool can be further divided into several dependent steps such as, for example, sign association, connectivity (tum lines), tum lane geometry, bulb association, and priority. Based upon the Applicant’s description in [0060] of the instant application where a priority order may include, but is not limited to (e.g., other combinations or orders are possible), (1) lights for road users other than vehicles do not associate with any lane segment; (2) lights not facing ego road (from orientation) or not at current intersection (from 3D geometry); (3) if U-turn arrow bulb is present, it controls the U-turn lanes only; (4) if left arrow bulb is present, it controls the left turn lanes; and if U-turn bulb is not present, it also controls the left U-turn lanes; (5) if right arrow bulb is present, it controls the right turn lanes; (6) if solid circle bulb is present, it controls the remaining lanes; (7) if straight arrow bulb is present, it controls the straight lanes. This is interpreted as including rules of priority for assigning active signals to lane segments based on the signal state and lane direction.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by further incorporating the sign association, bulb association, and priority teachings of Han, such that as the semantic lane associating teachings of Han are considered within Chen, the sign association, bulb association, and priority teachings are also considered.
_____________________________________________
Claims 8, 10-13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, and Weiland et al. (US Pat. No. 8,892,356 B1), hereinafter referred to as Weiland.
Regarding claim 8, Chen discloses:
A system comprising: one or more processors ([0034] s. 3, one or more processors) to:
apply, to one or more machine learning models ([0038] s. 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks), at least:
one or more first features corresponding to a plurality of lanes ([0016] s.5-7, autonomous system’s perception system may detect its current lane parameters, construed as geometric information, and that of adjacent lanes (e.g., oncoming traffic) including that of direction to assess risk/cost and perform vehicle operations based on object avoidance); and
one or more second features corresponding to a plurality of traffic control devices ([0019] object avoidance decisions are based on semantic classes of information including traffic signal information and geometric characteristics of the class, the position of the object (e.g., height of the object, distance of the object from the ground surface, shape of the object, etc.), [0045] machine learning models and/or neural networks perform classification of the semantic information, and [0084] communication connection(s) may allow the vehicle to communicate with other nearby computing device(s) (e.g., traffic signals)); and
cause a machine to perform one or more control operations based at least on the association ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle).
However, Chen does not explicitly disclose:
obtain one or more machine learning models trained using ground-truth information indicating associations between traffic control signals and lanes of a driving surface, the ground-truth information generated based at least on one or more rules associated with positioning of traffic control devices relative to corresponding lanes.
However, Han teaches in column (col) 28 line(s) 51-67 that the HD map system may generate a lane element graph that represents a network of lanes to allow a vehicle to plan a legal path between a source and a destination. A lane element graph may allow navigation of autonomous vehicles through a mapped area. Each lane element may be associated with the traffic restrictions that apply to it such as speed limit, speed bump, and traffic signs and signals. A lane element graph may represent the navigable road surface that is divided into lane elements, and may include connectivity among lane elements (e.g., where vehicles can go from a current lane element) as well as semantic association between lane elements and features (e.g., a speed limit in a current lane element) to assist in on-vehicle routing and planning needs. The lane elements may be topologically connected, and each lane element may be known to its successors, predecessors, and left and right neighbors. This is construed as explicitly generating a lane element graph in which individual lane elements are semantically associated with the traffic restrictions and controls that apply to them (i.e., traffic signals and signs, speed limits, etc.). These associations are not derived from visual perception alone; they are “ground-truth” semantic links that are encoded in the map’s lane element graph structure itself based upon legal navigability required for path planning.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, by incorporating the semantic lane associating teachings of Han, such that the combination would provide for the predictable result of improving autonomous decision making based upon associated traffic parameters.
However, Chen, as modified by Han, does not explicitly disclose:
associate, based at least on the one or more machine learning models processing the one or more first features and the one or more second features, at least a traffic control device of the plurality of traffic control devices with at least a lane of the plurality of lanes.
However, Weiland teaches in column (col) 8 lines (ln) 50-62 of confidence indications for vehicles entering an intersection in a specific lane corresponding to traffic signals that indicates the likelihood that the geometry of the maneuver accurately predicts or represents a vehicle’s path. Furthermore, in claim 6 Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection. This is construed as a computing model for predicting based on a plurality of confidence scores, an association between one or more lane segments and one or more active traffic signals. Lastly, see Fig 18 below regarding active signal for one or more lane segments.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the lane associating based on confidence level teachings of Weiland, such that the combination would provide for the predictable result of improving autonomous decision making without the intervention of a human driver.
Regarding claim 10, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the one or more machine learning models include one or more deep neural networks (DNNs) ([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks; [0045] s.7 the semantic class estimation may include segmenting and/or classifying extracted deep convolutional features into semantic data (e.g., rigidity, hardness, safety risk, risk of position change, class or type, potential direction of travel, etc.); and [0091] deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN)).
Regarding claim 11, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the one or more first features are one or more first non-image features and the one or more second features are one or more second non-image features (Han teaches, as discussed in claim 1, in col 28 ln 51-67 non-visual, rule-based ground truth associations).
Regarding claim 12, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the one or more first features are indicative of at least one of:
one or more geometries associated with the plurality of lanes ([0016] s.5-7, autonomous system’s perception system may detect its current lane parameters, construed as geometric information, and that of adjacent lanes (e.g., oncoming traffic) including that of direction to assess risk/cost and perform vehicle operations based on object avoidance); or
one or more directions associated with the plurality of lanes ([0016] s.5-7, autonomous system’s perception system may detect its current lane parameters, construed as geometric information, and that of adjacent lanes (e.g., oncoming traffic) including that of direction to assess risk/cost and perform vehicle operations based on object avoidance).
Regarding claim 13, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the one or more second features are indicative of at least one of:
one or more geometries associated with the plurality of traffic control devices ([0019] object avoidance decisions are based on semantic classes of information including traffic signal information and geometric characteristics of the class, the position of the object (e.g., height of the object, distance of the object from the ground surface, shape of the object, etc.), [0045] machine learning models and/or neural networks perform classification of the semantic information, and [0084] communication connection(s) may allow the vehicle to communicate with other nearby computing device(s) (e.g., traffic signals));
one or more road user classifications associated with the plurality of traffic control devices ([0019] object avoidance decisions are based on semantic classes of information including traffic signal information and geometric characteristics of the class, the position of the object (e.g., height of the object, distance of the object from the ground surface, shape of the object, etc.), [0045] machine learning models and/or neural networks perform classification of the semantic information, and [0084] communication connection(s) may allow the vehicle to communicate with other nearby computing device(s) (e.g., traffic signals)); or
one or more states associated with the plurality of traffic control devices.
Regarding claim 15, Chen, as modified by Han and Weiland, discloses:
The system of claim 8, wherein the association of the traffic control device with the lane comprises associating, based at least on the one or more machine learning models processing the one or more first features and the one or more second features (see claim 1), one or more illuminated signals of the traffic control device with one or more directions associated with the lane (see Fig. 18 above where one or more illuminated signals of a traffic control device with one or more directions associated with a respective lane).
________________________________________
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, Moustafa et al. (US Pat. Pub. No. 2022/0126864 A1), hereinafter referred to as Moustafa, and Weiland et al. (US Pat. No. 8,892,356 B1), hereinafter referred to as Weiland, and Baalke et al. (US Pat. Pub. No. 2019/0324459 A1), hereinafter referred to as Baalke.
Regarding claim 7, Chen, as modified by Han, Moustafa, and Weiland, discloses:
The method of claim 1, further comprising:
comparing a first confidence score associated with a first pairing between a first active signal and a first lane segment with one or more second confidence scores associated with one or more second pairings between one or more second active signals and one or more second lane segments (claim 6 of Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection);
determining, based at least on the comparing, that the first confidence score is greater than the one or more second confidence scores by more than a threshold (Weiland teaches in col 8 ln 50-62 of confidence indications for vehicles entering an intersection in a specific lane corresponding to traffic signals that indicates the likelihood that the geometry of the maneuver accurately predicts or represents a vehicle’s path; in claim 6 of Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection, which is construed as a computing model for predicting based on a plurality of confidence scores, an association between one or more lane segments and one or more active traffic signals; and see Fig 18 regarding active signal for one or more lane segments); and
associating, as a valid pair, the first active signal and the first lane segment (see Fig 18 regarding active signal for one or more lane segments),
but Chen, as modified by Han, Moustafa and Weiland, does not explicitly disclose:
based at least on the first confidence score being greater than the one or more second confidence scores by more than the threshold.
However, Baalke teaches in [0008] s.2, that an autonomous vehicle may formulate a first and second confidence score and when a confidence score exceeds the other as well as a threshold score it performs an operation of autonomously navigating the current lane.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the lane associating based on confidence level teachings of Weiland, by incorporating the confidence score comparison teachings of Tang, as taught in [0074] s.3, such that it allows for improved confidence e performance of the vehicle on a specific path.
______________________________________
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, Weiland et al. (US Pat. No. 8,892,356 B1), hereinafter referred to as Weiland, and Baalke et al. (US Pat. Pub. No. 2019/0324459 A1), hereinafter referred to as Baalke.
Regarding claim 14, Chen, as modified by Weiland and Baalke, discloses:
The system of claim 8, the one or more processors further to:
compute, using the one or more machine learning models and based at least on the one or more first features and the one or more second features, a plurality of scores indicative of whether the traffic control device corresponds to the lane or one or more second lanes of the plurality of lanes (claim 6 of Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection); and
determine that the traffic control device corresponds to the lane (Weiland teaches in col 8 ln 50-62 of confidence indications for vehicles entering an intersection in a specific lane corresponding to traffic signals that indicates the likelihood that the geometry of the maneuver accurately predicts or represents a vehicle’s path; in claim 6 of Weiland teaches associating a plurality of maneuvers with one of the plurality of traffic signals, wherein the plurality of maneuvers describe multiple geometric paths within an intersection from an incoming lane to a plurality of outgoing lanes, wherein the plurality of maneuvers provide routing data for the navigation system, wherein at least one of the plurality of maneuvers is associated with a confidence value indicative of a probability that a vehicle traveling from the incoming lane into the intersection to one of the plurality of outgoing lanes follows a specified path through the intersection, which is construed as a computing model for predicting based on a plurality of confidence scores, an association between one or more lane segments and one or more active traffic signals; see Fig 18 regarding active signal for one or more lane segments, and ),
wherein the association of the traffic control device with the lane is based at least on the first score being greater than the one or more second scores (see Fig 18 regarding active signal for one or more lane segments).
However, Chen, as modified by Han and Weiland, does not explicitly disclose:
based at least on a first score of the plurality of scores being greater than one or more second scores of the plurality of scores, that the traffic control device corresponds to the lane.
However, Baalke teaches in [0008] s.2, that an autonomous vehicle may formulate a first and second confidence score and when a confidence score exceeds the other as well as a threshold score it performs an operation of autonomously navigating the current lane.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the lane associating based on confidence level teachings of Weiland, by incorporating the confidence score comparison teachings of Tang, as taught in [0074] s.3, such that it allows for improved confidence e performance of the vehicle on a specific path.
___________________________________________
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, Weiland et al. (US Pat. No. 8,892,356 B1), hereinafter referred to as Weiland, and an article by Patle et al. titled, “Matrix-Binary Codes based Genetic Algorithm for path planning of mobile robot”, hereinafter referred to as Patle.
Regarding claim 16, Chen, as modified by Han and Weiland, discloses:
The system of claim 8 (see claim 1), the one or more processors further to:
a plurality of entries indicative of one or more valid pairings between respective traffic control devices of the plurality of traffic control devices and respective lanes of the plurality of lanes (see claim 1);
the association of at least the traffic control device with the lane (see claim 1),
but Chen, as modified by Han and Weiland, does not explicitly disclose:
is based at least on the binary matrix.
However, Patle teaches in the introduction s.3 that Matrix-Binary Codes representation for solving the navigational problem of autonomous robot system especially for dynamic goal and dynamic obstacle problems as seen in navigation.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the lane associating based on confidence level teachings of Weiland, by incorporating the binary matrix teachings of Patle, as taught in the introduction s.3, such that it allows for improved efficiency and robustness as it does not require extra information about the problem.
____________________________________________
Claims 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pat. Pub. No. 2025/0200987 A1), hereinafter referred to as Chen, in view of Han et al. (US Pat. No. 11,738,770 B2), hereinafter referred to as Han, Moustafa et al. (US Pat. Pub. No. 2022/0126864 A1), hereinafter referred to as Moustafa.
Regarding claim 18, Chen discloses:
One or more processing units comprising:
processing circuitry to update one or more parameters of one or more deep neural networks (DNNs) to train the one or more DNNs to associate one or more active traffic control signals with one or more directions of one or more lanes of a driving surface using (see claim 1 regarding machine learning and/or neural networks, [0045] s.7 the semantic class estimation may include segmenting and/or classifying extracted deep convolutional features into semantic data (e.g., rigidity, hardness, safety risk, risk of position change, class or type, potential direction of travel, etc.), and [0091] deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN)).
However, Chen does not disclose:
from non-visual data based at least on one or more rules associated with positioning one or more traffic control devices relative to one or more corresponding lane segments.
However, Han teaches in column (col) 28 line(s) 51-67 that the HD map system may generate a lane element graph that represents a network of lanes to allow a vehicle to plan a legal path between a source and a destination. A lane element graph may allow navigation of autonomous vehicles through a mapped area. Each lane element may be associated with the traffic restrictions that apply to it such as speed limit, speed bump, and traffic signs and signals. A lane element graph may represent the navigable road surface that is divided into lane elements, and may include connectivity among lane elements (e.g., where vehicles can go from a current lane element) as well as semantic association between lane elements and features (e.g., a speed limit in a current lane element) to assist in on-vehicle routing and planning needs. The lane elements may be topologically connected, and each lane element may be known to its successors, predecessors, and left and right neighbors. This is construed as explicitly generating a lane element graph in which individual lane elements are semantically associated with the traffic restrictions and controls that apply to them (i.e., traffic signals and signs, speed limits, etc.). These associations are not derived from visual perception alone; they are “ground-truth” semantic links that are encoded in the map’s lane element graph structure itself based upon legal navigability required for path planning.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, by incorporating the semantic lane associating teachings of Han, such that the combination would provide for the predictable result of improving autonomous decision making based upon associated traffic parameters.
However, Chen, as modified by Han, does not explicitly disclose:
a training dataset that is synthetically generated.
However, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by incorporating the synthetic data trained machine learning model teachings of Moustafa, such that the combination would provide for the predictable result of, as acknowledged by Moustafa in [0439] s.9, improving training accessibility by intelligently controlling the synthetic data creation, such that the system may create images (for training) that would otherwise require a very long time for a vehicle to encounter in real life.
Regarding claim 19, Chen, as modified by Han and Moustafa, discloses:
The one or more processing units of claim 18, wherein the training dataset is synthetically generated, at least, by:
projecting three-dimensional (3D) geometry for a plurality of lanes to two-dimensional (2D) image space based at least on one or more intrinsic or extrinsic camera parameters ([0091] Dimensionality Reduction Algorithms which may reduce a 3D geometry onto a 2D image space based on parameters);
projecting 3D geometry for a plurality of traffic control devices to 2D image space based at least on the one or more intrinsic or extrinsic camera parameters (see [0091] as discussed above); and
generating a plurality of traffic control device to lane pairs between the plurality of lanes and the plurality of traffic control devices based at least on one or more traffic regulations ([0019] object avoidance decisions are based on semantic classes of information including traffic signal information and geometric characteristics of the class, the position of the object (e.g., height of the object, distance of the object from the ground surface, shape of the object, etc.), [0045] machine learning models and/or neural networks perform classification of the semantic information, and [0084] communication connection(s) may allow the vehicle to communicate with other nearby computing device(s) (e.g., traffic signals)).
Regarding claim 20, Chen, as modified by Han and Moustafa, discloses
The one or more processing units of claim 18, wherein the one or more processing units are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle and [0002] use in autonomous vehicles);
a perception system for an autonomous or semi-autonomous machine ([0082] the vehicle computing device can include one or more system controllers, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle and [0017] s.4, perception pipelines and sensor systems);
a system for performing one or more digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets ([0050] determine a bounding box or region associated with the object and determine if the bounding box intersects with a corridor (e.g., a three-dimensional representation of the planned path));
a system for performing one or more deep learning operations (([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks; [0045] s.7 the semantic class estimation may include segmenting and/or classifying extracted deep convolutional features into semantic data (e.g., rigidity, hardness, safety risk, risk of position change, class or type, potential direction of travel, etc.); and [0091] deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN)));
a system implemented using an edge device;
a system implemented using a robot ([0078] s.2, use within a robotic system);
a system for performing one or more generative AI operations ([0038] sentence (s.) 2, the vehicle may process the captured sensor data using various machine learned models and/or neural networks which machine learning is a known subset of AI operations);
a system for performing operations using a large language model;
a system for performing operations using one or more vision language models (VLMs);
a system for performing operations using one or more multi-modal language models;
a system for using or deploying one or more inference microservices;
a system for performing one or more conversational AI operations;
a system for generating synthetic data (as discussed in claim 18, Moustafa teaches in [0439] s.1-3 an autonomous vehicle system may create synthetic data in order to bolster data sets lacking real data for one or more contexts, such as a generative adversarial network (GAN) image generator. GAN is a type of generative model that uses machine learning, more specifically deep learning, to generate images (e.g., still images or video clips) based on a list of keywords presented as input to the GAN. This is construed as partially training a machine learning model using one or more synthetically generated training datasets);
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center.
However, Chen does not explicitly disclose:
a system for performing one or more simulation operations; and
a system implemented at least partially using cloud computing resources.
However, Han is further relied upon to teach in col 5 ln 34-67 cloud-based service that may allow clients such as a vehicle computing system 120 ( e.g., vehicle computing systems 120a-d) to make requests for information and services. Furthermore, Han teaches in col 47 ln 3-5 can also be applied for displaying maps for purposes of computer simulation.
Therefore it would have been obvious to one of ordinary skill in the art of data science, data engineering, and vehicle controls before the effective filing date of the current invention to modify the machine learning modeling of Chen, as already modified by the semantic lane associating teachings of Han, by further incorporating the simulation and cloud teachings of Han, such that as the semantic lane associating teachings of Han are considered within Chen, the simulation and cloud teachings are also considered.
Prior Art
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Please see:
Song et al. article “Synthetic datasets for autonomous driving: A survey” is directed towards the evolution of synthetic dataset generation methods and review the work to date in synthetic datasets related to single and multi-task categories for the autonomous driving perception study as researchers are turning to synthetic datasets to easily generate rich and changeable data as an effective complement to the real world and to improve the performance of algorithms; and
Bai et al. article “Bridging the domain gap between synthetic and real-world data for autonomous driving” is directed towards simulations are necessary for modern autonomous systems' development and evaluation and the Scoping Autonomous Vehicle Simulation (SAVeS) platform benchmarks the performance of simulated environments for autonomous ground vehicle testing between synthetic and real-world domains.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEITH ALLEN VON VOLKENBURG whose telephone number is (703)756-5886. The Examiner can normally be reached Monday-Friday 8:30 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin D. Bishop can be reached at (571) 270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Keith A von Volkenburg/ Examiner, Art Unit 3665
/Erin D Bishop/ Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights