Detailed Office Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the Applicant’s amendments and remarks filed 07/07/2025. The applicant has amended claims 1 and 20. Applicant has cancelled Claim 13. Claims 1-12 and 14-20 are presently pending and are presented for examination.
Response to Amendment
The amendment filed 07/07/2025 has been entered. Claims 1-12 and 14-20 remain pending in the application.
Reply to Applicant’s Remarks
Applicant’s remarks filed 07/07/2025 have been fully considered and are addressed as follows:
Claim Rejections Under 35 U.S.C. 101:
Applicant’s amendments to the claims filed 07/07/2025 have not overcome the 35 U.S.C 101 rejections previously set forth. Regarding the Applicant’s argument that “Applicant submits that these claim limitations result in the amended independent claims reciting "additional elements"”, the Examiner respectfully disagrees. The examiner asserts that the limitation “obtaining a high definition (HD) map and a map schema associated with the HD map wherein said HD map is in an XML format, and wherein said map scheme is in an XML Schema Definition format” as whole, whether narrowed down to a specific data format or not, still constitutes the simple sending, receiving, and storing of data, which is an insignificant extra solution activity, and can therefore not integrate the abstract idea into a practical application.
Please see detailed rejection below.
Claim Rejections Under 35 U.S.C. 102/103:
Applicant’s arguments, see Arguments/Remarks, filed 07/07/2025, with regard to the rejections of Claim 1 under 35 U.S.C. 102 which now contains the subject matter of currently cancelled Claim 13, which was previously rejected under 103, the arguments have been fully considered but are respectfully not persuasive. Applicant argues that Akbarzadeh fails to teach the use of map schema and fails to teach the comparing of the HD map to a map schema. The submitted Specification for the invention states, “A map schema may describe the type and/or values that each map element (e.g. lane segments, road edges, road markers) or attribute can have”. The examiner asserts that Akbarzadeh teaches the use of map schema and teaches the comparing of the HD map to the map schema (see at least [¶ 4, 22, 42, and 56] various data pipelines—such as sensor data—from the ego-machine may be provided to a map health verifier (e.g., source code or other executable logic executed by one or more processors) and compared against elements of the HD map—such as a lane graph layer, a static obstacles layer, etc)…The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map)… different layers (e.g., lane planning, lane graph, static obstacles, etc.)…The map 102 as described herein may include any data structure and/or type...a map layer may represent a class of data on the map 102....a core layer may contain non-image features and may be replaced with other layers specified. A schema—such as a flatbuffers schema—may be used to define each road segment of a layer) Because the layers of the HD map describe the type and/or values that each map element can have, the layers are analogous to a map schema. Additionally, Akbarzadeh goes on to explicitly describe how each road segment of the layers can be defined with a flatbuffer schema. Further, when the system compares the current perception of the vehicle with the element information from the layers/scheme, it is comparing an HD map (the perception of the vehicle) to a map scheme (the layers, data structures, flatbuffer schema).
Further, the applicant argues that Akbarzadeh does not teach nor suggest that the HD map is in XML format and that the map schema is in XML Schema Definition format. However, as stated in the previous 103 rejection for Claim 13, Akbarzadeh already discloses both the use of an HD map and map schema and therefore teaches the claimed invention except for wherein the map schema is in an XML Schema Definition format, and wherein said HD map is in an XML format. It would have been obvious to anyone of ordinary skill in the art before the effective filling date of the claimed invention to have included map schema is in an XML Schema Definition format, and HD map in an XML format since it has been held to be within the general skill of a worker in the art to select such formats based on their suitability for the intended use as a matter of design choice.
Further, the applicant argues that Akbarzadeh fails to teach or suggest “selecting an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map”. The examiner asserts that Akbarzadeh teaches selecting an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map (see at least Akbarzadeh [¶ Abstract, 23-24, 37, 38, and 40-41] Where errors are identified that indicate a drop in health of the HD map, updated data may be crowdsourced from one or more vehicles corresponding to a location of disagreement within the HD map, and the updated data may be used to update, verify, and validate the HD map…when the perception of the ego-machine and the information provided by the HD map are not in agreement with one another, the system may determine that there is an error in the HD map…the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms…ego-machines generate map health data during operation and provide the map health data associated with road segments or other portions of the map 102 to server computer systems…the system (e.g., the one or more server computer systems, the ego-machine, or a combination) may determine an amount of validation to perform (e.g., frequency, portions of the map 102, number of validating ego-machines, scope of validation, etc.)…validation of the map 102 is performed at a road segment-level (e.g., 50 meter portions of roads included in the map 102) and individual road segments are marked as valid or invalid based at least in part on a result of a localization health check and/or a map health check as described in the present disclosure…road segment-level validation can include validation (or invalidation) of specific layers of the map 102. For example, as a result of a localization health check and/or a map health check, a camera layer of the map 102 and/or a road sign layer of the map 102 may be invalidated).
The system disclosed in Akbarzadeh determines tests to execute that will validate the map based on localization health, map health, and other criteria, because a set of tests are determined based on criteria, the disclosure is teaching selecting an optimized set of test cases. Further, because the test cases are selected based on map health (or in other words if the map has many disagreements with the perception of the ego vehicle) the selected test cases are based on variabilities of the HD map. Further, because the system determines the amount of validation to perform, the test cases used for validation are based on the plurality of generated test cases. Further, because tests are determined based on map health, and because an unhealthy portion of the map needs to be validated, it follows that the unhealthy portions of the map are unvalidated and are selected as tests for comparison as a result.
See detailed rejection below.
Claim Objections
Claims 1 and 20 are objected to because of the following informalities: the acronym XML remains undefined in the claims. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 and 14-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims’ subject matter eligibility will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”).
101 Analysis - With respect to Claim 1
Claims 1 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
101 Analysis - Step 1:
Claim 1 is directed towards a method which is directed to the statutory category of a process. Claim 20 is directed towards a system which is directed to the statutory category of a machine. Therefore Claims 1 and 20 are within at least one of the four statutory categories.
101 Analysis- Step 2A Prong One:
Regarding Prong One of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental process.
Independent claim 1 includes limitations that recite an abstract idea (emphasized below) and will be used as a representative claim for the remainder of the 101 rejection.
Claim 1 recites, inter alai:
“A method of validating a map for use with an autonomous vehicle, the method comprising:
obtaining a high definition (HD) map and a map schema associated with the HD map, wherein said HD map is in an XML format, and wherein said map scheme is in an XML Schema Definition format;
comparing, by a processor, said HD map to said map schema;
storing said HD map in a map database based on said comparing;
identifying one or more variabilities in said HD map;
generating, by the processor, a map graph based on said HD map, said map graph comprising nodes representing lane segments and edges representing connections between said lane segments;
generating, by the processor, a plurality of test cases based on the map graph, testing requirements, and at least one previously validated lane segment;
selecting an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map;
generating a plurality of execution instances, said optimized set of test cases being subdivided among said plurality of execution instances; and
validating said HD map by executing said plurality of execution instances”
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind.
For example, “comparing”, “generating”, “identifying”, “selecting”, and “validating” in the context of this claim, all encompass a person looking at available data and forming a simple judgement (determination, analysis, comparison, etc.) either manually or using a pen and paper. Accordingly, the claim recites at least one abstract idea. The examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
As drafted, the above claims, under their broadest reasonable interpretation, cover mental processes performed in the human mind (including an observation, evaluation, judgement, opinion), that are merely completed via generic computer components. Accordingly, the claims recite an abstract idea.
Step 2A Prong Two Analysis:
Regarding Prong Two of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract idea into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application”.
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
Claim 1 recites, inter alai:
“A method of validating a map for use with an autonomous vehicle, the method comprising:
obtaining a high definition (HD) map and a map schema associated with the HD map, wherein said HD map is in an XML format, and wherein said map scheme is in an XML Schema Definition format;
comparing, by a processor, said HD map to said map schema;
storing said HD map in a map database based on said comparing;
identifying one or more variabilities in said HD map;
generating, by the processor, a map graph based on said HD map, said map graph comprising nodes representing lane segments and edges representing connections between said lane segments;
generating, by the processor, a plurality of test cases based on the map graph, testing requirements, and at least one previously validated lane segment;
selecting an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map;
generating a plurality of execution instances, said optimized set of test cases being subdivided among said plurality of execution instances; and
validating said HD map by executing said plurality of execution instances”
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “obtaining a high definition (HD) map and a map schema associated with the HD map, wherein said HD map is in an XML format, and wherein said map scheme is in an XML Schema Definition format” and “storing said HD map in a map database based on said comparing“ these limitation merely describes the sending, receiving, and storing of data which is in insignificant extra solution activity. See MPEP § 2106.05(g).
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B Analysis:
The claims do not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Further, the act of collecting data and displaying data amounts to no more than merely storing and displaying information of the exception and thus is an extra-solution activity. The claims are not patent eligible.
Regarding dependent claims 2-12 and 14-19, no claim further adds a limitation that introduces any practical applications to the claimed invention, the dependent claims merely add more mental process, mathematical concepts, and post-solution activities and are thus not patent eligible.
Therefore, Claims 1-12 and 14-20 are ineligible under 35 USC §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 10-12, 14-18, and 20 rejected under 35 U.S.C. 103 as being unpatentable over Akbarzadeh et al (US 20220341750 A1), hereafter referred to as Akbarzadeh.
Regarding Claim 1, Akbarzadeh teaches a method of validating a map for use with an autonomous vehicle (see at least Akbarzadeh [¶ 4] systems and methods are disclosed that compare a perception system of an ego-machine to a HD map to determine the accuracy of the HD map. For example, various data pipelines—such as sensor data—from the ego-machine may be provided to a map health verifier (e.g., source code or other executable logic executed by one or more processors) and compared against elements of the HD map—such as a lane graph layer, a static obstacles layer, etc)
the method comprising:
obtaining a high definition (HD) map and a map schema associated therewith (see at least Akbarzadeh [¶ 36, 22, 56] a car map manager 114 may obtain the map 102 and determine one or more regions of the map 102 to validate...The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map)… different layers (e.g., lane planning, lane graph, static obstacles, etc.) may be weighted differently based at least in part on how significant an error in a particular layer will affect the driving experience… a core layer may contain non-image features and may be replaced with other layers specified. A schema—such as a flatbuffers schema—may be used to define each road segment of a layer) Because the layers of the HD map describe the type and/or values that each map element can have, the layers are analogous to a map schema. Additionally, Akbarzadeh goes on to explicitly describe how each road segment of the layers can be defined with a flatbuffer schema.
comparing, by a processor, said HD map to said map schema (see at least Akbarzadeh [¶ 4, 27, 42, 45, 56] The map 102 as described herein may include any data structure and/or type...a map layer may represent a class of data on the map 102....a core layer may contain non-image features and may be replaced with other layers specified. A schema—such as a flatbuffers schema—may be used to define each road segment of a layer...systems and methods are disclosed that compare a perception system of an ego-machine to a HD map to determine the accuracy of the HD map…sensor data from the ego-machine may be provided to a map health verifier…and compared against elements of the HD map—such as a lane graph layer, a static obstacles layer, etc…The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map)) When the system compares the current perception of the vehicle with the element information from the layers/scheme, it is comparing an HD map (the perception of the vehicle) to a map scheme (the layers, data structures, flatbuffer schema).
storing said HD map in a map database based on said comparing (see at least Akbarzadeh [¶ 37] testing of updates to the map 102 may be performed without interfering with a user of the ego-machine and map validity information may be stored locally (e.g., by the ego-machine and/or within the map data in the local 112 data store))
identifying one or more variabilities in said HD map (see at least Akbarzadeh [¶ 6] disagreements between the HD map, components of the perception system, and/or with an alignment system that exceed a threshold level may be aggregated and stored)
generating, by the processor, a map graph based on said HD map, said map graph comprising nodes representing lane segments and edges representing connections between said lane segments (see at least Akbarzadeh [¶ 31, 77] This module may compare live perception (e.g., data generated by one or more models using sensor data) to a map, in embodiments, and may be able to provide (1) lane graph—e.g., graph lane piece by piece (between perception and map—compare channel type and other details). For example, the output may be an association and confidence per association. Overall lane graph confidence may also be provided…frame graphs representing the pose links may be divided into road segments—e.g., the road segments that are used for relative localization—and the poses corresponding to each road segment may undergo optimization. The resulting, finalized poses within each segment, in various examples, may be used to fuse various sensor data and/or perception outputs for generating a final fused and/or updated HD map—or portion or segment thereof)
generating, by the processor, a plurality of test cases based on the map graph, testing requirements, and at least one previously validated lane segment (see at least Akbarzadeh [¶ 37, 59, 22] the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms....the map can have a “validation layer” 202 for road segments that indicate whether it is suitable for driving; (2) as more mapstreams 220 are uploaded, these may be tested by one or more server computer systems...The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map))
selecting an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map (see at least Akbarzadeh [¶ Abstract, 23-24, 37, 38, and 40-41] Where errors are identified that indicate a drop in health of the HD map, updated data may be crowdsourced from one or more vehicles corresponding to a location of disagreement within the HD map, and the updated data may be used to update, verify, and validate the HD map…when the perception of the ego-machine and the information provided by the HD map are not in agreement with one another, the system may determine that there is an error in the HD map…the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms…ego-machines generate map health data during operation and provide the map health data associated with road segments or other portions of the map 102 to server computer systems…the system (e.g., the one or more server computer systems, the ego-machine, or a combination) may determine an amount of validation to perform (e.g., frequency, portions of the map 102, number of validating ego-machines, scope of validation, etc.)…validation of the map 102 is performed at a road segment-level (e.g., 50 meter portions of roads included in the map 102) and individual road segments are marked as valid or invalid based at least in part on a result of a localization health check and/or a map health check as described in the present disclosure…road segment-level validation can include validation (or invalidation) of specific layers of the map 102. For example, as a result of a localization health check and/or a map health check, a camera layer of the map 102 and/or a road sign layer of the map 102 may be invalidated) The system disclosed in Akbarzadeh determines tests to execute that will validate the map based on localization health, map health, and other criteria, because a set of tests are determined based on criteria, the disclosure is teaching selecting an optimized set of test cases. Further, because the test cases are selected based on map health (or in other words if the map has many disagreements with the perception of the ego vehicle) the selected test cases are based on variabilities of the HD map. Further, because the system determines the amount of validation to perform, the test cases used for validation are based on the plurality of generated test cases. Further, because tests are determined based on map health, and because an unhealthy portion of the map needs to be validated, it follows that the unhealthy portions of the map (which can be segments) are unvalidated and are selected as tests for comparison as a result.
generating a plurality of execution instances, said optimized set of test cases being subdivided among said plurality of execution instances; and validating said HD map by executing said plurality of execution instances (see at least Akbarzadeh [¶ 28, 37] various functions may be carried out by a processor executing instructions stored in memory. In some embodiments, the systems, methods, and processes described herein may be executed using similar components, features, and/or functionality to those of example autonomous vehicle 800…the system provides server-side validation (e.g., as executed by the map creation and cross-validation 116 component) and real-time verification 106 of the map 102. In such embodiments, the system determines mapstreams 120 to use for map 102 creation and/or validation. In addition, in various examples, the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms).
However, Akbarzadeh does not explicitly teach wherein the map schema is in an XML Schema Definition format, and wherein said HD map is in an XML format.
However, Akbarzadeh teaches the use of map schema and HD maps for use in autonomous vehicles [¶ 4].
Therefore, Akbarzadeh discloses the claimed invention except for wherein the map schema is in an XML Schema Definition format, and wherein said HD map is in an XML format. It would have been obvious to anyone of ordinary skill in the art before the effective filling date of the claimed invention to have included map schema is in an XML Schema Definition format, and HD map in an XML format since it has been held to be within the general skill of a worker in the art to select such formats based on their suitability for the intended use as a matter of design choice.
Regarding Claim 20, Akbarzadeh teaches a system for performing map validation (see at least Akbarzadeh [¶ 4] systems and methods are disclosed that compare a perception system of an ego-machine to a HD map to determine the accuracy of the HD map. For example, various data pipelines—such as sensor data—from the ego-machine may be provided to a map health verifier (e.g., source code or other executable logic executed by one or more processors) and compared against elements of the HD map—such as a lane graph layer, a static obstacles layer, etc)
The system comprising:
One or more processors (see at least Akbarzadeh [¶ 4] executable logic executed by one or more processors)
a non-transitory computer-readable storage medium having stored thereon computer-executable instructions (see at least Akbarzadeh [¶ 205] As used herein, computer storage media does not comprise signals per se)
that, when executed by the one or more processors, cause the one or more processors to:
obtain a high definition (HD) map and a map schema associated therewith (see at least Akbarzadeh [¶ 36, 22, 56] a car map manager 114 may obtain the map 102 and determine one or more regions of the map 102 to validate...The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map)… different layers (e.g., lane planning, lane graph, static obstacles, etc.) may be weighted differently based at least in part on how significant an error in a particular layer will affect the driving experience… a core layer may contain non-image features and may be replaced with other layers specified. A schema—such as a flatbuffers schema—may be used to define each road segment of a layer) Because the layers of the HD map describe the type and/or values that each map element can have, the layers are analogous to a map schema. Additionally, Akbarzadeh goes on to explicitly describe how each road segment of the layers can be defined with a flatbuffer schema.
compare said HD map to said map schema (see at least Akbarzadeh [¶ 4, 27, 42, 45, 56] The map 102 as described herein may include any data structure and/or type...a map layer may represent a class of data on the map 102....a core layer may contain non-image features and may be replaced with other layers specified. A schema—such as a flatbuffers schema—may be used to define each road segment of a layer...systems and methods are disclosed that compare a perception system of an ego-machine to a HD map to determine the accuracy of the HD map…sensor data from the ego-machine may be provided to a map health verifier…and compared against elements of the HD map—such as a lane graph layer, a static obstacles layer, etc…The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map)) When the system compares the current perception of the vehicle with the element information from the layers/scheme, it is comparing an HD map (the perception of the vehicle) to a map scheme (the layers, data structures, flatbuffer schema).
store said HD map in a map database based on said comparing (see at least Akbarzadeh [¶ 37] testing of updates to the map 102 may be performed without interfering with a user of the ego-machine and map validity information may be stored locally (e.g., by the ego-machine and/or within the map data in the local 112 data store))
identify one or more variabilities in said HD map (see at least Akbarzadeh [¶ 6] disagreements between the HD map, components of the perception system, and/or with an alignment system that exceed a threshold level may be aggregated and stored)
generate a map graph based on said HD map, said map graph comprising nodes representing lane segments and edges representing connections between said lane segments (see at least Akbarzadeh [¶ 31, 77] This module may compare live perception (e.g., data generated by one or more models using sensor data) to a map, in embodiments, and may be able to provide (1) lane graph—e.g., graph lane piece by piece (between perception and map—compare channel type and other details). For example, the output may be an association and confidence per association. Overall lane graph confidence may also be provided…frame graphs representing the pose links may be divided into road segments—e.g., the road segments that are used for relative localization—and the poses corresponding to each road segment may undergo optimization. The resulting, finalized poses within each segment, in various examples, may be used to fuse various sensor data and/or perception outputs for generating a final fused and/or updated HD map—or portion or segment thereof)
generate a plurality of test cases based on the map graph, testing requirements, and at least one previously validated lane segment (see at least Akbarzadeh [¶ 37, 59, 22] the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms....the map can have a “validation layer” 202 for road segments that indicate whether it is suitable for driving; (2) as more mapstreams 220 are uploaded, these may be tested by one or more server computer systems...The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map))
select an optimized set of test cases based on said plurality of generated test cases, said one or more variabilities in said HD map, and unvalidated lane segments in said HD map (see at least Akbarzadeh [¶ Abstract, 23-24, 37, 38, and 40-41] Where errors are identified that indicate a drop in health of the HD map, updated data may be crowdsourced from one or more vehicles corresponding to a location of disagreement within the HD map, and the updated data may be used to update, verify, and validate the HD map…when the perception of the ego-machine and the information provided by the HD map are not in agreement with one another, the system may determine that there is an error in the HD map…the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms…ego-machines generate map health data during operation and provide the map health data associated with road segments or other portions of the map 102 to server computer systems…the system (e.g., the one or more server computer systems, the ego-machine, or a combination) may determine an amount of validation to perform (e.g., frequency, portions of the map 102, number of validating ego-machines, scope of validation, etc.)…validation of the map 102 is performed at a road segment-level (e.g., 50 meter portions of roads included in the map 102) and individual road segments are marked as valid or invalid based at least in part on a result of a localization health check and/or a map health check as described in the present disclosure…road segment-level validation can include validation (or invalidation) of specific layers of the map 102. For example, as a result of a localization health check and/or a map health check, a camera layer of the map 102 and/or a road sign layer of the map 102 may be invalidated). The system disclosed in Akbarzadeh determines tests to execute that will validate the map based on localization health, map health, and other criteria, because a set of tests are determined based on criteria, the disclosure is teaching selecting an optimized set of test cases. Further, because the test cases are selected based on map health (or in other words if the map has many disagreements with the perception of the ego vehicle) the selected test cases are based on variabilities of the HD map. Further, because the system determines the amount of validation to perform, the test cases used for validation are based on the plurality of generated test cases. Further, because tests are determined based on map health, and because an unhealthy portion of the map needs to be validated, it follows that the unhealthy portions of the map (which can be segments) are unvalidated and are selected as tests for comparison as a result.
generate a plurality of execution instances, said optimized set of test cases being subdivided among said plurality of execution instances; and validate said HD map by executing said plurality of execution instances (see at least Akbarzadeh [¶ 28, 37] various functions may be carried out by a processor executing instructions stored in memory. In some embodiments, the systems, methods, and processes described herein may be executed using similar components, features, and/or functionality to those of example autonomous vehicle 800…the system provides server-side validation (e.g., as executed by the map creation and cross-validation 116 component) and real-time verification 106 of the map 102. In such embodiments, the system determines mapstreams 120 to use for map 102 creation and/or validation. In addition, in various examples, the system determines to tests to execute for mapstreams 120 against a map 102 either using localization health, map health, and/or alternate mechanisms).
However, Akbarzadeh does not explicitly teach wherein the map schema is in an XML Schema Definition format, and wherein said HD map is in an XML format.
However, Akbarzadeh teaches the use of map schema and HD maps for use in autonomous vehicles [¶ 4].
Therefore, Akbarzadeh discloses the claimed invention except for wherein the map schema is in an XML Schema Definition format, and wherein said HD map is in an XML format. It would have been obvious to anyone of ordinary skill in the art before the effective filling date of the claimed invention to have included map schema is in an XML Schema Definition format, and HD map in an XML format since it has been held to be within the general skill of a worker in the art to select such formats based on their suitability for the intended use as a matter of design choice.
Regarding Claim 3, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein each of said test cases comprises a map route comprising a plurality of lane segments to be validated (see at least Akbarzadeh [¶ 41] validation of the map 102 is performed at a road segment-level (e.g., 50 meter portions of roads included in the map 102) and individual road segments are marked as valid or invalid based at least in part on a result of a localization health check and/or a map health check as described in the present disclosure…road segment-level validation can include validation (or invalidation) of specific layers of the map 102. For example, as a result of a localization health check and/or a map health check, a camera layer of the map 102 and/or a road sign layer of the map 102 may be invalidated… individual elements of the map 102 may be validated. For example, traffic lights, road segments, tiles, road patterns, or any other data included in the map 102 can be validated individually and/or as a result of features within the element being validated).
Regarding Claim 10, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein said optimized set of test cases is subdivided among said plurality of execution instances so as to minimize a difference in execution time between each of said execution instances (see at least Akbarzadeh [¶ 28, 35, 37, 62] the system may consider variations where validation on a “test map” may be running continuously, there may be a minimum or maximum time per ego-machine, or for the duration while a “test” map is being testing…in various embodiments, as the road structure, layout, conditions, surroundings, and/or other information change, health checking or monitoring may be performed to update the map data more quickly—e.g., in real-time or substantially real-time. For example, the real-time verification 106 component process sensor data and compares the sensor data to the map (e.g., using one or more perception systems) to verify one or more components of the map 102).
Regarding Claim 11, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein said map schema includes a definition for each map element and attributes associated with each respective map element (see at least Akbarzadeh [¶ 41-43, 24] individual elements of the map 102 may be validated. For example, traffic lights, road segments, tiles, road patterns, or any other data included in the map 102 can be validated individually and/or as a result of features within the element being validated (e.g., a tile can be valid as a result of all features within the tile being validated)… the map manifest for the map 102 may define one or more tiles that may be stored in a tile cache 110…a road segment within a tile may represent a portion of a roadway (e.g., 50 meters), and each tile may include a plurality of road segments defined therein—e.g., each of the road segments within the tile geographic region may be included within the tile).
Regarding Claim 12, Akbarzadeh teaches all limitations of Claim 12 as set forth above. Akbarzadeh further teaches wherein said map elements include one or more of lane segments, road edges, and/or road markers (see at least Akbarzadeh [¶ 6, 24] Different layers of the HD map (e.g., lane planning, lane graph, static obstacles, etc.) may be weighted differently based at least in part on how significant an error in a particular layer will affect the driving experience...when the system determines that there is a disagreement between the HD map, components of the perception system, and/or with an alignment system, the system may apply a weight to the error to produce an error value corresponding to a road segment and/or an element on the road segment (e.g., sign, lane, wait line, traffic light, tunnel, etc).
Regarding Claim 14, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein said testing requirements include at least one of application requirements and execution requirements (see at least Akbarzadeh [¶ 36, 153] there may be different options and requirements on or from downstream systems (e.g., the system may determine how much validation is required)…in client-side validation, maps, map data, and/or portions thereof may be tested by the ego-machine before driving. For example, a car map manager 114 may obtain the map 102 and determine one or more regions of the map 102 to validate. In real-time verification examples (e.g., where the ego-machine includes the real-time verification 106 component), maps, map data, and/or portions thereof may be validated as a user drives (e.g., compared to perception)).
Regarding Claim 15, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein the map database includes previous versions of said HD map (see at least Akbarzadeh [¶ 80] to make sure verification data can be used from previous drives, the system may maintain road segment identification information whenever the system performs a re-fuse operation…In other embodiments, the system may not switch maps (e.g., from version one to version two) if the version of the map does not have verification data for routes that may be needed by the system and the system will continue driving using a previous map version).
Regarding Claim 16, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches wherein said identifying one or more variabilities in said HD map is based on selecting an interested map feature as a variational point for said HD map relative to previous versions of said HD map (see at least Akbarzadeh [¶ 6, 22] disagreements between the HD map, components of the perception system, and/or with an alignment system that exceed a threshold level may be aggregated and stored. Different layers of the HD map (e.g., lane planning, lane graph, static obstacles, etc.) may be weighted differently based at least in part on how significant an error in a particular layer will affect the driving experience…The system, in various embodiments, then compares current perception of the ego-machine with element information or other information provided by the HD map (e.g., stored in one or more layers of the HD map). For example, the system may evaluate a lane center with respect to the ego-machine to determine whether the ego-machine localization within the HD map agrees with the current perception of the ego-machine. Based at least in part on this evaluation, the system may determine whether the perception system of the ego-machine and the HD map are aligned and in agreement, in an embodiment).
Regarding Claim 17, Akbarzadeh teaches all limitations of Claim 1 as set forth above. Akbarzadeh further teaches generating a defect report based on said validating said HD map against said map schema (see at least Akbarzadeh [¶ 6, 87] disagreements between the HD map, components of the perception system, and/or with an alignment system that exceed a threshold level may be aggregated and stored. Different layers of the HD map (e.g., lane planning, lane graph, static obstacles, etc.) may be weighted differently based at least in part on how significant an error in a particular layer will affect the driving experience—e.g., will affect the safety of the ego-machine when relying on the respective layer of the HD map if an error or disagreement is present …FIG. 7 is a flow diagram showing a method 700 for causing a remote server to update a portion of map data, in accordance with some embodiments of the present disclosure. The method 700, at block B702, includes generating an indication of map health corresponding to a portion of map data. As described above, in various embodiments, a disagreement between map data and perception of an environment may cause the system executing the method 700 to invalidate one or more portions of a map (e.g., tiles, road segments, etc)).
Regarding Claim 18, Akbarzadeh teaches all limitations of Claim 17 as set forth above. Akbarzadeh further teaches wherein said defect report includes a discontinuity in a lane-level network of said HD map (see at least Akbarzadeh [¶ 5] the system may evaluate a lane center with respect to the ego-machine to determine whether the ego-machine localization within the HD map agrees with the current perception of the ego-machine. When the perception of the ego-machine and the information provided by the HD map are not in agreement (misaligned) with one another, there may be an error in the HD map).
Claim 2 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Akbarzadeh et al (US 20220341750 A1) in view of Devitt et al (US 20210158081 A1). Hereafter referred to as Akbarzadeh and Devitt respectively.
Regarding Claim 2, Akbarzadeh teaches all limitations of Claim 1 as set forth above. However, Akbarzadeh does not explicitly teach wherein generating said map graph comprises parsing said HD map using a multithreaded parsing and thread synchronization mechanism, and storing parsed information in a list structure.
Devitt, in the same field as the endeavor, teaches wherein generating said map graph comprises parsing said HD map using a multithreaded parsing and thread synchronization mechanism, and storing parsed information in a list structure (see at least Devitt [ 61, 101] the cost module can validate the correspondence map by comparing a correspondence map…with a second correspondence map…The correspondence maps to be compared can be determined in series and/or parallel....S500 preferably functions to determine whether the corresponding pixel in the opposing image, identified by the correspondence vector, is a good match with the analysis pixel. This is determined by comparing the analysis pixel…to corresponding pixel(s) in the corresponding image…S500 is preferably performed for all pixels within an image in parallel…The set of correspondence vectors can be associated with: a single pixel (e.g., analysis pixel), the entire image (e.g., cooperatively form a correspondence map), or be associated with any other suitable data structure).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to have modified the system set forth in Akbarzadeh to contain a system for wherein generating said map graph comprises parsing said HD map using a multithreaded parsing and thread synchronization mechanism, and storing parsed information in a list structure with reasonable expectation of success. One of ordinary skill in the art would have been motivated to make such a modification for benefit of simplifying calculations and saving processing time and resources as discussed in Devitt (see at least Devitt [¶ 26]).
Regarding Claim 19, Akbarzadeh teaches all limitations of Claim 1 as set forth above. However, Akbarzadeh does not explicitly teach wherein said plurality of execution instances are executed in parallel.
Devitt, in the same f