DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed September 2, 2025 has been entered. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant’s arguments, see Pages 7-8 of Remarks, filed September 2, 2025, with respect to the rejection(s) of claim(s) 1-8 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of User: 'barf' (BlenderGIS - How to drape ortho-photos onto a DEM/DSM) and Pancroma (Extracting DEMs from Topographic Maps).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-6, 8-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gundling et al. (US 11176735 B2) in view of Choi et al. (JP 2021103283 A), User: 'barf' (BlenderGIS - How to drape ortho-photos onto a DEM/DSM), and Pancroma (Extracting DEMs from Topographic Maps), hereinafter Gundling, Choi, Barf, and Pancroma respectively.
Regarding claim 1, Gundling teaches a method performed by one or more processors (Col. 13 lines 41-43, Col. 18 lines 66-67, Col. 19 lines 1-7 – “the mesh generation module 250 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations)…The method 600 may be performed by any suitable system, apparatus, or device with respect to the simulated driving environment. For example…the mesh generation module 250 of FIG. 2, or the computing system 500 of FIG. 5 (e.g., as directed by one or more modules) may perform one or more of the operations associated with the method 600”; Note: the method can be performed by the mesh generation module, which itself is implemented by a processor), the method comprising:
obtaining road map data of a specific area (Col. 5 lines 18-23, Col. 7 lines 22-24 – “the map data 102 may include data describing a geographic area. The map data 102 may describe a geographic area in the real world. The map data 102 may include multiple layers of data describing the same geographic area. For example, the map data 102 may include road data 103 describing roads…the filtering module 110 may be configured to select (which may include filtering) road data 103 of the map data 102”; Note: the filtering module obtains road map data of a specific geographic area);
generating, based on point cloud data of the road map, a first mesh associated with a road in the specific area (Col. 5 lines 43-45, Col. 17 lines 39-46 – “The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…additional meshes may be generated from a point cloud. For example, in some embodiments, meshes may also be generated between the first sidewalk mesh 410A and the first dashed white line mesh 420A, for example as a road mesh. For example, in some embodiments, a point cloud may include points between the first sidewalk mesh 410A and the first dashed white line mesh 420A corresponding to, for example, a road”; Note: a mesh is generated for a road based on a point cloud of map elements);
generating a second mesh associated with a terrain (Col. 11 lines 3-4, Col. 13 lines 60-62 – “the HD mapping data 212 may include road data, general area data, and terrain data…the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212”; Note: a mesh is generated for terrain data) of the specific area (Col. 5 lines 26-28 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area”);
and generating, by matching the first mesh and the second mesh, a virtual environment for verifying an autonomous driving service (Col. 15 lines 53-62, Col. 20 lines 5-10 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic…a three-dimensional environmental representation that represents the geographic area may be generated based on the filtered road data and the filtered general area data. The three-dimensional environmental representation may be configured as a simulation environment with respect to testing the autonomous-vehicle software”; Note: the simulation environment is equivalent to the virtual environment, and it tests autonomous vehicles. The simulation environment is generated by combining meshes of the roads and terrain).
Gundling does not teach generating, using a mobile mapping system, a road map of a specific area. However, Choi teaches generating, using a mobile mapping system, a road map (Paragraph 0006 – “a mobile surveying system called MMS (Mobile Mapping System) is used to create high-precision road maps”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Choi to generate a road map using a mobile mapping system because “High-definition road maps can also be used to collect precise location-based road event information…In addition, it can be used to exchange information between connected cars equipped with cameras, and to accurately determine the location of various road facilities and event information when collecting information from a variety of corporate-owned vehicles equipped with cameras” (Choi: Paragraph 0005). Additionally, mobile mapping systems provide quick and easy access to a road map.
Furthermore, Gundling modified by Choi does not teach receiving satellite orthoimagery corresponding to the specific area; generating, based on a digital topographic map corresponding to the specific area, digital elevation model information; nor generating, based on the digital elevation model information and the satellite orthoimagery, a second mesh. However, Barf teaches receiving satellite orthoimagery corresponding to the specific area (Page 2, 4 – “This tutorial will cover importing (GeoTIFF) LiDAR DEM/DSM products, and satellite terrain and aerial photos directly into blender…Ortho-photo import…Click Import georaster”; Note: it is implied that satellite orthoimagery is received because it is imported into the software); and generating, based on the digital elevation model information and the satellite orthoimagery, a mesh (Page 4 – “Ortho-photo import…Select the object to apply the raster to. (Ie; the DEM/DSM you imported previously)…To optimise the mesh, it is a good idea 'dissolve' redundant vertices”; Note: An ortho-photo is applied to a previously imported DEM, which generates a mesh). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to receive satellite orthoimagery and generate a mesh based on the digital elevation model information and satellite orthoimagery because both are useful for creating a detailed and textured mesh model. The digital elevation model provides the shape of the landscape, while the satellite orthoimagery provides the outward appearance of the landscape. Additionally, Gundling already suggests using elevation data and satellite data in creation of the mesh (Col. 5 lines 31-33 and 43-45, Col. 6 lines 24-25, Col. 17 lines 39-46 – “The map data 102 may include information found on a street map or information observable from an aerial photo or a satellite photo…The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…The terrain data 107 may include elevation data of the geographic area… meshes may be generated from a point cloud”). While Gundling does not suggest combining both types of data, it would have been beneficial to do so, like in Barf, for creating a better-quality mesh.
Gundling modified by Choi and Barf still does not teach generating, based on a digital topographic map corresponding to the specific area, digital elevation model information. However, Pancroma teaches generating, based on a digital topographic map corresponding to the specific area, digital elevation model information (Paragraph 3 on Page 1, Paragraph 4 on Page 4 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data. This is done using a multi step process. The raster elevation contours must first be converted to vectors. Next, the vector contours must be “tagged” with their corresponding elevation values. The tagged vector data is then transferred to a superimposed grid by an interpolation algorithm…Once you have all your contour lines tagged, the next step is to convert the tagged image to a USGS DEM file by selecting ‘File’, ‘3D Data’, ‘Create 3D DEM File’”; Note: a digital elevation model, called DEM, is created from a topographic map). Since Gundling already has a topographic map (Col. 6 lines 25-27 – “The terrain data 107 may include elevation data of the geographic area, for example, a topographical map of the geographic area”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to generate a digital elevation model from a topographic map because digital elevation models are commonly used models to map the ground and are beneficial for analyzing and representing terrain.
Regarding claim 2, Gundling in view of Choi, Barf, and Pancroma teaches the method according to claim 1. Gundling further teaches wherein the first mesh comprises a mesh associated with a vehicle travel path and a mesh associated with a lane of the vehicle travel path (Col. 15 lines 57-62 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with road edges, which are equivalent to vehicle travel paths. There is also a mesh associated with lane lines, which are equivalent to lanes of the vehicle travel path).
Regarding claim 4, Gundling in view of Choi, Barf, and Pancroma teaches the method according to claim 1. Gundling further teaches generating the virtual environment by matching a mesh associated with a structure in the specific area with the first mesh and the second mesh (Col. 5 lines 26-28, Col. 15 lines 57-62 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area…The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with buildings, which is equivalent to the mesh associated with a structure. The mesh of buildings is combined with meshes of the road and the terrain, all of which are in the same geographic area), and wherein the mesh associated with the structure in the specific area is generated based on three-dimensional (3D) object model information of the structure and location information of the structure (Col. 5 lines 39-51, Col. 6 lines 17-18, Col. 11 lines 3-5, Col. 13 lines 60-65 – “each data point in the point cloud may include positional information (e.g., x-, y-, and z-coordinates on three-dimensional axes) as well as intensity information (e.g., reflective information). The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data… The general area data 105 may include information about buildings…the HD mapping data 212 may include road data, general area data, and terrain data such as that described above with respect to FIG. 1….the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212. In these and other embodiments, a mesh may include a 3-D representation of the points in the point cloud with polygons connecting the data points”; Note: HD mapping data includes general area data, which includes building data. Point clouds are generated based on HD mapping data, and thus based on building data. The meshes are then generated based on point clouds of buildings, which is equivalent to 3D object model information. Additionally, point clouds contain location information, as each point has coordinates).
Regarding claim 5, Gundling teaches a system (Col. 17 line 58 – “The computing system 500”) comprising:
a memory storing one or more computer-readable programs (Col. 17 lines 58-61, Col. 18 lines 13-15, Col. 18 lines 33-36 – “The computing system 500 may include a processor 502, a memory 504, and a data storage 506…the processor 502 may be configured to interpret and/or execute program instructions and/or process data stored in the memory 504…The memory 504 and the data storage 506 may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon”);
and one or more processors coupled to the memory, wherein the one or more computer-readable programs comprise instructions that, when executed by the one or more processors (Col. 17 lines 58-61, Col. 18 lines 13-15 – “The computing system 500 may include a processor 502, a memory 504, and a data storage 506. The processor 502, the memory 504, and the data storage 506 may be communicatively coupled…the processor 502 may be configured to interpret and/or execute program instructions and/or process data stored in the memory 504”), cause the system to:
obtain road map data of a specific area (Col. 5 lines 18-23, Col. 7 lines 22-24 – “the map data 102 may include data describing a geographic area. The map data 102 may describe a geographic area in the real world. The map data 102 may include multiple layers of data describing the same geographic area. For example, the map data 102 may include road data 103 describing roads…the filtering module 110 may be configured to select (which may include filtering) road data 103 of the map data 102”; Note: the filtering module obtains road map data of a specific geographic area);
generate, based on point cloud data of the road map, a first mesh associated with a road in the specific area (Col. 5 lines 43-45, Col. 17 lines 39-46 – “The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…additional meshes may be generated from a point cloud. For example, in some embodiments, meshes may also be generated between the first sidewalk mesh 410A and the first dashed white line mesh 420A, for example as a road mesh. For example, in some embodiments, a point cloud may include points between the first sidewalk mesh 410A and the first dashed white line mesh 420A corresponding to, for example, a road”; Note: a mesh is generated for a road based on a point cloud of map elements);
generate a second mesh associated with a terrain (Col. 11 lines 3-4, Col. 13 lines 60-62 – “the HD mapping data 212 may include road data, general area data, and terrain data…the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212”; Note: a mesh is generated for terrain data) of the specific area (Col. 5 lines 26-28 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area”);
and generate, by matching the first mesh and the second mesh, a virtual environment for verifying an autonomous driving service (Col. 15 lines 53-62, Col. 20 lines 5-10 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic…a three-dimensional environmental representation that represents the geographic area may be generated based on the filtered road data and the filtered general area data. The three-dimensional environmental representation may be configured as a simulation environment with respect to testing the autonomous-vehicle software”; Note: the simulation environment is equivalent to the virtual environment, and it tests autonomous vehicles. The simulation environment is generated by combining meshes of the roads and terrain).
Gundling does not teach a communication device; nor generating, using a mobile mapping system, a road map of a specific area. However, Choi teaches a communication device (Paragraph 0043 – “The communication unit (130) communicates with the map generation server (200) and transmits the high-precision road map generated by the map generation device (100)”; Note: the map generation device is equivalent to the communication device, as it performs communication using a communication unit); and generating, using a mobile mapping system, a road map (Paragraph 0006 – “a mobile surveying system called MMS (Mobile Mapping System) is used to create high-precision road maps”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Choi to generate a road map using a mobile mapping system because “High-definition road maps can also be used to collect precise location-based road event information…In addition, it can be used to exchange information between connected cars equipped with cameras, and to accurately determine the location of various road facilities and event information when collecting information from a variety of corporate-owned vehicles equipped with cameras” (Choi: Paragraph 0005). Additionally, mobile mapping systems provide quick and easy access to a road map. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Choi to have a communication device for the benefit of sending the road map to the server (Choi: Paragraph 0043), which would allow the map to be accessed and utilized by multiple users.
Furthermore, Gundling modified by Choi does not teach a display; receiving satellite orthoimagery corresponding to the specific area; generating, based on a digital topographic map corresponding to the specific area, digital elevation model information; nor generating, based on the digital elevation model information and the satellite orthoimagery, a second mesh. However, Barf teaches a display (Images on Page 2 – The images show a display; see screenshot below); as well as, receiving satellite orthoimagery corresponding to the specific area (Page 2, 4 – “This tutorial will cover importing (GeoTIFF) LiDAR DEM/DSM products, and satellite terrain and aerial photos directly into blender…Ortho-photo import…Click Import georaster”; Note: it is implied that satellite orthoimagery is received because it is imported into the software); and generating, based on the digital elevation model information and the satellite orthoimagery, a mesh (Page 4 – “Ortho-photo import…Select the object to apply the raster to. (Ie; the DEM/DSM you imported previously)…To optimise the mesh, it is a good idea 'dissolve' redundant vertices”; Note: An ortho-photo is applied to a previously imported DEM, which generates a mesh).
PNG
media_image1.png
692
1119
media_image1.png
Greyscale
Screenshot of Display (taken from Barf)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to have a display for the benefit of allowing users to view the virtual environment. It would not be possible to view the virtual environment without one. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to receive satellite orthoimagery and generate a mesh based on the digital elevation model information and satellite orthoimagery because both are useful for creating a detailed and textured mesh model. The digital elevation model provides the shape of the landscape, while the satellite orthoimagery provides the outward appearance of the landscape. Additionally, Gundling already suggests using elevation data and satellite data in creation of the mesh (Col. 5 lines 31-33 and 43-45, Col. 6 lines 24-25, Col. 17 lines 39-46 – “The map data 102 may include information found on a street map or information observable from an aerial photo or a satellite photo…The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…The terrain data 107 may include elevation data of the geographic area… meshes may be generated from a point cloud”). While Gundling does not suggest combining both types of data, it would have been beneficial to do so, like in Barf, for creating a better-quality mesh.
Gundling modified by Choi and Barf still does not teach generating, based on a digital topographic map corresponding to the specific area, digital elevation model information. However, Pancroma teaches generating, based on a digital topographic map corresponding to the specific area, digital elevation model information (Paragraph 3 on Page 1, Paragraph 4 on Page 4 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data. This is done using a multi step process. The raster elevation contours must first be converted to vectors. Next, the vector contours must be “tagged” with their corresponding elevation values. The tagged vector data is then transferred to a superimposed grid by an interpolation algorithm…Once you have all your contour lines tagged, the next step is to convert the tagged image to a USGS DEM file by selecting ‘File’, ‘3D Data’, ‘Create 3D DEM File’”; Note: a digital elevation model, called DEM, is created from a topographic map). Since Gundling already has a topographic map (Col. 6 lines 25-27 – “The terrain data 107 may include elevation data of the geographic area, for example, a topographical map of the geographic area”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to generate a digital elevation model from a topographic map because digital elevation models are commonly used models to map the ground and are beneficial for analyzing and representing terrain.
Regarding claim 6, Gundling in view of Choi, Barf, and Pancroma teaches the system according to claim 5. Gundling further teaches wherein the first mesh comprises a mesh associated with a vehicle travel path and a mesh associated with a lane of the vehicle travel path (Col. 15 lines 57-62 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with road edges, which are equivalent to vehicle travel paths. There is also a mesh associated with lane lines, which are equivalent to lanes of the vehicle travel path).
Regarding claim 8, Gundling in view of Choi, Barf, and Pancroma teaches the system according to claim 5. Gundling further teaches wherein the instructions, when executed by the one or more processors (Col. 18 lines 53-56 – “Computer-executable instructions may include, for example, instructions and data configured to cause the processor 502 to perform a certain operation or group of operations”), cause the system to: generate, based on three-dimensional (3D) object model information of a structure and location information of the structure, a mesh associated with the structure in the specific area (Col. 5 lines 39-51, Col. 6 lines 17-18, Col. 11 lines 3-5, Col. 13 lines 60-65 – “each data point in the point cloud may include positional information (e.g., x-, y-, and z-coordinates on three-dimensional axes) as well as intensity information (e.g., reflective information). The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data… The general area data 105 may include information about buildings…the HD mapping data 212 may include road data, general area data, and terrain data such as that described above with respect to FIG. 1….the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212. In these and other embodiments, a mesh may include a 3-D representation of the points in the point cloud with polygons connecting the data points”; Note: HD mapping data includes general area data, which includes building data. Point clouds are generated based on HD mapping data, and thus based on building data. The meshes are then generated based on point clouds of buildings, which is equivalent to 3D object model information. Additionally, point clouds contain location information, as each point has coordinates); and generate the virtual environment by matching the mesh associated with the structure in the specific area with the first mesh and the second mesh (Col. 5 lines 26-28, Col. 15 lines 57-62 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area…The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with buildings, which is equivalent to the mesh associated with a structure. The mesh of buildings is combined with meshes of the road and the terrain, all of which are in the same geographic area).
Regarding claim 9, Gundling in view of Choi, Barf, and Pancroma teaches the method according to claim 1. Gundling does not teach wherein the generating, based on the digital elevation model information and the satellite orthoimagery, the second mesh associated with the terrain of the specific area comprises matching the digital elevation model information and the satellite orthoimagery. However, Barf teaches matching the digital elevation model information and the satellite orthoimagery (Page 3-4 – “when we import any geo-spatial data we must harmonise into one CRS: the native Cartesian system in meters that AC uses for it's physics and visuals…Ortho-photo import…Select the object to apply the raster to. (Ie; the DEM/DSM you imported previously)…The scene geo-referencing items should automatically be set…To optimise the mesh, it is a good idea 'dissolve' redundant vertices”; Note: the DEM and ortho-photo are georeferenced, meaning they are matched with the same coordinate system). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to match the digital elevation model information and satellite orthoimagery because if they did not match, then the mesh would not be accurate and may appear distorted. They must align and correspond to the same coordinate system in order for the mesh to be properly generated.
Regarding claim 10, Gundling in view of Choi, Barf, and Pancroma teaches the method according to claim 1. Gundling does not teach wherein the digital topographic map comprises contour information, and wherein the generating, based on the digital topographic map corresponding to the specific area, the digital elevation model information comprises converting the contour information to the digital elevation model information. However, Pancroma teaches wherein the digital topographic map comprises contour information (Paragraph 3 on Page 1 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data…The raster elevation contours must first be converted to vectors”; Note: there are elevation contours on the topographic map), and wherein the generating, based on the digital topographic map corresponding to the specific area, the digital elevation model information comprises converting the contour information to the digital elevation model information (Paragraph 3 on Page 1, Paragraph 4 on Page 4 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data. This is done using a multi step process. The raster elevation contours must first be converted to vectors. Next, the vector contours must be “tagged” with their corresponding elevation values. The tagged vector data is then transferred to a superimposed grid by an interpolation algorithm…Once you have all your contour lines tagged, the next step is to convert the tagged image to a USGS DEM file by selecting ‘File’, ‘3D Data’, ‘Create 3D DEM File’”; Note: contours are converted and used to create a digital elevation model file). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to have the digital topographic map comprise contour information because contours are inherently part of topographic maps. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to convert the contours into digital elevation model information because the contours are what represent elevation within a topographical map. When using only a topographical map to create a digital elevation model, the contours are crucial for providing the elevation data for the model.
Regarding claim 11, Gundling in view of Choi, Barf, and Pancroma teaches the method according to claim 1. Gundling further teaches wherein the first mesh associated with the road in the specific area indicates a color, corresponding to the road, associated with a type of lane (Col. 14 11-20 – “after generation of a mesh, a texture may be applied to the mesh. In these and other embodiments, the texture may include colors, patterns, surface details, and/or other information to transform the mesh into a more realistic 3-D representation of an object. In these and other embodiments, different data topics may correspond with different textures. For example, in some embodiments, yellow lane lines and white lane lines may represent different data topics and may be associated with different textures”; Note: the mesh may have a texture where the road has lanes of different colors depending on the type of lane).
Regarding claim 12, Gundling in view of Choi, Barf, and Pancroma teaches the system of claim 5. Gundling does not teach wherein the instructions, when executed by the one or more processors, cause the system to generate, based on the digital elevation model information and the satellite orthoimagery, the second mesh associated with the terrain of the specific area by causing the system to match the digital elevation model information and the satellite orthoimagery. However, Barf teaches matching the digital elevation model information and the satellite orthoimagery (Page 3-4 – “when we import any geo-spatial data we must harmonise into one CRS: the native Cartesian system in meters that AC uses for it's physics and visuals…Ortho-photo import…Select the object to apply the raster to. (Ie; the DEM/DSM you imported previously)…The scene geo-referencing items should automatically be set…To optimise the mesh, it is a good idea 'dissolve' redundant vertices”; Note: the DEM and ortho-photo are georeferenced, meaning they are matched with the same coordinate system). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to match the digital elevation model information and satellite orthoimagery because if they did not match, then the mesh would not be accurate and may appear distorted. They must align and correspond to the same coordinate system in order for the mesh to be properly generated.
Regarding claim 13, Gundling in view of Choi, Barf, and Pancroma teaches the system of claim 5. Gundling does not teach wherein the digital topographic map comprises contour information, and wherein the instructions, when executed by the one or more processors, cause the system to generate, based on the digital topographic map corresponding to the specific area, the digital elevation model information by causing the system to convert the contour information to the digital elevation model information. However, Pancroma teaches wherein the digital topographic map comprises contour information (Paragraph 3 on Page 1 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data…The raster elevation contours must first be converted to vectors”; Note: there are elevation contours on the topographic map), and converting the contour information to the digital elevation model information (Paragraph 3 on Page 1, Paragraph 4 on Page 4 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data. This is done using a multi step process. The raster elevation contours must first be converted to vectors. Next, the vector contours must be “tagged” with their corresponding elevation values. The tagged vector data is then transferred to a superimposed grid by an interpolation algorithm…Once you have all your contour lines tagged, the next step is to convert the tagged image to a USGS DEM file by selecting ‘File’, ‘3D Data’, ‘Create 3D DEM File’”; Note: contours are converted and used to create a digital elevation model file). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to have the digital topographic map comprise contour information because contours are inherently part of topographic maps. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to convert the contours into digital elevation model information because the contours are what represent elevation within a topographical map. When using only a topographical map to create a digital elevation model, the contours are crucial for providing the elevation data for the model.
Regarding claim 14, Gundling in view of Choi, Barf, and Pancroma teaches the system of claim 5. Gundling further teaches wherein the first mesh associated with the road in the specific area indicates a color, corresponding to the road, associated with a type of lane (Col. 14 11-20 – “after generation of a mesh, a texture may be applied to the mesh. In these and other embodiments, the texture may include colors, patterns, surface details, and/or other information to transform the mesh into a more realistic 3-D representation of an object. In these and other embodiments, different data topics may correspond with different textures. For example, in some embodiments, yellow lane lines and white lane lines may represent different data topics and may be associated with different textures”; Note: the mesh may have a texture where the road has lanes of different colors depending on the type of lane).
Regarding claim 15, Gundling teaches one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a computing device, cause the computing device to (Col. 18 lines 33-56 – “The memory 504 and the data storage 506 may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon…such computer-readable storage media may include tangible or non-transitory computer-readable storage media…Computer-executable instructions may include, for example, instructions and data configured to cause the processor 502 to perform a certain operation or group of operations”):
obtain road map data of a specific area (Col. 5 lines 18-23, Col. 7 lines 22-24 – “the map data 102 may include data describing a geographic area. The map data 102 may describe a geographic area in the real world. The map data 102 may include multiple layers of data describing the same geographic area. For example, the map data 102 may include road data 103 describing roads…the filtering module 110 may be configured to select (which may include filtering) road data 103 of the map data 102”; Note: the filtering module obtains road map data of a specific geographic area);
generate, based on point cloud data of the road map, a first mesh associated with a road in the specific area (Col. 5 lines 43-45, Col. 17 lines 39-46 – “The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…additional meshes may be generated from a point cloud. For example, in some embodiments, meshes may also be generated between the first sidewalk mesh 410A and the first dashed white line mesh 420A, for example as a road mesh. For example, in some embodiments, a point cloud may include points between the first sidewalk mesh 410A and the first dashed white line mesh 420A corresponding to, for example, a road”; Note: a mesh is generated for a road based on a point cloud of map elements);
generate a second mesh associated with a terrain (Col. 11 lines 3-4, Col. 13 lines 60-62 – “the HD mapping data 212 may include road data, general area data, and terrain data…the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212”; Note: a mesh is generated for terrain data) of the specific area (Col. 5 lines 26-28 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area”);
and generate, by matching the first mesh and the second mesh, a virtual environment for verifying an autonomous driving service (Col. 15 lines 53-62, Col. 20 lines 5-10 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic…a three-dimensional environmental representation that represents the geographic area may be generated based on the filtered road data and the filtered general area data. The three-dimensional environmental representation may be configured as a simulation environment with respect to testing the autonomous-vehicle software”; Note: the simulation environment is equivalent to the virtual environment, and it tests autonomous vehicles. The simulation environment is generated by combining meshes of the roads and terrain).
Gundling does not teach generating, using a mobile mapping system, a road map of a specific area. However, Choi teaches generating, using a mobile mapping system, a road map (Paragraph 0006 – “a mobile surveying system called MMS (Mobile Mapping System) is used to create high-precision road maps”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Choi to generate a road map using a mobile mapping system because “High-definition road maps can also be used to collect precise location-based road event information…In addition, it can be used to exchange information between connected cars equipped with cameras, and to accurately determine the location of various road facilities and event information when collecting information from a variety of corporate-owned vehicles equipped with cameras” (Choi: Paragraph 0005). Additionally, mobile mapping systems provide quick and easy access to a road map.
Furthermore, Gundling modified by Choi does not teach receiving satellite orthoimagery corresponding to the specific area; generating, based on a digital topographic map corresponding to the specific area, digital elevation model information; nor generating, based on the digital elevation model information and the satellite orthoimagery, a second mesh. However, Barf teaches receiving satellite orthoimagery corresponding to the specific area (Page 2, 4 – “This tutorial will cover importing (GeoTIFF) LiDAR DEM/DSM products, and satellite terrain and aerial photos directly into blender…Ortho-photo import…Click Import georaster”; Note: it is implied that satellite orthoimagery is received because it is imported into the software); and generating, based on the digital elevation model information and the satellite orthoimagery, a mesh (Page 4 – “Ortho-photo import…Select the object to apply the raster to. (Ie; the DEM/DSM you imported previously)…To optimise the mesh, it is a good idea 'dissolve' redundant vertices”; Note: An ortho-photo is applied to a previously imported DEM, which generates a mesh). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Barf to receive satellite orthoimagery and generate a mesh based on the digital elevation model information and satellite orthoimagery because both are useful for creating a detailed and textured mesh model. The digital elevation model provides the shape of the landscape, while the satellite orthoimagery provides the outward appearance of the landscape. Additionally, Gundling already suggests using elevation data and satellite data in creation of the mesh (Col. 5 lines 31-33 and 43-45, Col. 6 lines 24-25, Col. 17 lines 39-46 – “The map data 102 may include information found on a street map or information observable from an aerial photo or a satellite photo…The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data…The terrain data 107 may include elevation data of the geographic area… meshes may be generated from a point cloud”). While Gundling does not suggest combining both types of data, it would have been beneficial to do so, like in Barf, for creating a better-quality mesh.
Gundling modified by Choi and Barf still does not teach generating, based on a digital topographic map corresponding to the specific area, digital elevation model information. However, Pancroma teaches generating, based on a digital topographic map corresponding to the specific area, digital elevation model information (Paragraph 3 on Page 1, Paragraph 4 on Page 4 – “Creation of a DEM from a topographic map requires that the elevation contours on the topo map be somehow converted to xyz data. This is done using a multi step process. The raster elevation contours must first be converted to vectors. Next, the vector contours must be “tagged” with their corresponding elevation values. The tagged vector data is then transferred to a superimposed grid by an interpolation algorithm…Once you have all your contour lines tagged, the next step is to convert the tagged image to a USGS DEM file by selecting ‘File’, ‘3D Data’, ‘Create 3D DEM File’”; Note: a digital elevation model, called DEM, is created from a topographic map). Since Gundling already has a topographic map (Col. 6 lines 25-27 – “The terrain data 107 may include elevation data of the geographic area, for example, a topographical map of the geographic area”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gundling to incorporate the teachings of Pancroma to generate a digital elevation model from a topographic map because digital elevation models are commonly used models to map the ground and are beneficial for analyzing and representing terrain.
Regarding claim 16, Gundling in view of Choi, Barf, and Pancroma teaches one or more non-transitory computer-readable media of claim 15. Gundling further teaches wherein the first mesh comprises a mesh associated with a vehicle travel path and a mesh associated with a lane of the vehicle travel path (Col. 15 lines 57-62 – “The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with road edges, which are equivalent to vehicle travel paths. There is also a mesh associated with lane lines, which are equivalent to lanes of the vehicle travel path).
Regarding claim 18, Gundling in view of Choi, Barf, and Pancroma teaches one or more non-transitory computer-readable media of claim 15. Gundling further teaches wherein the instructions, when executed by the one or more processors (Col. 18 lines 53-56 – “Computer-executable instructions may include, for example, instructions and data configured to cause the processor 502 to perform a certain operation or group of operations”), cause the computing device to: generate the virtual environment by matching the mesh associated with a structure in the specific area with the first mesh and the second mesh (Col. 5 lines 26-28, Col. 15 lines 57-62 – “The road data 103, the general area data 105, and the terrain data 107 may all describe the same geographic area…The simulation environment 260 may include a combination of the meshes generated for the different data topics to represent the corresponding environment and its elements such as a variety of lane lines, curbs, road edges, buildings, trees, and/or other information associated with autonomous vehicles and traffic”; Note: there is a mesh associated with buildings, which is equivalent to the mesh associated with a structure. The mesh of buildings is combined with meshes of the road and the terrain, all of which are in the same geographic area); and wherein the mesh associated with the structure in the specific area is generated, based on three-dimensional (3D) object model information of the structure and location information of the structure (Col. 5 lines 39-51, Col. 6 lines 17-18, Col. 11 lines 3-5, Col. 13 lines 60-65 – “each data point in the point cloud may include positional information (e.g., x-, y-, and z-coordinates on three-dimensional axes) as well as intensity information (e.g., reflective information). The points in the point cloud may correspond to elements in the geographic setting associated with the HD map data… The general area data 105 may include information about buildings…the HD mapping data 212 may include road data, general area data, and terrain data such as that described above with respect to FIG. 1….the mesh generation module 250 may generate a mesh for each distinct instance of each data topic of the point cloud of the HD mapping data 212. In these and other embodiments, a mesh may include a 3-D representation of the points in the point cloud with polygons connecting the data points”; Note: HD mapping data includes general area data, which includes building data. Point clouds are generated based on HD mapping data, and thus based on building data. The meshes are then generated based on point clouds of buildings, which is equivalent to 3D object model information. Additionally, point clouds contain location i