DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is reply to the Application Number 18/494,097 filed on 10/25/2023
Claims 1 – 20 are currently pending and have been examined
This action is made NON-FINAL
Information Disclosure Statement
The information disclosure statements filed 10/25/2023, 05/02/2024, 06/21/2024 and 08/11/2025 have been received and considered.
Specification
The disclosure is objected to because of the following informalities: In the specification, paragraph 0060, line 10 states a duplicate “distance” in the limitation of “which adjusts a weighting of the line type distance distance”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 – 3, 11, 12, 18 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhou et al. (Lane Information Extraction for High Definition Maps Using Crowdsourced Data).
Regarding claim 1, Zhou teaches a system for resolving discrepancies in map data, the system comprising: (Zhou: Page 7780, Abstract: “The proposed method is quantitatively evaluated against a real-world HD map produced by a mobile mapping vehicle. The experimental results show that more than 80% of the extracted lane markings meet the accuracy requirements of HD maps. In conclusion, our method can be used as a low-cost and efficient approach for updating the lane information in HD maps for autonomous vehicles.”)
one or more central computers in wireless communication with one or more vehicles, and wherein the one or more central computers are programmed to: (Zhou: Page 7780, I. Introduction, paragraph 1: “The data collected in this manner are called crowdsourced vehicle data. They are wirelessly transmitted back to the HD map cloud center. Based on these data, a cloud computer can then estimate the lane information to update the HD map.” )
receive a first map dataset and a second map dataset, wherein both the first map dataset and the second map dataset represent a predefined geographical area; (Zhou: Page 7782, D. Extraction From Road Images, paragraph 2: “Compared to traditional methods, the deep learning based method can learn more road features. With that said, the deep learning-based road semantic mapping is worthy of exploration [30]. However, visual sensors are usually sensitive to light conditions, and the accuracy of the lane information collected from road images is unstable. To improve this accuracy, a large number of crowdsourced data on the same road should be collected to extract the lane information.”)
receive a plurality of crowdsourced map datasets, wherein each of the plurality of crowdsourced map datasets represents the predefined geographical area; (Zhou: Page 7782, D. Extraction From Road Images, paragraph 3: “Table I summarizes all of the abovementioned methods with their own merits and demerits. This study extracts lane information from road images collected by multiple crowdsourced vehicles.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the HD map is updated based on the lane information of the crowdsourced vehicles, therefore interpreted as crowdsourced map datasets)
compare each of the plurality of crowdsourced map datasets with the first map dataset and the second map dataset to determine one or more common lane lines; and
determine a fused map dataset based at least in part on the first map dataset, the second map dataset, the plurality of crowdsourced map datasets, and the one or more common lane lines. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”; Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information. Considering that the position and speed of the vehicle are continuously changing when it is driven on the road, a B-spline was utilized herein to fit the lane information. The B-spline represents the geometrical information for the lane via a set of control points and knots that can reduce the number of control points as long as the fitting accuracy is maintained. A gradual fitting algorithm was employed to determine the control points and the B-spline parameters in accordance with the main points. This algorithm consisted of three steps: (1) calculation of the initial fitting vertices; (2) fitting of the initial B-spline; and (3) gradual optimization.”,
Supplemental Note: the lane lines are compared and then placed in a fused map. The lane information of an HD map includes points which are associated with the lane lines found in the crowd sourced images)
Regarding claim 2, Zhou teaches wherein the predefined geographical area is a segment of a roadway containing one or more lane lines (Zhou: Page 7783, B. Projection From Perspective Space into 3D Space, paragraph 2: “Fig. 3(b) illustrates three coordinate systems, namely the local GNSS, camera, and pixel coordinate systems, in the proposed method. The origins of the local GNSS and camera coordinate systems were the GNSS receiver and the camera, respectively. The axis directions of the two coordinate systems were consistent with those of the WGS84 world coordinate system. The detected lane markings are described herein in terms of the pixel coordinate.”,
Supplemental Note: as shown below in Fig. 3, two lanes can be detected by the crowdsourced vehicles)
PNG
media_image1.png
591
610
media_image1.png
Greyscale
Regarding claim 3, Zhou teaches wherein to compare each of the plurality of crowdsourced map datasets with the first map dataset and the second map dataset, the one or more central computers are further programmed to:
generate a plurality of aligned map datasets, wherein the plurality of aligned map datasets includes a first subset and a second subset, wherein the first subset is generated by aligning each of the plurality of crowdsourced map datasets with the first map dataset by executing a map-matching registration algorithm, and wherein the second subset is generated by aligning each of the plurality of crowdsourced map datasets with the second map dataset by executing the map-matching registration algorithm; (Zhou, Page 7783, B. Projection From Perspective Space Into 3D Space, paragraphs 1 – 2 : “In this procedure, the lane markings extracted from the images were projected from a perspective space into a 3D space. Depth information is lost in single images. Hence, two assumptions were made to estimate the local geometrical parameters of the road lanes: 1) the road plane was flat; and 2) the camera was rigidly connected to the position device. Fig. 3(a) is an installation diagram of the sensors. The distance between the camera and the GNSS receiver was close and can be obtained by the measurement tool. Fig. 3(b) illustrates three coordinate systems, namely the local GNSS, camera, and pixel coordinate systems, in the proposed method. The origins of the local GNSS and camera coordinate systems were the GNSS receiver and the camera, respectively. The axis directions of the two coordinate systems were consistent with those of the WGS84 world coordinate system.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the multiple images are compared based on the coordinate information and then are placed in a 3D space. Each of the images being placed in a 3D environment consistent of a world coordinate system. These are then matched with the HD map to update it, thus interpreted as the claimed aligned maps)
and determine the one or more common lane lines based at least in part on the plurality of aligned map datasets. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”)
Regarding claim 11, Zhou teaches a method for resolving discrepancies in map data, the method comprising: (Zhou: Page 7780, Abstract: “The proposed method is quantitatively evaluated against a real-world HD map produced by a mobile mapping vehicle. The experimental results show that more than 80% of the extracted lane markings meet the accuracy requirements of HD maps. In conclusion, our method can be used as a low-cost and efficient approach for updating the lane information in HD maps for autonomous vehicles.”)
comparing each of a plurality of crowdsourced map datasets with a first map dataset and a second map dataset to determine one or more common lane lines using one or more central computers, (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the lane lines are compared and then placed in a fused map)
wherein the plurality of crowdsourced map datasets, (Zhou: Page 7782, D. Extraction From Road Images, paragraph 3: “Table I summarizes all of the abovementioned methods with their own merits and demerits. This study extracts lane information from road images collected by multiple crowdsourced vehicles.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the HD map is updated based on the lane information of the crowdsourced vehicles, therefore interpreted as crowdsourced map datasets)
the first map dataset, and the second map dataset represent a predefined geographical area, (Zhou: Page 7782, D. Extraction From Road Images, paragraph 2: “Compared to traditional methods, the deep learning based method can learn more road features. With that said, the deep learning-based road semantic mapping is worthy of exploration [30]. However, visual sensors are usually sensitive to light conditions, and the accuracy of the lane information collected from road images is unstable. To improve this accuracy, a large number of crowdsourced data on the same road should be collected to extract the lane information.”)
and wherein each of the plurality of crowdsourced map datasets, the first map dataset, and the second map dataset includes a plurality of points representing one or more lane lines; and
determining a fused map dataset using the one or more central computers based at least in part on the first map dataset, the second map dataset, the plurality of crowdsourced map datasets, and the one or more common lane lines. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”; Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information. Considering that the position and speed of the vehicle are continuously changing when it is driven on the road, a B-spline was utilized herein to fit the lane information. The B-spline represents the geometrical information for the lane via a set of control points and knots that can reduce the number of control points as long as the fitting accuracy is maintained. A gradual fitting algorithm was employed to determine the control points and the B-spline parameters in accordance with the main points. This algorithm consisted of three steps: (1) calculation of the initial fitting vertices; (2) fitting of the initial B-spline; and (3) gradual optimization.”,
Supplemental Note: the lane lines are compared and then placed in a fused map. The lane information of an HD map includes points which are associated with the lane lines found in the crowd sourced images)
Regarding claim 12, Zhou teaches wherein comparing each of the plurality of crowdsourced map datasets with the first map dataset and the second map dataset further comprises:
generating a plurality of aligned map datasets using the one or more central computers, wherein the plurality of aligned map datasets includes a first subset and a second subset, wherein the first subset is generated by aligning each of the plurality of crowdsourced map datasets with the first map dataset by executing a map-matching registration algorithm, and wherein the second subset is generated by aligning each of the plurality of crowdsourced map datasets with the second map dataset by executing the map-matching registration algorithm; (Zhou, Page 7783, B. Projection From Perspective Space Into 3D Space, paragraphs 1 – 2 : “In this procedure, the lane markings extracted from the images were projected from a perspective space into a 3D space. Depth information is lost in single images. Hence, two assumptions were made to estimate the local geometrical parameters of the road lanes: 1) the road plane was flat; and 2) the camera was rigidly connected to the position device. Fig. 3(a) is an installation diagram of the sensors. The distance between the camera and the GNSS receiver was close and can be obtained by the measurement tool. Fig. 3(b) illustrates three coordinate systems, namely the local GNSS, camera, and pixel coordinate systems, in the proposed method. The origins of the local GNSS and camera coordinate systems were the GNSS receiver and the camera, respectively. The axis directions of the two coordinate systems were consistent with those of the WGS84 world coordinate system.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the multiple images are compared based on the coordinate information and then are placed in a 3D space. Each of the images being placed in a 3D environment consistent of a world coordinate system. These are then matched with the HD map to update it, thus interpreted as the claimed aligned maps)
and determining the one or more common lane lines using the one or more central computers based at least in part on the plurality of aligned map datasets. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”)
Regarding claim 18, Zhou teaches a system for resolving discrepancies in map data, the system comprising: (Zhou: Page 7780, Abstract: “The proposed method is quantitatively evaluated against a real-world HD map produced by a mobile mapping vehicle. The experimental results show that more than 80% of the extracted lane markings meet the accuracy requirements of HD maps. In conclusion, our method can be used as a low-cost and efficient approach for updating the lane information in HD maps for autonomous vehicles.”)
one or more central computers in wireless communication with one or more vehicles, wherein the one or more central computers are programmed to: (Zhou: Page 7780, I. Introduction, paragraph 1: “The data collected in this manner are called crowdsourced vehicle data. They are wirelessly transmitted back to the HD map cloud center. Based on these data, a cloud computer can then estimate the lane information to update the HD map.” )
receive a first map dataset and a second map dataset, wherein both the first map dataset and the second map dataset represent a predefined geographical area, and wherein the predefined geographical area is a segment of a roadway containing one or more lane lines; (Zhou: Page 7782, D. Extraction From Road Images, paragraph 2: “Compared to traditional methods, the deep learning based method can learn more road features. With that said, the deep learning-based road semantic mapping is worthy of exploration [30]. However, visual sensors are usually sensitive to light conditions, and the accuracy of the lane information collected from road images is unstable. To improve this accuracy, a large number of crowdsourced data on the same road should be collected to extract the lane information.”)
receive a plurality of crowdsourced map datasets, wherein each of the plurality of crowdsourced map datasets represents the predefined geographical area; (Zhou: Page 7782, D. Extraction From Road Images, paragraph 3: “Table I summarizes all of the abovementioned methods with their own merits and demerits. This study extracts lane information from road images collected by multiple crowdsourced vehicles.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the HD map is updated based on the lane information of the crowdsourced vehicles, therefore interpreted as crowdsourced map datasets)
generate a plurality of aligned map datasets, wherein the plurality of aligned map datasets includes a first subset and a second subset, wherein the first subset is generated by aligning each of the plurality of crowdsourced map datasets with the first map dataset by executing a map-matching registration algorithm, and wherein the second subset is generated by aligning each of the plurality of crowdsourced map datasets with the second map dataset by executing the map-matching registration algorithm; (Zhou, Page 7783, B. Projection From Perspective Space Into 3D Space, paragraphs 1 – 2 : “In this procedure, the lane markings extracted from the images were projected from a perspective space into a 3D space. Depth information is lost in single images. Hence, two assumptions were made to estimate the local geometrical parameters of the road lanes: 1) the road plane was flat; and 2) the camera was rigidly connected to the position device. Fig. 3(a) is an installation diagram of the sensors. The distance between the camera and the GNSS receiver was close and can be obtained by the measurement tool. Fig. 3(b) illustrates three coordinate systems, namely the local GNSS, camera, and pixel coordinate systems, in the proposed method. The origins of the local GNSS and camera coordinate systems were the GNSS receiver and the camera, respectively. The axis directions of the two coordinate systems were consistent with those of the WGS84 world coordinate system.”; Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the multiple images are compared based on the coordinate information and then are placed in a 3D space. Each of the images being placed in a 3D environment consistent of a world coordinate system. These are then matched with the HD map to update it, thus interpreted as the claimed aligned maps)
determine one or more common lane lines based at least in part on the plurality of aligned map datasets; (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”)
and determine a fused map dataset based at least in part on the first map dataset, the second map dataset, the plurality of crowdsourced map datasets, and the one or more common lane lines. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”; Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information. Considering that the position and speed of the vehicle are continuously changing when it is driven on the road, a B-spline was utilized herein to fit the lane information. The B-spline represents the geometrical information for the lane via a set of control points and knots that can reduce the number of control points as long as the fitting accuracy is maintained. A gradual fitting algorithm was employed to determine the control points and the B-spline parameters in accordance with the main points. This algorithm consisted of three steps: (1) calculation of the initial fitting vertices; (2) fitting of the initial B-spline; and (3) gradual optimization.”,
Supplemental Note: the lane lines are compared and then placed in a fused map. The lane information of an HD map includes points which are associated with the lane lines found in the crowd sourced images)
Regarding claim 19, Zhou teaches wherein to determine the one or more common lane lines, the one or more central computers are further programmed to:
identify a plurality of detected lane lines, wherein each of the plurality of detected lane lines is present in at least one of the first map dataset and the second map dataset;
determine a quantity of votes for each of the plurality of detected lane lines based at least in part on the plurality of detected lane lines, the plurality of crowdsourced map datasets, and the plurality of aligned map datasets; and
determine the one or more common lane lines based at least in part on the quantity of votes for each of the plurality of detected lane lines, wherein the one or more common lane lines includes one or more of the plurality of detected lane lines having greater than or equal to a predetermined quantity of votes. (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the specifications state in paragraph 0062 a vote is an indication that the detected lane line is present in one of the plurality of crowdsourced data bases. The lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The points found on both the images and the HD map are interpreted as votes as they indicate that the detected lane line is present in a HD map database. The main point is interpreted as the first quantity as that is the point that best matches with the HD map)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 – 7 and 13 - 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (Lane Information Extraction for High Definition Maps Using Crowdsourced Data), further in view of Atsmon et al. (WO 2011042876 A1).
Regarding claim 4, Zhou teaches wherein each of the first map dataset, the second map dataset, and the plurality of crowdsourced map datasets includes a plurality of points representing the one or more lane lines, and wherein to execute the map-matching registration algorithm for a first of the plurality of crowdsourced map datasets, the one or more central computers are further programmed to:
determine a plurality of associated point pairs, wherein a first point of each of the plurality of associated point pairs is one of the plurality of points in the first map dataset, and wherein a second point of each of the plurality of associated point pairs is one of the plurality of points in the first of the plurality of crowdsourced map datasets; and (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information. Considering that the position and speed of the vehicle are continuously changing when it is driven on the road, a B-spline was utilized herein to fit the lane information. The B-spline represents the geometrical information for the lane via a set of control points and knots that can reduce the number of control points as long as the fitting accuracy is maintained. A gradual fitting algorithm was employed to determine the control points and the B-spline parameters in accordance with the main points. This algorithm consisted of three steps: (1) calculation of the initial fitting vertices; (2) fitting of the initial B-spline; and (3) gradual optimization.”,
Supplemental Note: the lane information of an HD map includes points which are associated with the lane lines found in the crowd sourced images)
… between the first point and the second point of each of the plurality of associated point pairs. (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 – 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fitting points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point. In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
In sum, Zhou teaches wherein each of the first map dataset, the second map dataset, and the plurality of crowdsourced map datasets includes a plurality of points representing the one or more lane lines, and wherein to execute the map-matching registration algorithm for a first of the plurality of crowdsourced map datasets, the one or more central computers are further programmed to: determine a plurality of associated point pairs, wherein a first point of each of the plurality of associated point pairs is one of the plurality of points in the first map dataset, and wherein a second point of each of the plurality of associated point pairs is one of the plurality of points in the first of the plurality of crowdsourced map datasets; between the first point and the second point of each of the plurality of associated point pairs. Zhou however does not teach apply a transformation to the first of the plurality of crowdsourced map datasets to generate a first aligned map dataset of the first subset of the plurality of aligned map datasets, wherein the transformation is chosen to minimize a lateral offset and a color distance whereas Atsmon does.
Atsmon teaches apply a transformation to the first of the plurality of crowdsourced map datasets to generate a first aligned map dataset of the first subset of the plurality of aligned map datasets, wherein the transformation is chosen to minimize a lateral offset and a color distance (Atsmon: Abstract: “A method of analyzing images over time is provided herein. The method includes: capturing a plurality of images each associated with specified objects in specified locations such that a specified area is covered; specifying regions of interest (ROI) in each of the captured images; repeating the capturing with at least one of: a different location, a different orientation, and a different timing such that the captured images are associated with the specified covered area; and comparing the captured imaged produced in the capturing with the captured imaged produced in the repeating of the capturing to yield comparison between the captured objects by comparing specified ROI.”; Paragraph 0080: “Such transformation can even further use 250 data such as 258 and/or 260 to create estimated needed compensation transform such is there is a 5° deviation between two images of the same building it can correct the resulting keypoint accordingly. Using methods such as Haar wavelet transform. Comparing the color histograms of the object to other color histograms. The methods can be used separately, one after another or in parallel. In case a heavily computational method such as (a) is used it is advisable to use a GPU such as 310 to attain a reasonable response time and to run parallel algorithms.”,
Supplemental Note: deviations between two images can be identified by comparing colors)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Atsmon with a reasonable expectation of success. One of ordinary skill in the art acknowledges that lanes comes in different patterns and colors representing the boundaries and operation of the roadways. These markings provide an autonomous vehicle or driver visual aid on how to safely drive on the roadway. For this reason, the ability of Atsmon of identifying deviation between the colors in it’s images would be obvious to try by one with knowledge in the art with the lane detection and map updating system of Zhou. This will improve the system of Zhou in further identifying correct location of lane lines on the ground as now deviation between the colors of the lane lines can also be detected. This leads to a more accurate HD map to be used by these vehicles, thus increasing the usability and accuracy of the map.
Regarding claim 5, Zhou, as modified, teaches wherein the lateral offset is a total lateral distance between the first point and the second point of each of the plurality of associated point pairs and (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 – 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fitting points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point. In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
In sum, Zhou teaches wherein the lateral offset is a total lateral distance between the first point and the second point of each of the plurality of associated point pairs. Zhou however does not teach the color distance is a total difference in color between the first point and the second point of each of the plurality of associated point pairs whereas Atsmon does.
Atsmon teaches the color distance is a total difference in color between the first point and the second point of each of the plurality of associated point pairs. (Atsmon: Abstract: “A method of analyzing images over time is provided herein. The method includes: capturing a plurality of images each associated with specified objects in specified locations such that a specified area is covered; specifying regions of interest (ROI) in each of the captured images; repeating the capturing with at least one of: a different location, a different orientation, and a different timing such that the captured images are associated with the specified covered area; and comparing the captured imaged produced in the capturing with the captured imaged produced in the repeating of the capturing to yield comparison between the captured objects by comparing specified ROI.”; Paragraph 0080: “Such transformation can even further use 250 data such as 258 and/or 260 to create estimated needed compensation transform such is there is a 5° deviation between two images of the same building it can correct the resulting keypoint accordingly. Using methods such as Haar wavelet transform. Comparing the color histograms of the object to other color histograms. The methods can be used separately, one after another or in parallel. In case a heavily computational method such as (a) is used it is advisable to use a GPU such as 310 to attain a reasonable response time and to run parallel algorithms.”,
Supplemental Note: deviations between two images can be identified by comparing colors)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Atsmon with a reasonable expectation of success. Please refer to the rejection of claim 4 as both state the same language and therefore rejected under the same pretenses.
Regarding claim 6, Zhou, as modified, teaches wherein to determine the one or more common lane lines, the one or more central computers are further programmed to:
identify a plurality of detected lane lines, wherein each of the plurality of detected lane lines is present in at least one of the first map dataset and the second map dataset;
determine a quantity of votes for each of the plurality of detected lane lines based at least in part on the plurality of detected lane lines, the plurality of crowdsourced map datasets, and the plurality of aligned map datasets; and
determine the one or more common lane lines based at least in part on the quantity of votes for each of the plurality of detected lane lines, wherein the one or more common lane lines includes one or more of the plurality of detected lane lines having greater than or equal to a predetermined quantity of votes. (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the specifications state in paragraph 0062 a vote is an indication that the detected lane line is present in one of the plurality of crowdsourced data bases. The lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The points found on both the images and the HD map are interpreted as votes as they indicate that the detected lane line is present in a HD map database)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
Regarding claim 7, Zhou, as modified, teaches wherein to determine the quantity of votes for one of the plurality of detected lane lines, the one or more central computers are further programmed to:
determine a first quantity of the plurality of crowdsourced map datasets including the one of the plurality of detected lane lines based at least in part on the plurality of aligned map datasets; and determine the quantity of votes for the one of the plurality of detected lane lines, wherein the quantity of votes is the first quantity. (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the specifications state in paragraph 0062 a vote is an indication that the detected lane line is present in one of the plurality of crowdsourced data bases. The lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The points found on both the images and the HD map are interpreted as votes as they indicate that the detected lane line is present in a HD map database. The main point is interpreted as the first quantity as that is the point that best matches with the HD map)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
Regarding claim 13, Zhou, as modified, teaches wherein executing the map-matching registration algorithm for a first of the plurality of crowdsourced map datasets further comprises:
determining a plurality of associated point pairs using the one or more central computers, wherein a first point of each of the plurality of associated point pairs is one of the plurality of points in the first map dataset, and wherein a second point of each of the plurality of associated point pairs is one of the plurality of points in the first of the plurality of crowdsourced map datasets; and (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information. Considering that the position and speed of the vehicle are continuously changing when it is driven on the road, a B-spline was utilized herein to fit the lane information. The B-spline represents the geometrical information for the lane via a set of control points and knots that can reduce the number of control points as long as the fitting accuracy is maintained. A gradual fitting algorithm was employed to determine the control points and the B-spline parameters in accordance with the main points. This algorithm consisted of three steps: (1) calculation of the initial fitting vertices; (2) fitting of the initial B-spline; and (3) gradual optimization.”,
Supplemental Note: the lane information of an HD map includes points which are associated with the lane lines found in the crowd sourced images)
… between the first point and the second point of each of the plurality of associated point pairs. (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 – 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fitting points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point. In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
In sum, Zhou teaches wherein executing the map-matching registration algorithm for a first of the plurality of crowdsourced map datasets further comprises: determining a plurality of associated point pairs using the one or more central computers, wherein a first point of each of the plurality of associated point pairs is one of the plurality of points in the first map dataset, and wherein a second point of each of the plurality of associated point pairs is one of the plurality of points in the first of the plurality of crowdsourced map datasets; between the first point and the second point of each of the plurality of associated point pairs. Zhou however does not teach applying a transformation to the first of the plurality of crowdsourced map datasets to generate a first aligned map dataset of the first subset of the plurality of aligned map datasets using the one or more central computers, wherein the transformation is chosen to minimize a lateral offset and a color distance whereas Atsmon does.
Atsmon teaches applying a transformation to the first of the plurality of crowdsourced map datasets to generate a first aligned map dataset of the first subset of the plurality of aligned map datasets using the one or more central computers, wherein the transformation is chosen to minimize a lateral offset and a color distance (Atsmon: Abstract: “A method of analyzing images over time is provided herein. The method includes: capturing a plurality of images each associated with specified objects in specified locations such that a specified area is covered; specifying regions of interest (ROI) in each of the captured images; repeating the capturing with at least one of: a different location, a different orientation, and a different timing such that the captured images are associated with the specified covered area; and comparing the captured imaged produced in the capturing with the captured imaged produced in the repeating of the capturing to yield comparison between the captured objects by comparing specified ROI.”; Paragraph 0080: “Such transformation can even further use 250 data such as 258 and/or 260 to create estimated needed compensation transform such is there is a 5° deviation between two images of the same building it can correct the resulting keypoint accordingly. Using methods such as Haar wavelet transform. Comparing the color histograms of the object to other color histograms. The methods can be used separately, one after another or in parallel. In case a heavily computational method such as (a) is used it is advisable to use a GPU such as 310 to attain a reasonable response time and to run parallel algorithms.”,
Supplemental Note: deviations between two images can be identified by comparing colors)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Atsmon with a reasonable expectation of success. Please refer to the rejection of claim 4 as both state the same language and therefore rejected under the same pretenses.
Regarding claim 14, Zhou, as modified, teaches wherein determining the one or more common lane lines further comprises:
identifying a plurality of detected lane lines using the one or more central computers, wherein each of the plurality of detected lane lines is present in at least one of the first map dataset and the second map dataset;
determining a quantity of votes for each of the plurality of detected lane lines based at least in part on the plurality of detected lane lines, the plurality of crowdsourced map datasets, and the plurality of aligned map datasets using the one or more central computers; and
determining the one or more common lane lines based at least in part on the quantity of votes for each of the plurality of detected lane lines using the one or more central computers, wherein the one or more common lane lines includes one or more of the plurality of detected lane lines having greater than or equal to a predetermined quantity of votes. (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the specifications state in paragraph 0062 a vote is an indication that the detected lane line is present in one of the plurality of crowdsourced data bases. The lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The points found on both the images and the HD map are interpreted as votes as they indicate that the detected lane line is present in a HD map database)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
Regarding claim 15, Zhou, as modified, teaches wherein determining the quantity of votes for each of the plurality of detected lane lines further comprises:
determining a first quantity of the plurality of crowdsourced map datasets including the one of the plurality of detected lane lines using the one or more central computers based at least in part on the plurality of aligned map datasets; and
determining the quantity of votes for the one of the plurality of detected lane lines using the one or more central computers, wherein the quantity of votes is the first quantity. (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the specifications state in paragraph 0062 a vote is an indication that the detected lane line is present in one of the plurality of crowdsourced data bases. The lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The points found on both the images and the HD map are interpreted as votes as they indicate that the detected lane line is present in a HD map database. The main point is interpreted as the first quantity as that is the point that best matches with the HD map)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
Claims 8 – 10 and 16 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (Lane Information Extraction for High Definition Maps Using Crowdsourced Data) and Atsmon et al. (WO 2011042876 A1) as applied to independent claim 4 above for claims 8 – 10 and as applied to independent claim 13 above for claims 16 – 17, and further in view of Juan et al. (Lane Detection Using Histogram-Based Segmentation and Decision Trees).
Regarding claim 8, Zhou, as modified, teaches wherein to determine the fused map dataset, the one or more central computers are further programmed to:
based on the first subset of the plurality of aligned map datasets, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
based on the second subset of the plurality of aligned map datasets, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
and determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the lane lines are compared and then placed in a fused map)
In sum, Zhou teaches wherein to determine the fused map dataset, the one or more central computers are further programmed to: based on the first subset of the plurality of aligned map datasets, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; based on the second subset of the plurality of aligned map datasets, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; and determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. Zhou however does not teach to generate a first and ssecond plurality of lateral offset histograms whereas Juan does.
Juan teaches generate a first plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
… generate a second plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Juan with a reasonable expectation of success. Juan teaches the ability to create histograms to detect lane lines in it’s images to determine their lateral offsets. One with knowledge in the art would find it obvious to try to implement this function of Juan with the vehicle system of Zhou. Juan teaches using histograms for lane detection results in high efficiency with low computational complexity as it takes the vehicle 20 – 25 ms to processes each frame of data (Jaun: Page 350, 6.2 Overall algorithm, paragraph 1). Using this function, lane detection and offset determination can be done quickly, with high efficiency and low processing, thus improving the function of lane detection as taught Zhou.
Regarding claim 9, Zhou, as modified, teaches wherein each of the first plurality of lateral offsets is determined from one of the first subset of the plurality of aligned map datasets, (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
… and wherein each of the second plurality of lateral offsets is determined from one of the second subset of the plurality of aligned map datasets. (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
In sum, Zhou teaches wherein each of the first plurality of lateral offsets is determined from one of the first subset of the plurality of aligned map datasets, and wherein each of the second plurality of lateral offsets is determined from one of the second subset of the plurality of aligned map datasets. Zhou however does not teach wherein each of the first plurality of lateral offset histograms includes a first plurality of lateral offsets for one of the one or more common lane lines, wherein each of the second plurality of lateral offset histograms includes a second plurality of lateral offsets for one of the one or more common lane lines whereas Juan does.
Juan teaches wherein each of the first plurality of lateral offset histograms includes a first plurality of lateral offsets for one of the one or more common lane lines, (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
… wherein each of the second plurality of lateral offset histograms includes a second plurality of lateral offsets for one of the one or more common lane lines, (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Juan with a reasonable expectation of success. Please refer to the rejection of claim 8 as both state the same language and therefore rejected under the same pretenses.
Regarding claim 10, Zhou, as modified, teaches to calculate a plurality of fused point sets based at least in part on the first plurality of probability distribution parameter sets and the second plurality of probability distribution parameter sets, wherein each of the plurality of fused point sets corresponds to one of the one or more common lane lines; and (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The are points found on both the images and the HD map)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
determine the fused map dataset, wherein the fused map dataset includes at least the plurality of fused point sets. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”; Page 7786, A. Data Sources and Comparative Data, paragraph 1: “The crowdsourcing vehicles were equipped with a standard low-cost GNSS receiver and a commonly used camera, as shown in Fig. 7. We evaluated the proposed method on an 8 km urban road in Wuhan, China, for which we created an HD map using a mobile measuring vehicle with a global accuracy of less than 20 cm. The global accuracy of the HD map was verified against 39 ground control points measured by static observation with a CHCNAV i70 receiver. We collected approximately 50 sets of crowdsourced data on × 105 the experimental road, including 1.16 road images. We then analyzed the experimental results in two parts, namely lane detection and lane extraction results.” ,
Supplemental Note: the lane lines are compared and then placed in a fused map. An example is shown in Fig. 7)
PNG
media_image6.png
386
592
media_image6.png
Greyscale
In sum, Zhou teaches to calculate a plurality of fused point sets based at least in part on the first plurality of probability distribution parameter sets and the second plurality of probability distribution parameter sets, wherein each of the plurality of fused point sets corresponds to one of the one or more common lane lines; and determine the fused map dataset, wherein the fused map dataset includes at least the plurality of fused point sets. Zhou however does not teach wherein to determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms, the one or more central computers are further programmed to: calculate a first plurality of probability distribution parameter sets based at least in part on the first plurality of lateral offset histograms, wherein each of the first plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines; calculate a second plurality of probability distribution parameter sets based at least in part on the second plurality of lateral offset histograms, wherein each of the second plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines whereas Juan does.
Jaun teaches wherein to determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms, the one or more central computers are further programmed to:
calculate a first plurality of probability distribution parameter sets based at least in part on the first plurality of lateral offset histograms, wherein each of the first plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines;
calculate a second plurality of probability distribution parameter sets based at least in part on the second plurality of lateral offset histograms, wherein each of the second plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines; (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Juan with a reasonable expectation of success. Please refer to the rejection of claim 8 as both state the same language and therefore rejected under the same pretenses.
Regarding claim 16, Zhou, as modified, teaches
wherein determining the fused map dataset further comprises: based on the first subset of the plurality of aligned map datasets using the one or more central computers, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
… based on the second subset of the plurality of aligned map datasets using the one or more central computers, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
and determining the fused map dataset using the one or more central computers based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the lane lines are compared and then placed in a fused map)
In sum, Zhou teaches wherein determining the fused map dataset further comprises: based on the first subset of the plurality of aligned map datasets using the one or more central computers, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; based on the second subset of the plurality of aligned map datasets using the one or more central computers, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; and determining the fused map dataset using the one or more central computers based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. Zhou however does not teach generating a first and second plurality of lateral offset histograms whereas Juan does.
Juan teaches generating a first plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
… generating a second plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Juan with a reasonable expectation of success. Please refer to the rejection of claim 8 as both state the same language and therefore rejected under the same pretenses.
Regarding claim 17, Zhou, as modified, teaches wherein determining the fused map dataset further comprises:
calculating a plurality of fused point sets based at least in part on the first plurality of probability distribution parameter sets and the second plurality of probability distribution parameter sets using the one or more central computers, wherein each of the plurality of fused point sets corresponds to one of the one or more common lane lines; (Zhou: Page 7785, C. Data Fitting, paragraphs 1 – 2: “Each cluster was divided into several parts with the same width along the vehicle trajectory direction to automatically construct the topological lane information. The improved DBSCAN algorithm was used again in each part to distinguish different lane marking types. The point from which the distances to all the other points in the cluster were the smallest was chosen as the main point. The lane information on an HD map must be fit to an appropriate curve to effectively and accurately express and store the lane’s geometrical information.”; Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraph 2: “In Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the lane lines found in the images are matched with the HD map, the point that best fits in the main point while the other lanes gradually get optimized to adjust as to decrease the offset distance of the lane line in the images with the HD map. The are points found on both the images and the HD map)
PNG
media_image2.png
284
631
media_image2.png
Greyscale
and determining the fused map dataset using the one or more central computers, wherein the fused map dataset includes at least the plurality of fused point sets. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”; Page 7786, A. Data Sources and Comparative Data, paragraph 1: “The crowdsourcing vehicles were equipped with a standard low-cost GNSS receiver and a commonly used camera, as shown in Fig. 7. We evaluated the proposed method on an 8 km urban road in Wuhan, China, for which we created an HD map using a mobile measuring vehicle with a global accuracy of less than 20 cm. The global accuracy of the HD map was verified against 39 ground control points measured by static observation with a CHCNAV i70 receiver. We collected approximately 50 sets of crowdsourced data on × 105 the experimental road, including 1.16 road images. We then analyzed the experimental results in two parts, namely lane detection and lane extraction results.” ,
Supplemental Note: the lane lines are compared and then placed in a fused map. An example is shown in Fig. 7)
PNG
media_image6.png
386
592
media_image6.png
Greyscale
In sum, Zhou teaches wherein determining the fused map dataset further comprises: calculating a plurality of fused point sets based at least in part on the first plurality of probability distribution parameter sets and the second plurality of probability distribution parameter sets using the one or more central computers, wherein each of the plurality of fused point sets corresponds to one of the one or more common lane lines; and determining the fused map dataset using the one or more central computers, wherein the fused map dataset includes at least the plurality of fused point sets. Zhou however does not teach calculating a first plurality of probability distribution parameter sets based at least in part on the first plurality of lateral offset histograms using the one or more central computers, wherein each of the first plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines; calculating a second plurality of probability distribution parameter sets based at least in part on the second plurality of lateral offset histograms using the one or more central computers, wherein each of the second plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines whereas Juan does.
Juan teaches calculating a first plurality of probability distribution parameter sets based at least in part on the first plurality of lateral offset histograms using the one or more central computers, wherein each of the first plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines;
calculating a second plurality of probability distribution parameter sets based at least in part on the second plurality of lateral offset histograms using the one or more central computers, wherein each of the second plurality of probability distribution parameter sets corresponds to one of the one or more common lane lines; (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (Lane Information Extraction for High Definition Maps Using Crowdsourced Data) as applied to claim 18 above, and further in view of Juan et al. (Lane Detection Using Histogram-Based Segmentation and Decision Trees).
Regarding claim 20, Zhou, as modified, teaches wherein to determine the fused map dataset, the one or more central computers are further programmed to: based on the first subset of the plurality of aligned map datasets, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
… based on the second subset of the plurality of aligned map datasets, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; (Zhou: Page 7785, 1) Calculation of the Initial Fitting Vertices, paragraphs 1 - 2: “As mentioned above, lane markings can be classified into various types. Only two points were needed to fit a dashed lane marking, whereas more control points were required for a curved lane marking. In most of the B-spline fitting algorithms, the initial control points are determined by the chord length or curvature. However, these two methods are unsuitable for calculating the initial vertices for lane markings. In this work, the initial fittin points were calculated using a method based on the Douglas–Peucker algorithm usually used for line simplification [36]. First, the straight line between the beginning and the end of the lane center points was identified. Next, the distances between the remaining points and the straight line were calculated. The maximum distance was and compared with the threshold distance thDP. A value that was greater than thDP was recorded. The lane center points were then divided into two parts at the corresponding point.n Fig. 5, the blue circles represent the lane center points (cp1, cp2,…,cpm), while the black ones depict the recorded initial fitting vertices. The red dashed line represents the straight line between the beginning and the end of the lane center points. The first two steps were repeated for both parts of the data until the maximum distance became less than the threshold distance. All recorded points were selected as the initial fitting vertices.”: Page 7786: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”; ; Page 7786, 3) Gradual Optimization, paragraph 1: “The B-spline was optimized according to the maximum distance between the initialized spline and the lane center points. At a maximum distance greater than the threshold distance thB, the fitted B-spline was optimized by adding a new principal vertex until the maximum distance became less than the thB value, as shown in Fig. 6(b).”,
Supplemental Note: the system is able to determine offsets of the crowdsourced vehicle image lane lines and the HD map lane lines)
PNG
media_image3.png
250
600
media_image3.png
Greyscale
and determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. (Zhou: Page 7784, A. Data Management, paragraph 1: “The lane information in the HD map has a fixed update period. During this cycle, the lane information collected through multiple crowdsourced vehicles keep on accumulating. Accordingly, a data management method must be designed to make the best advantage of the large amount of crowdsourced data. First, position data g was used to match with the existing map to determine the road to which the lane information belongs to through an HMM-based map-matching method [34] and fuse the data from different vehicles.”,
Supplemental Note: the lane lines are compared and then placed in a fused map)
In sum, Zhou teaches wherein to determine the fused map dataset, the one or more central computers are further programmed to: based on the first subset of the plurality of aligned map datasets, wherein each of the first plurality of lateral offset histograms corresponds to one of the one or more common lane lines; based on the second subset of the plurality of aligned map datasets, wherein each of the second plurality of lateral offset histograms corresponds to one of the one or more common lane lines; and determine the fused map dataset based at least in part on the first plurality of lateral offset histograms and the second plurality of lateral offset histograms. Zhou however does not teach to generate a first and second plurality of lateral offset histograms whereas Juan does.
Juan teaches generate a first plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
generate a second plurality of lateral offset histograms (Juan: Page 346, I. Introduction, paragraph 8: “The last step is to group the objects that are likely to be lane markers. We do this by using information about the confidence that we have in each object as a possible lane marker. We also consider the information about the direction and relative position of each one of them with respect to the others.”; Page 347, I. Introduction, paragraph 9: “The resulting system runs at 30Hz, and outputs information about lateral offset and the parameters of the lane boundaries that it finds.”; Page 347, 2. Histogram-Based Segmentation, paragraphs 1 – 4: “Histogram based segmentation is basically a region growing segmentation, where the growing is done based in histogram characteristics. This is one of the most important steps in the algorithm, and most of the computational savings are obtained from this step. It is assumed that in the lower scan lines of the image the only object present is the road. This is true in most situations of interest (driving on a highway in relatively good conditions and with no obstacles too close to the vehicle). Then, narrow horizontal bands of the image are considered. In our case we used variable-width bands that are 5 pixels high. Continuing with the assumption of no obstacles in the lower scan lines, the dominant object in the lower bands will be the road. For most acceptable road conditions, the road variations in such a small section are small enough to assure that the histogram of the first bands is going to be unimodal, and in some cases even close to Gaussian (Figure 1). From here we calculate the mean value of the gray level distribution of the road, as well as the maximum and minimum values of such distribution. Values above this maximum are assumed to be lane markers and values below the minimum are assumed to be objects (based on the fact that obstacles, especially vehicles, produce shadows which are darker than the road.). There are, of course, exceptions, but later steps of the algorithm will process these exceptions.” ; Pages 350 – 351, 6.2 Overall algorithm, paragraph 3: “In Figure 7 can be seen some measurements of lateral offset obtained from experimental data.”,
Supplemental Note: Figure 1 shows algorithm determining the lane lines from the images captured. This can be done for multiple images thus multiple graphs can be used. Figure 7 shows the offset measurements of the lane lines)
PNG
media_image4.png
691
704
media_image4.png
Greyscale
PNG
media_image5.png
584
605
media_image5.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention disclosed by Zhou with the teachings of Juan with a reasonable expectation of success. Please refer to the rejection of claim 8 as both state the same language and therefore rejected under the same pretenses.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVAM SHARMA whose telephone number is (703)756-1726. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Bishop can be reached at 571-270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVAM SHARMA/Examiner, Art Unit 3665
/Erin D Bishop/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights