Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/27/2026 has been entered.
Response to Amendment
The amendment filed on 3/27/2026 has been entered and made of record. Claims 1, 4-5, 8-11, 14-15 and 18-20 are amended. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments with respect to the rejections of independent claims 1 and 11 have been fully considered but they are moot because the arguments do not apply to the references being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Takiguchi et al. (US 2010/0034426 A1) in view of Tsuda (US 2011/0118967 A1), Casagranda et al. (WO2023/275478 A1) and Liu et al. (CN115509897A), further in view of Zhang (CN117346800 A).
As to Claim 1, Takiguchi teaches A method for generating road lines in a three-dimensional digital roadbed (Takiguchi, [0061]), the method comprising:
receiving a first two-dimensional image of the roadbed (Takiguchi discloses “the camera 230 captures images during the running and obtains time series image data and time of image data showing an image-capturing time of each image” in [0105]; “the stationary body is captured by stereo view of a plurality of images of the road ahead of the vehicle captured by a single camera from the running vehicle” in [0151]);
identifying a first plurality of road line pixels in the first two-dimensional image of the roadbed, wherein each road line pixel of the first plurality of road line pixels corresponds to an identified road line of the first two-dimensional image (Takiguchi discloses “the feature identification apparatus 300 identifies the features captured on the image by classifying into a moving body (a vehicle, a pedestrian, for example) and a stationary body (street, sidewalk, wall, other (a kilo-post, a sign, for example)) based on the image data and the LRF data” in [0116]; “line segments formed by a point cloud of the road surface shape model…calculates a second neighboring point which is closest to the straight line in a left side of the straight line among the point cloud forming the second line segment and a third neighboring point which is closest to the straight line in a right side of the straight line…” in [0061]; see also [0157]. Here, the left/right straight line refers to road line.);
identifying a first plurality of road line three-dimensional points corresponding to the first plurality of road line pixels of the first two-dimensional image of the roadbed (Takiguchi discloses “Next, the measurement image point obtaining unit 340 projects the road surface shape model… on the image-capturing plane of the camera 230” in [0157]; “In the feature identifying unit 330, a labeling unit 331 classifies the laser measured point cloud shown by the road surface shape model” in [0249]; See also [0205, 0221-0222]);
clustering at least some road line three-dimensional points of the first plurality of road line three-dimensional points into a first three-dimensional point cloud (Takiguchi discloses “Then, the feature determining unit 333 segmentalizes the laser measured point cloud of the first group into plural groups using the edge part as a border, and decides a type of the feature located at each laser measured point for each group (S203c. a feature determining process)” in [0252], see also [0253]).
Takiguchi is silent on medial axis from 3D point cloud. The combination of Tsuda further teaches following limitations:
generating a first medial axis from the first three-dimensional point cloud (Tsuda discloses “At this time, the three-dimensional position data of the position of the center of gravity of the landmark or the point cloud which approximately expresses the landmark shape (for example, the white line of the road, a picture of the traffic sign, or an outline of the road sign mark, etc.) detected on the road surface or around the road can be used for the position/shape point cloud data of the landmarks.” in [0037]; “At this time, the white line shape model 71 is a white line shape, which is expressed by the point cloud, of the existing white line detected by the sensor unit 1” in [0058]; “a position of the center of gravity G1 is obtained from the three-dimensional coordinate value of the point cloud” in [0060]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Takiguchi with the teaching of Tsuda so as to obtain the position data of the center of gravity of the point cloud to determine the position/shape point cloud data and generate road line data (Tsuda, [0037]).
Takiguchi and Tsuda don’t explicitly teach best fit of a line and applying a widening distance to a medial axis. The combination of Casagranda and Liu teaches following limitations:
wherein the first medial axis is a first line of best fit comprising a one-dimensional array of points through the first three-dimensional point cloud; generating a first generated road line by applying a predetermined widening distance to the first medial axis (Tsuda discloses “The coordinate of the position of the center of gravity of the point cloud data constituting the road line shape data 76 is obtained, a point which exists the nearest to the coordinate of the position of the center of gravity is selected from the point cloud data constituting the road line shape data 75… white line point cloud data 72 constituting the white line 70 is extracted around the selected point… Further, when a white line forms a continuous center line, white line point cloud data should be selected for a part of the white line” in [0057]. Here, the center line can be a line of best fitting through the point cloud. For example, Casagranda discloses “- a j<sup>ème</sup>vector represents a line that best fits the points” in [0017]; “The term "best fit" refers to finding a line or axis that minimizes the root mean square distance of the points to the line. Such an adjustment is illustrated by Figure 4. According to this figure, we can see, in the left part, a cloud of points inscribed in a coordinate system or a two-dimensional basis illustrated by the axes x1 and x2. We can see that, on the right side, the z1 axis is the line that fits best to the said points, the data or points being able to be projected into a new coordinate system or basis described by the z1 axis and the z2 axis normal to z1” in [0019], see also Fig 4 below:
PNG
media_image1.png
441
792
media_image1.png
Greyscale
Once center line is extracted, it is well-known to take center line as the midpoint of symmetry and obtain two road lane, for example, Liu discloses “Specifically, from the road and lane topology data in the map data, the coordinates of the continuous lane centerline waypoints for each lane, as well as the lane width information, are extracted. Then, based on the coordinates of the lane centerline waypoints and the lane width for each lane, the waypoint coordinates of the two lane edge lines are calculated and determined. For example, by taking the waypoint coordinates of the lane centerline as the midpoint of symmetry and the lane width as the distance, two waypoints are extended to obtain two waypoints on both sides of the lane” in [n0104].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Takiguchi and Tsuda with the teaching of Casagranda so as to obtain an axis line by best fitting a straight line through the point cloud. The combination of Liu is further to explain to obtain two load lane based on the symmetry from the center line.
In response to the added new limitations receiving a map of a roadbed; generating, using the map of the roadbed, the three-dimensional digital roadbed; modifying, using the first generated road line, the three-dimensional digital roadbed, Zhang further discloses “the initial lane lines and initial road edge lines in the lane line map are fused and their positions corrected to obtain determined updated lane lines and updated road edge lines, including: transforming the initial lane lines and initial road edge lines corresponding to different sets of road data into the same three-dimensional coordinate system; calculating the first distance between different initial lane lines and the second distance between different initial road edge lines in the three-dimensional coordinate system; and fusing the corresponding multiple initial lane lines and/or multiple road edge lines when the first distance and/or the second distance are less than the corresponding first distance threshold…” in [0035]; “In this optional embodiment, when performing fusion and position correction, the initial lane lines and initial road edge lines corresponding to different sets of road data are first transformed to the same three-dimensional coordinate system. In the initial mapping process, mapping can be carried out directly in the same three-dimensional coordinate system...” in [0036]. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Takiguchi, Tsuda, Casagranda and Liu with the teaching of Zhang so as to provide a road data lane matching method by fusing and correcting the initial lane lines and initial road edge lines in the lane line map to obtain determined updated lane lines and updated road edge lines (Zhang, [0005]).
As to Claim 2, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, further comprising clustering at least some road line three-dimensional points of the first plurality of road line three-dimensional points into a second three-dimensional point cloud (Takiguchi discloses “Further, the feature identification apparatus 300 classifies a laser measured point cloud which forms the road surface shape model into groups, and identifies a type of the feature shown by each group based on the shape which the laser measured point cloud forms” in [0080]; see also [0155] and Fig 21.)
As to Claim 3, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 2, wherein:
the method further comprises combining the first three-dimensional point cloud and the second three-dimensional point cloud to form a combined three-dimensional point cloud; and the first medial axis is generated from the combined three-dimensional point cloud (Takiguchi discloses the feature identifying process in Fig 21, such as, the first group A is a street, the first group B is the sidewalk, the first group C is a wall surface in [0248]. These groups are combined to generate a road surface model. Tsuda further discloses generating the center of gravity of the point cloud to form a continuous center line in [0037, 0057-0060].)
As to Claim 4, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 2, further comprising:
generating a second medial axis from the second three-dimensional point cloud, wherein the second medial axis is a second line of best fit comprising a one-dimensional array of points through the second three-dimensional point cloud; and generating a second generated road line by applying a predetermined widening distance to the second medial axis; and modifying, using the second generated road line, the three-dimensional digital roadbed (Takiguchi discloses generating a road surface shape model based on 3D point cloud data in [0057, 0263, 0279, 0364]. Tsuda further discloses a reference white line 70 and a measured road white line 71 in Fig 5. Here, there is no limitation on the number of medial axis extracted from point cloud by performing the method of Casagranda and Liu. Zhang, [0009, 0011, 0033-0036].)
As to Claim 5, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, further comprising receiving a second two-dimensional image of the roadbed; identifying a second plurality of road line pixels in the second two-dimensional image of the roadbed, wherein each road line pixel of the second plurality of road line pixels corresponds to an identified road line of the second two-dimensional image; identifying a second plurality of road line three-dimensional points corresponding to the second plurality of road line pixels of the second two-dimensional image of the roadbed; clustering at least some road line three-dimensional points of the second plurality of road line three-dimensional points into a second three-dimensional point cloud; generating a second medial axis from the second three-dimensional point cloud, wherein the second medial axis is a second line of best fit comprising a one-dimensional array of points through the second three-dimensional point cloud; generating a second generated road line by applying a predetermined widening distance to the second medial axis; and modifying, using the second generated road line, the three-dimensional digital roadbed (Takiguchi discloses “a motion stereo unit for generating a three-dimensional model of a stationary body for a plurality of images captured by a camera mounted on a running vehicle at different times by a motion stereo process as a stationary body model; a moving body removing unit for removing a difference between road Surface shape model which is a three-dimensional point cloud model generated based on distance and orientation data showing distance and orientation for a feature measured from the running vehicle and the stationary body model generated by the motion stereo unit from the road Surface shape model, and generating a moving body removed model which is made by removing a moving body region from the road Surface shape model; a feature identifying unit for determining a type of the stationary body represented by each point cloud based on a position and a shape shown by a point cloud of the moving body removed model generated by the moving body removing processing unit…” in [0057]. Here, the 2D image captured by camera and the corresponding road surface model from the point cloud can be obtained at the different time so that the medial axis and the clustered 3D road line point cloud can be generated in real time. See also the same function cited in claim 1. Zhang, [0009, 0011, 0033-0036].)
As to Claim 7, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, wherein the identifying of the first plurality of road line pixels in the first two-dimensional image of the roadbed includes:
identifying, by a road line pixel identifier, the identified road line of the first two-dimensional image, wherein the road line pixel identifier comprises an Al algorithm, a neural network, an image classification algorithm, an image detection algorithm, an image tagging algorithm, an image segmentation algorithm, or any combination thereof; identifying, by a user using a user interface, the identified road line of the first two- dimensional image; or any combination thereof (Takiguchi discloses identifying the color information of image pixels in [0188-0189, 0195]; corresponding point detecting in [0034, 0367]; labeling process in Fig 12 & 22; see also Fig 29 & 40.)
As to Claim 8, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, wherein modifying the three-dimensional digital roadbed comprises: comparing the first generated road line to a first pre-existing road line of the three-dimensional digital roadbed; and replacing the first pre-existing road line with the first generated road line based on a determination that one or more positions of the first generated road line exceed a predetermined deviation from one or more positions of the first pre-existing road line (Tsuda discloses “a processor unit obtaining running control data based on comparison of the road line shape data obtained by the sensor unit and reference data which has been previously stored” in Abstract; “the processor unit 11 for mounting on the preceding vehicle obtaining running control data of the preceding vehicle based on comparison of the road line shape data showing the three-dimensional position and the shape of the feature obtained by the sensor unit 1 and reference data including the three dimensional position and the data of the shape of the road surface which has been previously stored” in [0105]; see also matching process in [0058-0062]. Zhang also discloses “In a first aspect, this application provides a road data lane matching method, which includes: performing preliminary mapping on road data collected in the field for a predetermined road segment to obtain a lane line map of the predetermined road segment; fusing and correcting the initial lane lines and initial road edge lines in the lane line map to obtain determined updated lane lines and updated road edge lines; determining the lane positions and number of lanes of the predetermined road segment based on the updated lane lines and updated road edge lines; and grouping multiple sets of road data using the lane positions and number of lanes to obtain road data groups that match the position of each lane” in [0005]; see also [0033-0036].)
Claim 9 is rejected based upon similar rationale as Claim 8.
As to Claim 10, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, wherein modifying the three-dimensional digital roadbed comprises incorporating the first generated road line into the three-dimensional digital roadbed (Tsuda, Fig 5.)
Claim 11 recites similar limitations as claim 1 but in a system form. Therefore, the same rationale used for claim 1 is applied.
Claim 12 is rejected based upon similar rationale as Claim 2.
Claim 13 is rejected based upon similar rationale as Claim 3.
Claim 14 is rejected based upon similar rationale as Claim 4.
Claim 15 is rejected based upon similar rationale as Claim 5.
Claim 17 is rejected based upon similar rationale as Claim 7.
Claim 18 is rejected based upon similar rationale as Claim 8.
Claim 19 is rejected based upon similar rationale as Claim 9.
Claim 20 is rejected based upon similar rationale as Claim 10.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Takiguchi in view of Tsuda, Casagranda and Liu, further in view of Zhang and Teo et al. (US 2023/0303113 A1).
As to Claim 6, Takiguchi in view of Tsuda, Casagranda, Liu and Zhang teaches The method of claim 1, further comprising generating, using geospatial data of an image-sensing device which captured the first two-dimensional image of the roadbed, localization data of the first two-dimensional image of the roadbed, wherein the identifying of the first plurality of road line three-dimensional points in the three-dimensional digital roadbed includes using the localization data of the first two-dimensional image of the roadbed (Takiguchi discloses “According to the present invention, a road feature measurement apparatus includes: a feature identifying unit for determining a type of a feature represented by each point cloud based on a position and a shape shown by a point cloud of road surface shape model which is a three-dimensional point cloud model generated based on distance and orientation data showing distance and orientation for the feature measured from a running vehicle;” in [0058]; “The road feature measurement apparatus 100 calculates the position of the feature specified by the user based on the distance data, the angle velocity data, the positioning data, the image data, and the orientation/distance data.” in [0077]. Tsuda also discloses comparing the sensored data with the stored point cloud to detect the road line with more precision in Fig 5. Teo further discloses “In these examples, localization system 406 compares the at least one point cloud or the combined point cloud to two-dimensional (2D) and/or a three-dimensional (3D) map of the area stored in database 410. Localization system 406 then determines the position of the vehicle in the area based on localization system 406 comparing the at least one point cloud or the combined point cloud to the map” in [0067]. See also claim 1 for remaining functions.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Takiguchi, Tsuda, Casagranda, Liu and Zhang with the teaching of Teo so as to improve the determination of a precise real-time location of the vehicle (Teo, [0026]).
Claim 16 is rejected based upon similar rationale as Claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIMING HE whose telephone number is (571)270-1221. The examiner can normally be reached on Monday-Friday, 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached on 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEIMING HE/
Primary Examiner, Art Unit 2611
Prosecution Insights