Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/10/2025 was filed before the first action on the merits of the application. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
Regarding claim terminology interpretation, “time-series image” as read in light of the applicant’s specification is understood to be a singular image and/or a sequence of frames; this interpretation comes from [0050] of the applicant’s specification.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 8 and 18; they recite the limitation “present in overlapped areas of respective frames” renders the claims indefinite in that in light of the applicant’s specification ([0050]) the time series image is/can be only a singular image/frame; as such it becomes unclear how this weighting based on overlapping areas can occur in the scenario where there is only a singular image of the time-series.
When this weighting of the overlapped areas in disclosed in the applicant’s specification the examples given all have multiple different images/frames (e.g. [0082]) however the claims 8/18 (or any of the claims on which they depend on) do not recite or require that the time-series image has multiple frames.
Amending claims 8/18 (or a claim from which they depend) to recite that the time series image specifically has multiple frames would overcome this 112(b) rejection; as the indefiniteness comes specifically due to the Broadest reasonable interpretation of a time-series image including only a singular frame.
Claims 9-10 and 19-20 depend on claims 8 and 18 respectively and thus inherit the at issue limitation.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 11-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 20180283892 A1, “AUTOMATED IMAGE LABELING FOR VEHICLES BASED ON MAPS”, Behrendt et al
Regarding Claim 1, Behrendt teaches “An apparatus for generating a precision map, the apparatus comprising: a communication device configured to receive a time-series image and space information, that are associated with a surrounding environment of a vehicle ;a memory configured to store the time-series image and the space information; and a processor configured to generate a precision map based on the time-series image and the space information, wherein the processor is configured to: extract line information based on the time-series image; generate polylines based on the line information,”( [0017] Here teaches that the system is computerized (i.e. have a memory and processor and communication device for sending/receiving data from the sensors + [0030] “FIG. 5 illustrates the initial coarse solution for a complete track by creating a graph and matching the graph to the high-definition map. On the left side of FIG. 5, a graph of GPS measurements 505 and six degrees of freedom (6-DOF) relative motion edges 510 connecting pose vertices 515 are built (without the lane marker matches). A graph optimization then finds the minimum energy solution by moving the 6-DOF pose vertices around. After this step, the pose vertices may be inaccurate by up to several meters. To tightly align the graph to the road (i.e., the camera image), matches of detected lane markers 520 to all map lane markers are added based on a matching range threshold. All potential matches within matching range may be kept, as seen on the left side of FIG. 5. In some embodiments, three dimensional lane marker detections for alignment can be computed with simple techniques such as a top-hat filter and a stereo camera setup with, for example, a symmetrical local threshold filter. A simple detector (for example, an untrained object detector) may create outliers and misdetections, but these are generally not numerous enough to adversely affect the graph alignment. However, the simple detector is replaced by the first neural net detector for further robustness improvements. In some embodiments, line segments are extracted from these detections by running a Douglas-Peucker polygonization and the resulting 3D line segments are added to the corresponding pose vertices 515 for matching. Due to uncertainty in GPS measurements 505 with respect to elevation, the matching criterion only takes into account the 2D displacement of lane markers 520 in the plane space, tangential to the earth ellipsoid. An initial matching range of 4 meters may be used to robustly handle significant deviations between GPS measurements 505 and the map frame.” Here teaches the images (time-series) and space information (GPS measurements) are matched/aligned; with lines segments being extracted and from [0031] + fig. 6 posted below it can be seen that the extracted segments are generated into polylines to describe the lane, road barriers, etc );” and generate the precision map for autonomous driving based on the polylines and the space information.”( [0027] FIG. 3 illustrates a method of using automatically labeled images according to one embodiment. The method includes using the automatically labelled images to train an object detector (for example, a deep-learning, neural network) within the electronic processor 210. The method also includes using the automatically labelled images for navigation of the vehicle 100. In the example illustrated, the electronic processor 210 loads a high-definition map that includes a first plurality of features (block 305). The electronic processor 210 receives the image including a second plurality of features from the sensor 115 (block 310). In some embodiments, for example, the electronic processor 210 receives a camera image from a vehicle camera as the vehicle is operating. The electronic processor 210 then detects the second plurality of features with an object detector (block 315). As described below, the object detector may be a simple preprogrammed detector. The electronic processor 210 then projects the high-definition map onto the image (block 320). The electronic processor 210 aligns the high-definition map with the image by aligning the first plurality of features with the second plurality of features (block 325). The electronic processor 210 then copies a label describing one of the first plurality of features onto a corresponding one of the second plurality of features to create a labelled image (block 330). The labelled image is then used to assist in navigation of the vehicle 100 (block 335).” After the matching between the image and high-def map features this labelled/matched image (precision map) is used to perform autonomous navigation)
PNG
media_image1.png
326
502
media_image1.png
Greyscale
Regarding Claim 2, Behrendt teaches “preprocess the time-series image by applying a deep learning model; extract the line information by detecting a line of a road from the time-series image; generate the polylines based on the line information and to generate a polyline map based on the polylines;”([0030] “… A simple detector (for example, an untrained object detector) may create outliers and misdetections, but these are generally not numerous enough to adversely affect the graph alignment. However, the simple detector is replaced by the first neural net detector for further robustness improvements. In some embodiments, line segments are extracted from these detections by running a Douglas-Peucker polygonization and the resulting 3D line segments are added to the corresponding pose vertices 515 for matching … “ Here the image is preprocessed through a neural net detector (deep learning model); and then these detections from the model are run through a douglas-peucker algorithm to create 3D line segments (i.e. extract line information and generate polylines which are then added to the matching (generating a polyline map) );” and generate the precision map by applying the space information to the polyline map.”( [0027] FIG. 3 illustrates a method of using automatically labeled images according to one embodiment. The method includes using the automatically labelled images to train an object detector (for example, a deep-learning, neural network) within the electronic processor 210. The method also includes using the automatically labelled images for navigation of the vehicle 100. In the example illustrated, the electronic processor 210 loads a high-definition map that includes a first plurality of features (block 305). The electronic processor 210 receives the image including a second plurality of features from the sensor 115 (block 310). In some embodiments, for example, the electronic processor 210 receives a camera image from a vehicle camera as the vehicle is operating. The electronic processor 210 then detects the second plurality of features with an object detector (block 315). As described below, the object detector may be a simple preprogrammed detector. The electronic processor 210 then projects the high-definition map onto the image (block 320). The electronic processor 210 aligns the high-definition map with the image by aligning the first plurality of features with the second plurality of features (block 325). The electronic processor 210 then copies a label describing one of the first plurality of features onto a corresponding one of the second plurality of features to create a labelled image (block 330). The labelled image is then used to assist in navigation of the vehicle 100 (block 335).” The image and high-def map are matched to create a labelled image (precision map) used for autonomous navigation)
Regarding Claims 11-12, they are method equivalents to apparatus claims 1-2, they have the same grounds of rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Behrendt et al as applied to claims 2 and 12 above, and further in view of US 20230222734 A1, “CONSTRUCTION OF THREE-DIMENSIONAL ROAD NETWORK MAP”, Zhang.
Regarding Claim 3, Behrendt et al does not teach “wherein the processor is configured to generate a binary image by applying semantic segmentation to the time-series image.”
Zhang teaches a similar device which includes generating a binary image based on satellite imagery in order to predict a high-def map for autonomous vehicle navigation. (Abstract + [0036] “In one implementation, extracting the two-dimensional road based on the satellite remote sensing image includes: performing semantic segmentation for a road network in the satellite remote sensing image, and converting the satellite remote sensing image after the semantic segmentation into a binary image to obtain the two-dimensional road.” + [0048] from here it is known that the 2-D road information is/includes polygonal line segments)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application to modify Behrendt et al to modify the neural network detector ([0030] of Behrendt et al) to include semantic segmentation and creation of a binary image for segment extraction as taught by Zhang; one would be motivated to implement the semantic segmentation to binary image in particular to allow for the system to identify a wide range of objects/features in the environment; improving completeness of the resulting image/map. Zhang teaches this improvement of Semantic Segmentation in ([0037] In this implementation, the two-dimensional road is obtained through semantic segmentation, which can completely reflect the road conditions, and resulting data are more complete and reliable.)
Regarding Claim 13, it is a method equivalent to claim 3 it has the same grounds of rejection, combination, and motivation for combination as claim 3.
Claim(s) 4, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Behrendt in view of Zhang as applied to claims 3 and 13 above, and further in view of US 11275955 B2, “Lane Line Processing Method And Device”, Xie et al and further in view of WO 2026022924 A1, “LANE BOUNDARY LINE DETECTION METHOD AND LANE BOUNDARY LINE DETECTION DEVICE”, Sakai et al.
Regarding Claim 4, modified Behrendt (Behrendt + Reznei) as modified in claim 3, teaches “extract an edge of the line from the binary image”(Behrendt [0030] “…However, the simple detector is replaced by the first neural net detector for further robustness improvements. In some embodiments, line segments are extracted from these detections by running a Douglas-Peucker polygonization and the resulting 3D line segments are added to the corresponding pose vertices 515 for matching…”)
However the combination does not explicitly teach “determine feature points of the line from the edge; determine a similarity of the feature points by computing a distance and an angle between the feature points; and classify the feature points based on the similarity of the feature points.”
Xie teaches a lane line recognition system which includes “determine a similarity of the feature points by computing a distance (Column 5, lines 8-22, “For example, in the binary image, the connected domain analysis is performed according to the brightness values of the lane line points, and the adjacent points with the same or similar brightness values belong to the same connected domain. The adjacent points may include pixel points next to each other in their coordinate positions, and may also include pixel points having a small distance therebetween, e.g., a distance not exceeding a set value. Then, the lane line points in the first image are grouped by using the connected domains. Representation information of a lane line corresponding to a group is obtained by using individual lane line points in the group. The representation information of the lane line may include, but is not limited to, a curve equation, a start point, an end point of the lane line, and so on.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application, to modify Behrendt to include grouping/classifying of lane lines based on their similarity as determined by the distance between feature points as taught by Xie. Such a modification would be obvious under the KSR rational of “Combining Prior Art Elements According to Known Methods To Yield Predictable Results” (1) The separate references teach recited limitations; however they do not teach them combined in a single reference (2) A combination (integrating of Xie’s teachings) could be achieved through known method in that its integration requires only software implementation into the image analysis component(s)/processor and memory of Behrendt. In the combination the underlying purpose of Xies teachings (detecting and using feature points to detect lane lines of the road) is the same in the combination as it is in the original reference. Additionally the underlying principles of operation of Behrendt it is not being changed as it is still implementing image analysis to detect features of the surrounding environment; Xie’s teachings are implementing a specific form/way of determining overall features in the environment. (3) The result of the combination would function as expected to one of ordinary skill in the art, the combination is not changing the underlying principles of operation or function of the reference(s) and all references are analogous art applied to image analysis/recognition for autonomous driving.
The combination however would still lack teachings for determining of similarity based also on the angle between feature points. i.e. it does not teach “and an angle between” portion of claim 4.
Sakai et al teaches a lane line recognition system which includes determining of lane lines based in part on the angel between feature points on candidate lines ([0006] “…by extracting a sequence of feature points from the positional information of feature points of an object indicating the lane boundary line of the road on which a vehicle is traveling, relative to the link line, and the approximation line generated by linear approximation having an inferior angle of less than a predetermined angle with respect to the link line, and detecting the lane boundary line corresponding to the sequence of points and the lane boundary line corresponding to feature points not included in the sequence of points.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application, to further modify Behrendt, to implement the angle determination between feature points for determining lane lines as taught by Sakai in order to further improve the ability of the vehicle to detect lane lines when at areas such as merges and diverges (Sakai [0007]“According to the present invention, lane boundary lines can be accurately detected when the number of lanes on a road changes due to lane merging or branching.”)
Regarding Claims 14, it is a method equivalent to apparatus claim 4 above, it has the same grounds of rejection.
Allowable Subject Matter
Claims 5-10, and 15-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: .
Regarding Claims 5 and 15, while Behrendt does teach the use of a Douglas-Peucker algorithm for line detection ([0030] “…However, the simple detector is replaced by the first neural net detector for further robustness improvements. In some embodiments, line segments are extracted from these detections by running a Douglas-Peucker polygonization and the resulting 3D line segments are added to the corresponding pose vertices 515 for matching.”) it and the other found references do not teach or render obvious the use of distance and angle between polylines to determine similarity and classifying of polylines based on the similarity as recited in claims 5 and 15.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 20260071887 A1; US 20260071888 A1; US 12700244 B2; US 20250004476 A1; US 20230384117 A1; US 20220201256 A1;
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH MICHAEL DUNNE whose telephone number is (571)270-7392. The examiner can normally be reached Mon-Thurs 8:30-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Navid Z Mehdizadeh can be reached at (571) 272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH M DUNNE/Primary Examiner, Art Unit 3669