Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/4/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-12, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Viswanathan (Untied States Patent Application Publication US 2018/0293466 A1) in view of Xu et. al. (United States Patent Application Publication US 2025/0139934 A1).
Regarding claim 1, Viswanathan discloses a computer-implemented method of generating lane detector outputs (Viswanathan [0041]: the autoencoder network may be a lane line detector, traffic sign detector, building façade detector, generic static feature detector, and/or the like used to extract feature information/data from the captured image to generate an encoded representation of the captured image.), by applying horizontal perturbations to the multiple border points of each ground truth lane border, the horizontal perturbations determined using a learned perturbation model (Viswanathan [0047]: the remote apparatus may generate one or more instances of incorrect and/or modified pose information. For example, a perturbation, error, modification, and/or the like may be introduced, added to, and/or the like one or more instances of correct pose information/data corresponding to a first known image to generate an instance of incorrect and/or modified pose information/data corresponding to the first known image. For example, a position perturbation, error, and/or modification may be selected from a distribution of possible perturbations, errors, and/or modifications ranging from a few centimeters to several meters.); the learned perturbation model constructed to impose mutual correlation in the horizontal perturbations between vertically neighbouring border points of each ground truth lane border (Viswanathan [0047]: an instance of incorrect and/or modified pose information/data may include a position perturbation, error, and/or modification, a heading perturbation, error, and/or modification, or both, with respect to the corresponding correct pose information/data.).
Viswanathan fails to disclose the method comprising: receiving a ground truth lane image containing one or more ground truth lane borders, each ground truth lane border comprising multiple border points; and generating a lane detector output image, and comprising parameters learned by performing a statistical analysis of lane detector errors computed between computed output images of a modelled lane detector and ground truth lane border annotations corresponding to the computed output images.
Xu et. al. teaches the method comprising: receiving a ground truth lane image containing one or more ground truth lane borders, each ground truth lane border comprising multiple border points; and generating a lane detector output image (Xu et. al. [0005]: the current system may use one or more machine learning models that are computationally inexpensive and capable of real-time deployment to detect lanes and boundaries. The machine learning model(s) may be trained using low-resolution images, region of interest images, transformed images, ground truth labels or masks, and/or transformed ground truth labels or masks.), and comprising parameters learned by performing a statistical analysis of lane detector errors computed between computed output images of a modelled lane detector and ground truth lane border annotations corresponding to the computed output images (Xu et. al. [0040]: The DNN may be trained with labeled images using multiple iterations until the value of one or more loss functions of the network are below a threshold loss value. The loss function(s) may be used to measure error in the predictions of the DNN using one or more ground truth masks. In one example, a binary cross entropy function may be used as the loss function.).
This is important to the claimed invention because the ground truth lane image serves as a reference image for the machine learning model for lane detection. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Viswanathan and Xu et. al. so that these features are included in the solution of the claimed invention.
Regarding claim 17, which recites a computer system for testing an autonomous vehicle stack, the computer system comprising: at least one memory configured to store computer-readable instructions; and at least one hardware processor coupled to the at least one memory and configured to execute the computer-readable instructions, which upon execution cause the at least one hardware processor to implement: a simulator configured to run simulated scenarios comprising a simulated agent; a planner of the autonomous vehicle stack configured to make decisions for the simulated agent in dependence on one or more lane detection outputs computed for the simulated scenario; and a controller of the autonomous vehicle stack configured to generate a series of control signals for causing the simulated agent to execute the decisions of the planner as the simulated scenario progresses; wherein the computer system is configured to compute each lane detection output by executing the method of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 18, which recites a non-transitory medium embodying computer-readable instructions configured, when executed on one or more hardware processors, to train a perturbation model for modelling lane detector outputs computed by a lane detector of an autonomous vehicle by performing a method comprising of the method of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 2, Viswanathan and Xu et. al. disclose the method of claim 1, and Viswanathan further discloses wherein the parameters of the perturbation model define an error distribution, and wherein the horizontal perturbations are determined by sampling from the error distribution (Viswanathan [0047]: A position perturbation, error, and/or modification may be selected from a distribution of possible perturbations, errors, and/or modifications ranging from a few centimeters to several meters.).
Regarding claim 3, Viswanathan and Xu et. al. disclose the method of claim 1, and Xu et. al. further discloses wherein the ground truth lane borders are defined by a vector of values identifying a horizontal position of lane boundaries for each of a plurality of vertical positions within the ground truth lane image (Xu et. al. [0096]: an original image may be associated with vertices of a polygon (e.g., an annotation of vertices rendered with respect to the original image) representing a lane or boundary.).
PNG
media_image1.png
410
530
media_image1.png
Greyscale
Regarding claim 4, Viswanathan and Xu et. al. disclose the method of claim 1, and Xu et. al. further discloses wherein the output images of the modelled lane detector comprise detected lane borders, and wherein the lane detector errors are computed between the ground truth lane border annotations and detected lane border annotations of the detected lane borders (Xu et. al. [0111], [0113]: the machine learning model(s) may extract features of interest from the image(s) and predict a probability of a boundary class, a lane marking class, or another feature class in the images. The loss function may be used to measure loss (e.g. error) in the segmentation mask(s) (e.g. predictions generated by the machine learning model(s)) as compared to the ground truth data (e.g., the original and/or augmented labels, annotations, and/or masks.).
Regarding claim 5, Viswanathan and Xu et. al. disclose the method of claim 1, and Xu et. al. further discloses wherein the ground truth lane border annotations comprise a polynomial ground truth curve (Xu et. al. [0068]: Curve fitting may also be implemented in order to determine final shapes of the potential lanes and/or boundaries identified that most accurately reflect a natural curve of the lane markings and/or boundaries. Curve fitting may be performed using polyline fitting, polynomial fitting, clothoid fitting, and/or other types of curve-fitting algorithms.).
Regarding claim 6, Viswanathan and Xu et. al. disclose the method of claim 1, and Xu et. al. further discloses wherein the ground truth lane border points are fitted to a polynomial curve (Xu et. al. [0068]: Curve fitting may also be implemented in order to determine final shapes of the potential lanes and/or boundaries identified that most accurately reflect a natural curve of the lane markings and/or boundaries. Curve fitting may be performed using polyline fitting, polynomial fitting, clothoid fitting, and/or other types of curve-fitting algorithms. In some examples, curve fitting may be performed by resampling points (e.g., segmentation points) in the area(s) of interest included in the segmentation mask(s) (as indicated by the dashed line in Fig. 1A).).
Regarding claim 7, Viswanathan and Xu et. al. disclose the method of claim 4, and Xu et. al. further disclose wherein the detected lane border annotations comprise a polynomial lane detection curve for each detected lane border (Xu et. al. [0068]: Curve fitting may also be implemented in order to determine final shapes of the potential lanes and/or boundaries identified that most accurately reflect a natural curve of the lane markings and/or boundaries. Curve fitting may be performed using polyline fitting, polynomial fitting, clothoid fitting, and/or other types of curve-fitting algorithms. In some examples, curve fitting may be performed by resampling points (e.g., segmentation points) in the area(s) of interest included in the segmentation mask(s) (as indicated by the dashed line in Fig. 1A).).
Regarding claim 8, Viswanathan and Xu et. al. discloses the method of claim 2, and Xu et. al. further disclose wherein the horizontal perturbations are determined based on a polynomial error function, coefficients of the polynomial curve sampled from the at least one error distribution (Xu et. al. [0113]: The loss function may be used to measure loss (e.g. error) in the segmentation mask(s) (e.g., predictions generated by the machine learning model(s)) as compared to the ground truth data (e.g., the original and/or augmented labels, annotations, and/or masks.).
PNG
media_image2.png
518
540
media_image2.png
Greyscale
Regarding claim 9, Viswanathan and Xu et. al. disclose the method of claim 1, and Xu et. al. further disclose wherein the parameters of the perturbation model define covariance of a multivariate Gaussian distribution, the covariance encoding correlations between errors for vertically neighbouring points of the ground truth borders, and wherein the horizontal perturbations are determined by sampling from the multivariate Gaussian distribution (Xu et. al. [0061]: Dynamic programming may include determining a set of significant peak points (e.g., pixels) represented by 2D locations and associated confidence values for each area of interest (e.g., each detected lane and/or road boundary). In a non-limiting example, the set of significant points may be determined by performing non-maxima suppression after Gaussian smoothing of the points (e.g., pixels) in the area(s) of interest of the segmentation mask(s)).
Regarding claim 10, Viswanathan and Xu et. al. disclose the method of claim 1, and Viswanathan further discloses wherein the perturbation model is further constructed to impose a correlation between horizontal perturbations of vertically corresponding points of neighbouring ground truth lane borders (Viswanathan [0052]: a comparison of the goodness of fit of the correct map projection with respect to the known image (and/or an encoded representation thereof) provides an indication of how similar the incorrect and/or modified pose information/data is to the correct pose information/data.).
PNG
media_image3.png
905
524
media_image3.png
Greyscale
Regarding claim 11, Viswanathan and Xu et. al. disclose the method of claim 1. Xu et. al. discloses comprising receiving a time series of ground truth images, each comprising respective ground truth lane borders, and generating a time series of lane detector output images (Xu et. al. Fig. 3D).
PNG
media_image4.png
920
482
media_image4.png
Greyscale
Xu et. al. fails to disclose by applying horizontal perturbations to the multiple border points of each ground truth lane border to generate perturbed lane borders, wherein the perturbation model is further constructed to impose a correlation between corresponding perturbed lane borders of consecutive ground truth images of the time series of ground truth images.
Viswanathan teaches applying horizontal perturbations to the multiple border points of each ground truth lane border to generate perturbed lane borders, wherein the perturbation model is further constructed to impose a correlation between corresponding perturbed lane borders of consecutive ground truth images of the time series of ground truth images (Viswanathan [0047]: the remote apparatus may generate one or more instances of incorrect and/or modified pose information. For example, a perturbation, error, modification, and/or the like may be introduced, added to, and/or the like one or more instances of correct pose information/data corresponding to a first known image to generate an instance of incorrect and/or modified pose information/data corresponding to the first known image. For example, a position perturbation, error, and/or modification may be selected from a distribution of possible perturbations, errors, and/or modifications ranging from a few centimeters to several meters.).
This is important to the claimed invention because the ground truth lane image serves as a reference image for the machine learning model for lane detection. Furthermore, time series data is important because the vehicle is moving, and the system of Xu et. al. is capable of real-time deployment to detect lanes and boundaries. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Viswanathan and Xu et. al. so that these features are included in the solution of the claimed invention.
Regarding claim 12, Xu et. al. further discloses the method of claim 11, wherein generating the lane detector output images additionally comprises applying an existence model to the perturbed lane borders, wherein the existence model identifies at least some perturbed lane borders as undetected, wherein the undetected perturbed lane borders are omitted from the lane detector output images (Xu et. al. [0067]: In other examples, only a certain number of lanes and/or road boundaries may be labeled, and any remaining lanes and/or road boundaries may be labeled as undefined. In such examples, one or more of the remaining lanes and/or road boundaries that are identified may be removed and/or not included in any further processing by the vehicle.).
Regarding claim 19, Xu et. al. further discloses the computer system of claim 17, wherein the simulated agent plans and executes driving decisions in the simulated scenario in dependence on a time series of perception outputs computed for the simulated scenario, the perception outputs comprising at least one lane detection output image (Xu et. al. Fig. 3D).
Regarding claim 20, Xu et. al. further discloses the computer system of claim 17, wherein the ground truth lane border annotations are generated in training or testing by projecting a lane boundary from a static road layout to an image plane of a camera, wherein the projection is based on a computed location of the camera within the road layout (Xu et. al. Figure 8B, 3D, [0056]: The segmentation mask(s) output by the machine learning model(s) may represent portions of the input image(s) determined to correspond to lane markings or road boundaries of a driving surface of the vehicle.).
PNG
media_image5.png
560
912
media_image5.png
Greyscale
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Viswanathan (Untied States Patent Application Publication US 2018/0293466 A1) in view of Xu et. al. (United States Patent Application Publication US 2025/0139934 A1) as applied to claim 12 above, and further in view of Hansson et. al. (United States Patent Application Publication US 2020/0386557 A1).
Regarding claim 13, Viswanathan and Xu et. al. disclose the method of claim 12. However, Viswanathan and Xu et. al. fail to disclose wherein the existence model is a Markov model and each lane detector output image is associated with a detection state, wherein the Markov model provides a probability of a detection state for each image based on the detection state of a previous image in the time series of lane detector output images.
Hansson et. al. teaches wherein the existence model is a Markov model and each lane detector output image is associated with a detection state, wherein the Markov model provides a probability of a detection state for each image based on the detection state of a previous image in the time series of lane detector output images (Hansson et. al. [0039]-[0040]: As previously mentioned, the candidate lanes may be represented as hidden states in a Hidden Markov Model. A Hidden Markov Model (HMM) can be construed as a statistical model in which the system being modelled is assumed to be a stochastic process with unobserved, i.e. hidden, states.). This is important to the claimed invention because the Markov Model can help predict the future state of the system depending only on the current state. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Viswanathan, Xu et. al., and Hansson et. al. so that the Markov Model is included in the solution of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 July 1, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights