Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant’s response to the Non-Final Office Action dated 12/31/2025, filed with the office on 03/31/2026, has been entered and made of record.
Status of Claims
Claims 1 and 3-20 are pending. Claim 2 is cancelled.
Response to Amendments
In light of Applicant’s amendments, the objections of record with respect to the specification is withdrawn.
In light of Applicant’s amendments, the objections of record with respect to claim 12 is withdrawn.
Response to Arguments
Applicant’s amendments of independent claims 1, 19, and 20, which has altered the scope of the claims of the instant application, has necessitated the new ground(s) of rejection presented in this office action with respect to claims of the instant application. Accordingly, in response to Applicant’s arguments that are merely directed to the amended portion of the claims, new analyses have been presented below, which make Applicant’s arguments moot.
Consequently, THIS ACTION IS MADE FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 8, 11, 15-16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Neven et al. (“Towards End-to-End Lane Detection: an Instance Segmentation Approach” – published 2018) in view of Sun et al. ("Multi-Stage Hough Space Calculation for Lane Markings Detection via IMU and Vision Fusion" - Published 2019).
Regarding claim 1, Neven teaches “A method for predicting lane line locations (Neven page 1 right hand column paragraph 3 "the proposed method is that the lane fitting is robust against road plane changes and is specifically optimized for better fitting the lanes"), the method comprising:
detecting, by one or more sensors on a vehicle, image data comprising a first portion of a plurality of lane lines of a roadway (Neven Figure 1 and page 1 left hand column paragraph 1 "The goal in each case is to arrive at a full understanding of the environment around the car through the use of various sensors and control modules. Camera-based lane detection is an important step towards such environmental perception as it allows the car to properly position itself within the road lanes");
determining a plurality of location points indicating locations of the plurality of lane lines based on the image data (Neven page 2 right hand column paragraph 1 "the lane embedding branch, which is trained using a clustering loss function, assigns a lane id to each pixel from the lane segmentation branch")”
However, Neven is not relied on to teach “determining a plurality of continuous functions to respectively represent the plurality of lane lines based on the plurality of location points, wherein determining the plurality of continuous functions comprises determining a plurality of responsibility metrics indicating respective likelihoods that each continuous function represents a lane line, wherein determining the plurality of responsibility metrics comprises: determining a respective probability that each of the location points corresponds to each of the plurality of continuous functions; and determining a responsibility metric corresponding to each of the plurality of continuous functions based on the probability that each of the location points corresponds to the respective continuous function; and predicting a location of a second portion of
Sun teaches “determining a plurality of continuous functions to respectively represent the plurality of lane lines based on the plurality of location points, wherein determining the plurality of continuous functions comprises determining a plurality of responsibility metrics indicating respective likelihoods that each continuous function represents a lane line (Sun page 2 paragraph 3 "The proposed probabilistic Hough space is constructed by the outputs of this classification network and each point in this probabilistic space describes the confidence possibility of the corresponding line segment. A threshold (which is set to 0.7) is used to choose the valid line segments from the probabilistic Hough space"), wherein determining the plurality of responsibility metrics comprises:
determining a respective probability that each of the location points corresponds to each of the plurality of continuous functions (Sun page 5 paragraph 1 "After extraction of line segments, a post process is necessary to eliminate false detections such as those line segments overlapping fences. To solve this, a CNN-based classification network is proposed to classify line segments, and a probabilistic Hough space is constructed to record the confidence probability of each line segment. Valid line segments extracted from lane markings are labeled with high probability value in this proposed space (Figure 3). Table 1 shows the structure of the networks. The probabilistic Hough space is constructed by the outputs of the classification networks as demonstrated in Figure 4. A threshold (which is set to 0.7) is used to choose the final valid line segments from the probabilistic Hough space"); and
determining a responsibility metric corresponding to each of the plurality of continuous functions based on the probability that each of the location points corresponds to the respective continuous function (Sun page 9 paragraph 2 "By connecting valid line segments detected across frames as illustrated in Figure 10, the problem of lane fitting can be solved with extensive sequential information. To give the final outputs, a region-growth algorithm is used to divide these foreground points into different lane instances and a parabolic model is used to fit each lane in the current vehicle coordinate. Figure 10 shows the full process mentioned above. To limit the risk of over-fitting, L2 norm is added into the loss function as Equation (10) where a1 (set to be 0.9) and a2 (set to be 0.3) are tradeoff coefficients"); and
predicting a location of a second portion of (Sun page 9 Figure 9 and paragraph 2 "To give the final outputs, a region-growth algorithm is used to divide these foreground points into different lane instances and a parabolic model is used to fit each lane in the current vehicle coordinate").”
It would have been obvious to a person having ordinary skill in the art before effective filing date of the claimed invention of the instant application to combine a method for roadway lane prediction as taught by Neven to use multiple continuous functions and calculating the likelihood that each of the continuous functions corresponds to a location point as taught by Sun.
The suggestion/motivation for doing so would have been that there is a need in the field of automatic lane detection “Many methods are proposed to improve the performance of the lane-marking detection system. Line-segment extraction is a common step to detect lane markings. Well-known methods such as Hough Transform and LSD are very often employed. However, false positive results are given, and a post process is necessary to distinguish whether these line segments belong to lane markings or not. Geometry constraints (e.g., width-based constraints) are always used in this type of classification but it is difficult to deal with particular kinds of line segments, such as those extracted from fences. Meanwhile, numerous end-to-end networks are proposed to detect lanes in images" as noted by the Sun disclosure in page 1 paragraph 2.
Therefore, it would have been obvious to combine the disclosure of Neven with the Sun disclosure to obtain the invention as specified in claim 1 as there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Regarding claim 8, the combination of Neven and Sun teaches “The method of claim 1, further comprising: controlling an autonomous vehicle based on the predicted location of the second portion of the plurality of lane lines (Neven page 1 left hand column paragraph 1 "performing accurate camera-based lane detection in real-time is a key enabler of fully autonomous driving").”
Regarding claim 11, the combination of Neven and Sun teaches “The method of claim 1, wherein the plurality of continuous functions (Sun page 5 paragraph 1 “The probabilistic Hough space is constructed by the outputs of the classification networks as demonstrated in Figure 4. A threshold (which is set to 0.7) is used to choose the final valid line segments from the probabilistic Hough space") comprise a continuous parametric function (Neven page 2 right hand column paragraph 2 "Popular models are cubic polynomials [32], [25], splines [1] or clothoids [10]. To increase the quality of the fit while retaining computational efficiency, it is common to convert the image into a ”bird’s-eye view” using a perspective transformation [39] and perform the curve fitting there").”
The proposed combination as well as the motivation for combining Neven and Sun references presented in the rejection of claim 1, applies to claim 11. Finally the method recited in claim 11 is met by Neven and Sun
Regarding claim 15, the combination of Neven and Sun teaches “The method of claim 1, wherein predicting the location of the second portion of the plurality of lane lines comprises predicting an output based on a function that comprises a two-dimensional curve corresponding to the location of the plurality of lane lines along the roadway (Neven page 4 left hand column paragraph 5 "we train a neural network, H-Net, with a custom loss function: the network is optimized end-to end to predict the parameters of a perspective transformation H, in which the transformed lane points can be optimally fitted with a 2nd or 3rd order polynomial. The prediction is conditioned on the input image, allowing the network to adapt the projection parameters under ground-plane changes, so that the lane fitting will still be correct").”
Regarding claim 16, the combination of Neven and Sun teaches “The method of claim 1, wherein the predicted location of the second portion of the plurality of lane lines based on the plurality of continuous parametric functions (Sun page 5 paragraph 1 “The probabilistic Hough space is constructed by the outputs of the classification networks as demonstrated in Figure 4. A threshold (which is set to 0.7) is used to choose the final valid line segments from the probabilistic Hough space") is limited to a predefined distance from the vehicle (Neven page 5 right hand column paragraphs 3-4 "as already mentioned in Section II B, not all lane-points can be fitted under a fixed transformation (see also Fig. 3). When the slope of the ground-plane changes, points close to the vanishing-point cannot be fitted correctly and are therefore ignored in the MSE-measure, but still counted as missed points. Using the transformation matrix generated by H-Net, which is optimized for lane fitting, the results outperform the lane fitting with a fixed transformation. Not only do we get a better MSE-score, but using this method allows us to fit all points, no matter if the slope of the ground-plane changes").”
The proposed combination as well as the motivation for combining Neven and Sun references presented in the rejection of claim 1, applies to claim 16. Finally the method recited in claim 16 is met by Neven and Sun.
Regarding claim 18, the combination of Neven and Sun teaches “The method of claim 1, wherein predicting the location of the second portion of the plurality of lane lines comprises predicting a location of at least one lane line in a region out of range of the one or more sensors (Neven page 5 right hand column paragraphs 3-4 "as already mentioned in Section II B, not all lane-points can be fitted under a fixed transformation (see also Fig. 3). When the slope of the ground-plane changes, points close to the vanishing-point cannot be fitted correctly and are therefore ignored in the MSE-measure, but still counted as missed points. Using the transformation matrix generated by H-Net, which is optimized for lane fitting, the results outperform the lane fitting with a fixed transformation. Not only do we get a better MSE-score, but using this method allows us to fit all points, no matter if the slope of the ground-plane changes").”
Claims 3-7, 13-14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Neven and Sun in view of Huang ("Probabilistic Lane Estimation using Basis Curves" - Published 2010).
Regarding claim 3, the combination of Neven and Sun teaches “The method of claim 1, wherein determining the plurality of continuous functions (Sun page 2 paragraph 3 "The proposed probabilistic Hough space is constructed by the outputs of this classification network and each point in this probabilistic space describes the confidence possibility of the corresponding line segment. A threshold (which is set to 0.7) is used to choose the valid line segments from the probabilistic Hough space")”.
However, the combination of Neven and Sun is not relied on to teach “determining that at least one of the plurality of continuous functions does not represent a lane line based on the plurality of responsibility metrics; and filtering the at least one continuous function from the plurality of continuous functions”.
Huang teaches “determining that at least one of the plurality of continuous functions does not represent a lane line based on the plurality of responsibility metrics (Huang page 5 left hand column paragraph 4 "Not all observations correspond to the desired curve f"); and
filtering the at least one continuous function from the plurality of continuous functions (Huang page 5 left hand column paragraph 4 "To avoid corrupting the curve estimate, only true observations should be incorporated, and non-observations ignored or used to update a different curve estimate").”
It would have been obvious to a person having ordinary skill in the art before effective filing date of the claimed invention of the instant application to combine a method for roadway lane prediction as taught by Neven and Sun to include the method of filtering out functions that do not represent the lane as taught by Huang.
The suggestion/motivation for doing so would have been the method “generalizes the lateral uncertainty method to perform joint inference of multiple curves (i.e., the left and right boundaries of a lane). It provides a principled framework for ignoring sensor observations that are similar to, but do not correspond to, lane boundaries, and for using observations of one curve to update estimates of another (e.g., when one lane boundary is faded or occluded by traffic). We formulate lane estimation as a curve estimation problem, describe a novel representation for open 2D curves, and present a Bayesian lane estimation algorithm that is robust to the noise and outliers typical of image and LIDAR data” which addresses the problem of “previous work [8], we described the lateral uncertainty algorithm for estimating potential lane boundaries. The algorithm uses probabilistic methods to estimate individual curves from noisy observations, but does not address how to group curves to form lane estimates or track whole lanes over time. Neither does it distinguish true lane boundaries from long tree shadows or other non-boundary painted lines such as stop lines and pedestrian crossings” as noted by the Huang disclosure in page 2 left hand column paragraphs 3-4.
Therefore, it would have been obvious to combine the disclosure of Neven and Sun with the Huang disclosure to obtain the invention as specified in claim 3 as there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Regarding claim 4, the combination of Neven, Sun, and Huang teaches “The method of claim 1, wherein determining the plurality of continuous functions Sun page 2 paragraph 3 "The proposed probabilistic Hough space is constructed by the outputs of this classification network and each point in this probabilistic space describes the confidence possibility of the corresponding line segment. A threshold (which is set to 0.7) is used to choose the valid line segments from the probabilistic Hough space") comprises: determining a first plurality of hyperparameters at a first time (Huang page 4 left hand column paragraph 2 "The probability density of a curve g is defined as:
P
g
g
;
b
,
μ
,
Σ
=
N ̃(b,μ, Σ) where N ̃(b,μ, Σ) refers to the probability desnity function of the normal distrivution with mean μ and covariance Σ");
determining a respective probability corresponds to each of the plurality of continuous functions (Huang page 4 left hand column paragraph 8 "two random curves drawn from a basis curve normal distribution with different probability densities will also exhibit different shapes. Evaluating the likelihood that an arbitrary curve is drawn from a distribution becomes a simple process of computing curve and ray inter-sections") based on the first plurality of hyperparameters (Huang page 4 left hand column paragraph 2 "Together b,
μ
, and
Σ
define a distribution overs curves. We refer to a distribution of this form as a basis curve normal distribution, and represent it with the term
N
~
b
,
μ
,
Σ
. The probability density of a curve g is defined as:
P
g
g
;
b
,
μ
,
Σ
=
N ̃(b,μ, Σ) where N ̃(b,μ, Σ) refers to the probability desnity function of the normal distrivution with mean μ and covariance Σ");
determining a respective probability that each of the location points corresponds to each of the plurality of continuous functions Sun page 2 paragraph 3 "The proposed probabilistic Hough space is constructed by the outputs of this classification network and each point in this probabilistic space describes the confidence possibility of the corresponding line segment. A threshold (which is set to 0.7) is used to choose the valid line segments from the probabilistic Hough space") based on the first plurality of hyperparameters (Huang page 4 right hand column paragraph 2 "It is sometimes desirable to switch the basis curve upon which a curve distribution is defined, but without changing the underlying distribution. In general, it is not possible to match the original distribution exactly, and the approximation error introduced by the reparameterization is directly related to the amount by which the basis curve normal vectors change: approximation error is smallest when the new basis curve is locally parallel to the original basis curve");
determining an updated plurality of hyperparameters at a second time after the first time based on the respective probability that each of the location points corresponds to each of the plurality of continuous functions based on the first plurality of hyperparameters (Huang page 4 right hand column paragraph 2 "It is sometimes desirable to switch the basis curve upon which a curve distribution is defined, but without changing the underlying distribution. In general, it is not possible to match the original distribution exactly, and the approximation error introduced by the reparameterization is directly related to the amount by which the basis curve normal vectors change: approximation error is smallest when the new basis curve is locally parallel to the original basis curve. However, if the new basis curve
b
'
is similar to the original, then reasonable choices of a new mean and covariance,
μ
'
and
Σ
'
, are possible").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 4. Finally the method recited in claim 4 is met by Neven, Sun and Huang.
Regarding claim 5, the combination of Neven, Sun, and Huang teaches “The method of claim 4, wherein the updated plurality of hyperparameters is set to maximize a variational distribution over a plurality of parameters of the plurality of continuous functions (Huang page 5 right hand column paragraph 5 "approximation errors resulting from projection onto a basis curve are minimized when the basis curve geometry matches the true curve geometry. Therefore, we reparametrize the curve estimate, so that the basis curve coincides with the newly updated maximum likelihood curve estimate. Since this estimate is a variation of the current basis curve, reparameterization consists of offsetting the basis curve by the mean vector, then setting the mean vector to zero (Sec. IV-B). We also re-sample the basis curve control points to maintain nearly uniform control point spacing").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 5. Finally the method recited in claim 5 is met by Neven, Sun, and Huang.
Regarding claim 6, the combination of Neven, Sun, and Huang teaches “The method of claim 4, wherein determining the plurality of continuous functions comprises: determining that a convergence criterion is not satisfied based on an initial joint probability distribution evaluated at a first expected value of the plurality of parameters determined based on the first plurality of hyperparameters and an updated joint probability distribution evaluated at a second expected value of the plurality of parameters determined based on the updated plurality of hyperparameters (Huang page 5 right hand column paragraph 5 "approximation errors resulting from projection onto a basis curve are minimized when the basis curve geometry matches the true curve geometry. Therefore, we reparametrize the curve estimate, so that the basis curve coincides with the newly updated maximum likelihood curve estimate. Since this estimate is a variation of the current basis curve, reparameterization consists of offsetting the basis curve by the mean vector, then setting the mean vector to zero (Sec. IV-B). We also re-sample the basis curve control points to maintain nearly uniform control point spacing");
replacing the first plurality of hyperparameters with the updated plurality of hyperparameters (Huang page 4 right hand column paragraph 2 "It is sometimes desirable to switch the basis curve upon which a curve distribution is defined, but without changing the underlying distribution. In general, it is not possible to match the original distribution exactly, and the approximation error introduced by the reparameterization is directly related to the amount by which the basis curve normal vectors change: approximation error is smallest when the new basis curve is locally parallel to the original basis curve. However, if the new basis curve
b
'
is similar to the original, then reasonable choices of a new mean and covariance,
μ
'
and
Σ
'
, are possible"); and
determining a second updated joint probability distribution at a third time based on the plurality of updated hyperparameters (Huang page 4 right hand column paragraph 2 "It is sometimes desirable to switch the basis curve upon which a curve distribution is defined, but without changing the underlying distribution. In general, it is not possible to match the original distribution exactly, and the approximation error introduced by the reparameterization is directly related to the amount by which the basis curve normal vectors change: approximation error is smallest when the new basis curve is locally parallel to the original basis curve. However, if the new basis curve
b
'
is similar to the original, then reasonable choices of a new mean and covariance,
μ
'
and
Σ
'
, are possible" and page 5 right hand column paragraph 5 "approximation errors resulting from projection onto a basis curve are minimized when the basis curve geometry matches the true curve geometry. Therefore, we reparametrize the curve estimate, so that the basis curve coincides with the newly updated maximum likelihood curve estimate. Since this estimate is a variation of the current basis curve, reparameterization consists of offsetting the basis curve by the mean vector, then setting the mean vector to zero (Sec. IV-B). We also re-sample the basis curve control points to maintain nearly uniform control point spacing" - which indicated the parameters can be updated as needed).”
The proposed combination as well as the motivation for combining Neven, Sun and Huang references presented in the rejection of claim 3, applies to claim 6. Finally the method recited in claim 6 is met by Neven, Sun, and Huang.
Regarding claim 7, the combination of Neven, Sun, and Huang teaches “The method of claim 6, wherein determining that the convergence criteria is not satisfied comprises determining that a difference between the updated probability distribution and the initial probability distribution exceeds a threshold (Huang page 6 right hand column paragraph 5-6 "Given a lane distribution and observation as expressed above, we can apply a
x
2
test to determine if z is an observation of f. When estimating multiple lanes, we use a gated greedy assignment procedure to assign observations to lanes. Once an observation has been associated with a lane estimate, the standard Kalman update steps are used to update the mean and covariance. After the updated estimates have been computed, we once again reparametrize the distribution such that the basis curve coincides with the updated maximum likelihood estimate, to minimize approximation error in future update steps").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 7. Finally the method recited in claim 7 is met by Neven, Sun, and Huang.
Regarding claim 13, the combination of Neven, Sun, and Huang teaches “The method of claim 1, wherein the plurality of continuous functions are determined (Huang page 4 left hand column paragraph 5 "two random curves drawn from a basis curve normal distribution with different probability densities will also exhibit different shapes. Evaluating the likelihood that an arbitrary curve is drawn from a distribution becomes a simple process of computing curve and ray intersections") for at least a subset of a predefined maximum number of lane lines (Huang page 3 left hand column paragraph 2 "the number of lanes and their geometries are unknown. The goal of our system is to detect all nearby lanes, estimate their geometries, and update these estimates as the vehicle travels").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 13. Finally the method recited in claim 13 is met by Neven, Sun, and Huang.
Regarding claim 14, the combination of Neven, Sun, and Huang teaches “The method of claim 1, wherein predicting the location of the second portion of the plurality of lane lines (Neven page 5 right hand column paragraphs 3-4 "as already mentioned in Section II B, not all lane-points can be fitted under a fixed transformation (see also Fig. 3). When the slope of the ground-plane changes, points close to the vanishing-point cannot be fitted correctly and are therefore ignored in the MSE-measure, but still counted as missed points. Using the transformation matrix generated by H-Net, which is optimized for lane fitting, the results outperform the lane fitting with a fixed transformation. Not only do we get a better MSE-score, but using this method allows us to fit all points, no matter if the slope of the ground-plane changes") comprises predicting a probability distribution over an output of the plurality of continuous functions (Huang page 4 left hand column paragraph 2 "Together b,
μ
, and
Σ
define a distribution overs curves. We refer to a distribution of this form as a basis curve normal distribution, and represent it with the term
N
~
b
,
μ
,
Σ
. The probability density of a curve g is defined as:
P
g
g
;
b
,
μ
,
Σ
=
N ̃(b,μ, Σ) where N ̃(b,μ, Σ) refers to the probability desnity function of the normal distrivution with mean μ and covariance Σ").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 14. Finally the method recited in claim 14 is met by Neven, Sun, and Huang.
Regarding claim 17, the combination of Neven, Sun, and Huang teaches “The method of claim 1, wherein predicting the location of the second portion of the plurality of lane lines comprises predicting a location envelope of the plurality of lane lines based on an error function (Huang page 7 right hand column paragraph 5 "The centerline error of a lane estimate at a given point on the estimate is defined as the shortest distance from the estimated lane centerline point to the true centerline of the nearest lane").”
The proposed combination as well as the motivation for combining Neven, Sun, and Huang references presented in the rejection of claim 3, applies to claim 17. Finally the method recited in claim 17 is met by Neven, Sun, and Huang.
Claims 9, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Neven and Sun in view of Lin et al. (US 2023/0368547 A1).
Regarding claim 9, the combination of Neven and Sun teaches the method of claim 1. However the combination of Neven and Sun is not relied on to teach “displaying the predicted location of the second portion of the plurality of lane lines on a user interface of a vehicle.”
Lin teaches “displaying the predicted location of the second portion of the plurality of lane lines on a user interface of a vehicle (Lin paragraph [0040] "The monitor 220 displays the final lane detection result output from the processor 36 to be viewed by a driver of the autonomous vehicle").”
It would have been obvious to a person having ordinary skill in the art before
effective filing date of the claimed invention of the instant application to combine a method for roadway lane prediction as taught by Neven and Sun to include hardware capacities, such as a display as taught by Lin.
The suggestion/motivation for doing so would have “In good environmental conditions, lane detection can be performed by taking an image of the roadway using a camera and identifying the lane markings in the roadway using suitable algorithms. However, deteriorated environmental conditions can make it difficult to detect lane markings within the image using this method. For example, night or dark conditions yields dim images, and snow or heavy rain can obscure the lane markings on the roadway. Accordingly, it is desirable to provide a method for detecting lane markings in a roadway in unfavorable environmental conditions” as noted by the Lin disclosure in paragraph 2.
Therefore, it would have been obvious to combine the disclosure of Neven and Sun with the Lin disclosure to obtain the invention as specified in claim 9 as there is a
reasonable expectation of success and/or because doing so merely combines prior art
elements according to known methods to yield predictable results.
Claim 19 recites a system with elements corresponding to the steps of the method recited in claim 1. Therefore, the recited elements of the system of claim 19 are mapped to the proposed combination in the same manner as the corresponding steps of the method claim 1.
However, the combination of Neven and Sun is not relied on to teach “A system for predicting lane line locations, the system comprising one or more processors and memory storing one or more computer programs that include computer instructions, which when executed by the one or more processors”.
Lin teaches “A system for predicting lane line locations, the system comprising one or more processors and memory storing one or more computer programs that include computer instructions, which when executed by the one or more processors (Lin paragraph [0033] "The lane detection system 200 includes a perception, planning and control module 202 suitable for detecting lane markers" and paragraph [0027] "the term module refers to processing circuitry that may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality").”
The proposed combination as well as the motivation for combining Neven, Sun, and Lin references presented in the rejection of claim 9, applies to claim 19. Finally the system recited in claim 19 is met by Neven, Sun, and Lin.
Claim 20 recites a computer readable medium including computer executable instructions corresponding to the steps of the method recited in claim 1. Therefore, the recited instructions of the computer readable medium of claim 20 are mapped to the proposed combination in the same manner as the corresponding steps of the method claim 1.
However, the combination of Neven and Sun is not relied on to teach “a non-transitory computer readable storage medium storing instructions for predicting lane line locations, the instructions configured to be executed by one or more processors of a computing system to cause the system”.
Lin teaches “a non-transitory computer readable storage medium storing instructions for predicting lane line locations, the instructions configured to be executed by one or more processors of a computing system to cause the system (Lin paragraph [0032] "The controller 34 includes a processor 36 and a computer readable storage device or computer readable storage medium 38. The computer readable storage medium 38 includes programs or instructions 39 that, when executed by the processor 36, operate the autonomous vehicle based on sensor system outputs").”
The proposed combination as well as the motivation for combining Neven, Sun, and Lin references presented in the rejection of claim 9, applies to claim 20. Finally the computer readable storage medium recited in claim 20 is met by Neven, Sun, and Lin
Claims 10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Neven and Sun in view of Efrat Sela et al. (US 2021/0276574 A1).
Regarding claim 10, the combination of Neven and Sun teaches “The method of claim 1, wherein determining the plurality of location points indicating locations of the plurality of lane lines based on the image data the image data (Neven page 2 right hand column paragraph 1 "the lane embedding branch, which is trained using a clustering loss function, assigns a lane id to each pixel from the lane segmentation branch")”.
However, the combination of Neven and Sun does not teach “determining a two-dimensional grid of cells based on the image data; assigning at least one semantic label and at least one confidence value associated with the at least one semantic label to each cell, wherein the at least one semantic label and the at least one confidence value are indicative of whether the cell includes a lane line; and extracting a centroid from each cell that is assigned a semantic label indicating that the cell includes a lane line and a confidence value exceeding a threshold.”
Efrat teaches “determining a two-dimensional grid of cells based on the image data (Efrat paragraph [0046] "general curve representation is deployed for lane detection and localization tasks, employing dual pathway architectures to process the input image to Bird's Eye View (BEV) representation, with the BEV grid divided into coarse grid sections, and with parameters of each lane segment that passes through these grid sections being regressed");
assigning at least one semantic label and at least one confidence value associated with the at least one semantic label to each cell (Efrat paragraph [0036] "The ANN inference phase 450 includes identifying straight line segments in orthographic grid sections of the BEV feature maps. The straight line segments are concatenated to form lane edges in the BEV orthographic grids"), wherein the at least one semantic label and the at least one confidence value are indicative of whether the cell includes a lane line (Efrat paragraph [0063] "In addition to the offsets and orientation, each grid section also outputs the probability of a lane passing through this grid section"); and
extracting a centroid from each cell that is assigned a semantic label indicating that the cell includes a lane line (Efrat paragraph [0035] "Each of the straight line segments may be parameterized as a normal vector that is defined in relation to a center point of the orthographic grid section, with the normal vector being defined by a magnitude, a direction, and an altitude in relation to the center point of the orthographic grid section") and a confidence value exceeding a threshold (Efrat paragraph [0067] "To go from prediction of segment score, offsets and orientation in each grid section to lane points, the grid sections scores are subjected to a threshold to identify only the lane grid sections").
It would have been obvious to a person having ordinary skill in the art before
effective filing date of the claimed invention of the instant application to combine a method for roadway lane prediction as taught by Neven and Sun to image segmentation and lane identification as taught by Efrat.
The suggestion/motivation for doing so would have “Accurate detection of travel lanes plays a crucial role in autonomous driving for several reasons, including providing cues regarding available maneuvers of the vehicle, accurately locating the vehicle with respect to a digitized map, and enabling automatic construction of maps associated with accurate localization of the vehicle. As such, there is a need for accurate three-dimensional lane detection and localization of travel lanes. Furthermore, it is desirable to be able to quickly, accurately and precisely detect, monitor and respond to travel lanes of a travel surface that are in a trajectory of a vehicle employing information from an imaging sensor” as noted by the Efrat disclosure in paragraph 2.
Therefore, it would have been obvious to combine the disclosure of Neven and Sun with the Efrat disclosure to obtain the invention as specified in claim 10 as there is a reasonable expectation of success and/or because doing so merely combines prior art
elements according to known methods to yield predictable results.
Regarding claim 12, the combination of Neven, Sun, and Efrat teaches “The method of claim 10, wherein the continuous parametric function comprises a polynomial function (Neven page 2 right hand column paragraph 2 "Popular models are cubic polynomials [32], [25], splines [1] or clothoids [10]. To increase the quality of the fit while retaining computational efficiency, it is common to convert the image into a ”bird’s-eye view” using a perspective transformation [39] and perform the curve fitting there").”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASPREET KAUR whose telephone number is (571)272-5534. The examiner can normally be reached Monday - Friday 9:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASPREET KAUR/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights