DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/11/2024 has been considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority based on Chinese Patent Application No CN202210113247.9, filed on January 29, 2022.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1, recites the limitation "predicting a position of a missing point in the lane boundary on the basis of positions of valid points in the lane boundary” it is indefinite as what is the metes and bounds of valid points? The term “valid points” is vague and how it should be used to predict the missing points and it is also not clear how the processing is to be performed so that a missing point can be predicted? Appropriate correction is required.
Same rejection applies to claim 15.
Claim 3, recites the limitation "fusing the forward prediction result and the backward prediction result to acquire the position of the missing point” it is indefinite as what is the metes and bounds of fusion? As how a fusion of a forward and backward prediction result should be used so that a missing point can be determined? It is unclear what the term fusing means? Appropriate correction is required.
Same rejection applies to claims 4, 7 and 13-14.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
In particular, claims are directed to a judicial exception (abstract idea) without significantly more.
Re Claim 1:
Claim 1 recites:
A method for generating a lane boundary in an ego-vehicle ground truth system, comprising:
(a)generating a lane boundary on the basis of received offline measurement data;
(b) predicting a position of a missing point in the lane boundary on the basis of positions of valid points in the lane boundary;
and (c) correcting and/or smoothing positions of respective points in the lane boundary on the basis of lane constraint conditions.
Under Step 1 Claim 1 is a method claim same as claims 2-14 and 16-17.
Under Step 2A -Prong 1:
The identified claim limitations that recite an abstract idea fall within the enumerated groupings of abstract ideas in Section 1 of the 2019 Revised Patent Subject Matter Eligibility Guidance published in the Federal Register (84 FR 50) on January 7, 2019. These fall under mental process.
Claim 1 recites “A method for generating a lane boundary in an ego-vehicle ground truth system, comprising: (a) generating a lane boundary on the basis of received
Under Step 2A - Prong 2; the claims recite the additional element of “offline measurement data” step is not more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea without a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, claim 1 is directed to an abstract idea without a practical application.
Under Step 2B:
The claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception because, when considered separately and as an ordered combination, they do not add significantly more (also known as an “inventive concept”) to the exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a computer hardware amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. Thus, claims 1-14 and 16-17 are not patent eligible.
Therefore, the apparatus claim 15 is rejected under the same rationales used in the rejections of claim 1 outlined above.
Dependent claims 2-14 and 16-17 Dependent claims further define the abstract idea that is present in their respective independent claim 1 and thus correspond to Mental Processes and hence are abstract for the reasons presented above. The dependent claims do not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the dependent claims are directed to an abstract idea. Thus, the claims 1-17 are not patent-eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 8-9 and 15-17 are rejected under 35 U.S.C. 102 as being anticipated in view of Hussam et al (“Lane detection using lane boundary marker network with road geometry constraints” “https://openaccess.thecvf.com/content_WACV_2020/papers/Khan_Lane_detection_using_lane_boundary_marker_network_with_road_geometry_WACV_2020_paper.pdf”), (hereinafter Hussam).
Regarding claim 1, Hussam discloses a method for generating a lane boundary in an ego-vehicle ground truth system, comprising: (see Hussam abstract, fig 1 and page “1873” “we use a lane boundary marker network to detect keypoints along the lane boundaries”, “Our approach is aimed at a forward looking dashcam installed inside the driver’s cabin”),
(a) generating a lane boundary on the basis of received offline measurement data (see Hussam pages “1838-1839” “Second degree quadratic curves are fitted to randomly selected subset of markers”, “We evaluated our algorithm on a number of publicly available lane datasets, namely, CULane [33], TuSimple [36] and Caltech [3]”, “CULane dataset consists of 55 hours of traffic video providing a total of 133,235 images” and “Estimation of horizon and IPM on a short video clip completes the initialization of camera.” Regarding taking data from video and images from before or that was saved on the system which is done offline without getting new readings),
(b) predicting a position of a missing point in the lane boundary on the basis of positions of valid points in the lane boundary (see Hussam pages “1838-1839” “In order to estimate the missing lanes, we used the initialized lane width in pixels (LW p)” and “Adding co to the x-intercept of the detected lane boundaries in the rectified view, we predict the missing lane boundaries”),
and (c) correcting and/or smoothing positions of respective points in the lane boundary on the basis of lane constraint conditions (see Hussam pages “1838-1839” “After the missing lane boundary prediction, we perform a reciprocal weighted average of their x-coordinates near the forward VP and bottom of the image.” and “Three constraints are imposed on the curves in order to ensure correctness. First, the number of inliers, which have to be more than a minimum numbers (generally 25% of the total number of markers for lines and 50% for curves). For this, the perpendicular distance between the curve and a given marker is used as a measure. This process is repeated until the number of inliers cross a threshold or maximum
iterations are reached. If it fails, then straight lines are tried. After line/curve fitting, the second and the third constraints, parallelism and equidistant nature of lane lines, are ensured”).
Regarding claim 2, Hussam discloses generating a mask according to a comparison between the offline measurement data and a preset threshold (see Hussam abstract and page “1837” “In the output layer, we generate a one-hot mask for each marker just like Mask-RCNN, where only the marker’s pixel is marked as foreground or one. To avoid inter-class competition among multiple keypoints, a permask cross-entropy loss is computed. The total loss of this network is the average of the loss for all K keypoints. At inference time, unique marker in each output map is computed by selecting the one with the highest score”).
Regarding claim 3, Hussam discloses wherein step (b} comprises: positioning the missing point in the lane boundary; acquiring a forward prediction result and a backward prediction result via forward process traversal and backward process traversal, respectively; and fusing the forward prediction result and the backward prediction result to acquire the position of the missing point (see Hussam pages “1836-1838” “We represent each lane boundary using a fixed number of keypoints. The accurate estimation of horizon (which is critical for determining the IPM) is dependent upon the correct localization of Vanishing Point (VP). Therefore, during training, we sample more markers near the VP. Another reason for this is that points near the VP represent larger distance in the real world, so accurate localization of these points is crucial for precise line/curve fitting. The selected distribution of these markers along lane boundary on a sample TuSimple image is shown in Figure 2(b). To sample keypoints, we fit a cubic spline curve on the ground truth points for each lane boundary and then divide it vertically into three equal segments, where 50% of the total markers are sampled from a segment that is closer to the VP and 1/4th of the total markers from the rest of the two segments each” and “A frame is considered an inlier frame if it has at least one forward and one lateral vanishing point. A pair of consecutive parallel lines are selected and the distance between the lines is calculated per frame through: LW p = P f |c2−c1| √ a2+b 2 f . (1) where f is the number of inlier frames and LW p is the average initialized lane width in pixels.”).
Regarding claim 8, Hussam discloses wherein the lane constraint conditions comprise a lane width constraint condition and a smoothness constraint condition (see Hussam page “1838-1839” “A frame is considered an inlier frame if it has at least one forward and one lateral vanishing point. A pair of consecutive parallel lines are selected and the distance between the lines is calculated per frame through where f is the number of inlier frames and LW p is the average initialized lane width in pixels”, “In order to estimate the missing lanes, we used the initialized lane width in pixels (LW p). We calculate the lane boundary offset for the missing or low confidence lanes from Equation 1… where co is the required offset in x-intercept for the missing lines. Adding co to the x-intercept of the detected lane boundaries in the rectified view, we predict the missing lane boundaries. For example, for a detected first lane’s first boundary, we predict the number of missing lane boundaries on each side”).
Regarding claim 9, Hussam discloses wherein the lane constraint conditions comprise a lane width constraint condition and a smoothness constraint condition (see Hussam page “1838-1839” “A frame is considered an inlier frame if it has at least one forward and one lateral vanishing point. A pair of consecutive parallel lines are selected and the distance between the lines is calculated per frame through where f is the number of inlier frames and LW p is the average initialized lane width in pixels”, “In order to estimate the missing lanes, we used the initialized lane width in pixels (LW p). We calculate the lane boundary offset for the missing or low confidence lanes from Equation 1… where co is the required offset in x-intercept for the missing lines. Adding co to the x-intercept of the detected lane boundaries in the rectified view, we predict the missing lane boundaries. For example, for a detected first lane’s first boundary, we predict the number of missing lane boundaries on each side”).
Regarding claim 1, Hussam discloses an apparatus for generating a lane boundary in an ego- vehicle ground truth system (see Hussam abstract, fig 1 and page “1873” “we use a lane boundary marker network to detect keypoints along the lane boundaries”, “Our approach is aimed at a forward looking dashcam installed inside the driver’s cabin”),
comprising: a lane boundary generation device configured to generate a lane boundary on the basis of received offline measurement data (see Hussam pages “1838-1839” “Second degree quadratic curves are fitted to randomly selected subset of markers”, “We evaluated our algorithm on a number of publicly available lane datasets, namely, CULane [33], TuSimple [36] and Caltech [3]”, “CULane dataset consists of 55 hours of traffic video providing a total of 133,235 images” and “Estimation of horizon and IPM on a short video clip completes the initialization of camera.” Regarding taking data from video and images from before or that was saved on the system which is done offline without getting new readings),
a missing point prediction device configured to predict a position of a missing point in the lane boundary on the basis of positions of valid points in the lane boundary (see Hussam pages “1838-1839” “In order to estimate the missing lanes, we used the initialized lane width in pixels (LW p)” and “Adding co to the x-intercept of the detected lane boundaries in the rectified view, we predict the missing lane boundaries”),
and a processing device configured to correct and/or smooth positions of respective points in the lane boundary on the basis of lane constraint conditions (see Hussam pages “1838-1839” “After the missing lane boundary prediction, we perform a reciprocal weighted average of their x-coordinates near the forward VP and bottom of the image.” and “Three constraints are imposed on the curves in order to ensure correctness. First, the number of inliers, which have to be more than a minimum numbers (generally 25% of the total number of markers for lines and 50% for curves). For this, the perpendicular distance between the curve and a given marker is used as a measure. This process is repeated until the number of inliers cross a threshold or maximum iterations are reached. If it fails, then straight lines are tried. After line/curve fitting, the second and the third constraints, parallelism and equidistant nature of lane lines, are ensured”).
Regarding claim 16, Hussam discloses comprising an instruction, wherein when the instruction is run, the instruction performs the method (see Hussam page “1841” “For future work, we want to address auxiliary and loading lanes where the assumption of parallelism does not apply. We also want to improve the detection of curved lane boundaries. In its current form, our research takes the stateof-the-art forward and highlights new ways of dealing with one of the most critical challenges in autonomous driving by unifying deep learning and geometric computer vision to build a system utilizing best of both the worlds”).
Regarding claim 17, Hussam discloses a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, implements (see Hussam page “1841” “For future work, we want to address auxiliary and loading lanes where the assumption of parallelism does not apply. We also want to improve the detection of curved lane boundaries. In its current form, our research takes the stateof-the-art forward and highlights new ways of dealing with one of the most critical challenges in autonomous driving by unifying deep learning and geometric computer vision to build a system utilizing best of both the worlds”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 7 are rejected under 35 U.S.C. 102 as being unpatentable in view of Hussam et al (“Lane detection using lane boundary marker network with road geometry constraints” “https://openaccess.thecvf.com/content_WACV_2020/papers/Khan_Lane_detection_using_lane_boundary_marker_network_with_road_geometry_WACV_2020_paper.pdf”).
Regarding claim 7, Hussam discloses to use more points towards the vanishing point. Confronted with the problem to provide a relaxed and therefore faster missing point determination (see Hussam pages “1837-1838” “We represent each lane boundary using a fixed number of keypoints. The accurate estimation of horizon (which is critical for determining the IPM) is dependent upon the correct localization of Vanishing Point (VP). Therefore, during training, we sample more markers near the VP. Another reason for this is that points near the VP represent larger distance in the real world, so accurate localization of these points is crucial for precise line/curve fitting. The selected distribution of these markers along lane boundary on a sample TuSimple image is shown in Figure 2(b). To sample keypoints, we fit a cubic spline curve on the ground truth points for each lane boundary and then divide it vertically into three equal segments, where 50% of the total markers are sampled from a segment that is closer to the VP and 1/4th of the total markers from the rest of the two segments each” and “Critical part of initialization is the estimation of a vanishing line (horizon) from these lines. This requires two sets of vanishing points (VPs), forward and lateral VPs. Forward 1837 VPs can be found from the cross product of the world parallel lines whereas the process of finding lateral vanishing point is a bit involved. For that we use cross-ratios of three real-world parallel lines which are the lane lines as used by Ali et al. [2]. Once a set of forward and lateral VPs is found, we use their horizon estimation algorithm to find the best fitting horizon lh”).
But, Hussam fails to explicitly disclose wherein the fusing the forward prediction result and the backward prediction result to acquire the position of the missing point comprises: using the forward prediction result or the backward prediction result as the position of the missing point. However, it would be obvious for the person skilled in the art to use one of the detected points near the vanishing point and thus to use a forward or backward prediction result as the position of the missing point.
Allowable Subject Matter
Claims 4-6 and 10-14 are objected to as being dependent upon a rejected base claim, but would be allowable if they overcome the 101 rejection and if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HOSSAM M ABD EL LATIF whose telephone number is (571)272-5869. The examiner can normally be reached M-F 8 am-5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached on (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HOSSAM M ABD EL LATIF/Examiner, Art Unit 3664