DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 7 and 14 are objected to because of the following informalities:
Claim 7: add “and” at the end of third limitation after “;”
Claim 14: add “and” at the end of third limitation after “;”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 – 3, 8 – 10 and 15 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (See Machine Translation for CN115116017A; hereafter referred to as Wu) in view of Zou et al. (See Machine Translation for CN111444778A; hereafter referred to as Zou).
Regarding Claim 1, Wu teaches:
A lane line recognition method applied to an electronic device (Wu, page 2, summary, “the present application provides a lane recognition method”), the method comprising:
obtaining a target image comprising lane lines, which comprises a left lane line and a right lane line (Wu, page 2, summary, “Use the lane line fitting model to perform curve fitting on the data of the lane semantic map to obtain candidate lane lines, and select the left and right lane lines that satisfy the screening strategy from the candidate lane lines and the default lane lines as the target of the current frame Left and right lane lines; among”);
determining whether a recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve (Wu, page 4, Step S50, “Lane lines, select the left and right lane lines… the target left lane line and the target right lane line of the current frame are updated according to the variable weight fitting lane line fitting coefficient of the current frame, and the updated target left lane line The lane between the target right lane line and the current frame is the target lane. The above lane recognition method improves the accuracy and reliability of lane recognition”).
While Wu teaches obtaining an aerial view of the original image (Wu, page 7, para 11, “Camera view to bird's eye view conversion, binarization, dilation processing”) and performing curve fitting on the lane lines (Wu, abstract, “performing curve fitting on the data of the lane semantic map by using a lane line fitting model to obtain candidate lane lines, and selecting left and right lane lines meeting a screening strategy as target left and right lane lines of the current frame”), it fails to explicitly teach:
converting the target image into an aerial view of the lane lines; and
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view;
In the same field of endeavor, Zou teaches:
converting the target image into an aerial view of the lane lines (Zou, page 2, step 3, “convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”);
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view (Zou, page 2, step S4, “Step S4. Lane line fitting: select a parabola as the target model for lane line fitting, and use the random sampling consistency algorithm to fit the effective pixels of the lane line extracted in step S3”); and
Wu and Zou are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu with the method of Zou to make the invention that converts the target image into an aerial view of the lane lines and obtains a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view ; doing so can accurately identify lane lines with high precession detection (Zou, page 2, Summary); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 2, Wu in view of Zou teaches the lane line recognition method according to claim 1, wherein the acquiring of the target image comprising the lane lines comprises:
obtaining an original image by capturing a scene in front of the vehicle (Zou, Page 2, Step S1, “Step S1: Collect the original road image through the vehicle-mounted camera”);
extracting the lane lines from the original image (Zou, page 2, step S3, “the lane line pixels are extracted based on the sliding window”);
converting the extracted lane lines as a binarized image (Zou, page 2, Step S2. Image preprocessing: preprocessing the original road image based on the multi-threshold filtering method… and finally a binary image after multi-threshold filtering; the binary image of the road obtained in step S2”); and
obtaining the target image by superimposing pixel points in the binarized image (Zou, page 2, step S3, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation, and obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view. When the sliding window moves along the lane line, the lane line pixels are extracted based on the sliding window”; Zou, page 4, S32, “The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”).
Regarding Claim 3, Wu in view of Zou teaches the lane line recognition method according to claim 1, wherein the converting of the target image into the aerial view of the lane lines comprises:
determining non-zero pixel points from the target image and taking each of the non-zero pixel points as a target point, the non-zero pixel point being a pixel point of which at least one of an abscissa and an ordinate is not zero (Zou, page 4, step S32, “sliding windows, the height of the sliding window and the width of the sliding window according to the image size, and then compress the image by row to obtain the pixel statistics of the binary image in the bird's-eye view In the figure, the abscissa corresponding to the highest point in the left half of the statistical chart is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half is determined in the same way. Then determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”);
obtaining an inverse perspective transformation matrix according to a coordinate transformation formula and coordinates of each target point in the target image (Zou, page 3, S31, “S31 converts the binary image of the road into a bird's-eye view through inverse perspective transformation using the formula”; page 6, last para, “the positions of the optical center of the camera in the pixel coordinate system, corresponding to the center coordinates of the image matrix…. The parameters of the internal and external parameter matrix of the above-mentioned camera can be obtained through camera calibration. The bird's-eye view after inverse perspective transformation is shown in Figure 3.); and
obtaining the aerial view based on the inverse perspective transformation matrix (Zou, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”).
Regarding Claim 8, Wu teaches:
An electronic device (Wu, page 2, summary, “the present application provides a lane recognition device”), comprising:
a storage device (Wu, page 2, summary, a memory);
at least one processor (Wu, page 2, summary, a processor); and
the storage device storing one or more programs, which when executed by the at least one processor (Wu, page 2, summary, “the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above lane recognition method”), cause the at least one processor to:
obtain a target image comprising lane lines, which comprises a left lane line and a right lane line (Wu, page 2, summary, “Use the lane line fitting model to perform curve fitting on the data of the lane semantic map to obtain candidate lane lines, and select the left and right lane lines that satisfy the screening strategy from the candidate lane lines and the default lane lines as the target of the current frame Left and right lane lines; among”);
determining whether a recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve (Wu, page 4, Step S50, “Lane lines, select the left and right lane lines… the target left lane line and the target right lane line of the current frame are updated according to the variable weight fitting lane line fitting coefficient of the current frame, and the updated target left lane line The lane between the target right lane line and the current frame is the target lane. The above lane recognition method improves the accuracy and reliability of lane recognition”).
While Wu teaches obtaining an aerial view of the original image (Wu, page 7, para 11, “Camera view to bird's eye view conversion, binarization, dilation processing”) and performing curve fitting on the lane lines (Wu, abstract, “performing curve fitting on the data of the lane semantic map by using a lane line fitting model to obtain candidate lane lines, and selecting left and right lane lines meeting a screening strategy as target left and right lane lines of the current frame”), it fails to explicitly teach:
converting the target image into an aerial view of the lane lines; and
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view;
In the same field of endeavor, Zou teaches:
converting the target image into an aerial view of the lane lines (Zou, page 2, step 3, “convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”);
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view (Zou, page 2, step S4, “Step S4. Lane line fitting: select a parabola as the target model for lane line fitting, and use the random sampling consistency algorithm to fit the effective pixels of the lane line extracted in step S3”); and
Wu and Zou are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu with the method of Zou to make the invention that converts the target image into an aerial view of the lane lines and obtains a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view ; doing so can accurately identify lane lines with high precession detection (Zou, page 2, Summary); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 9, Wu in view of Zou teaches the electronic device according to claim 8, wherein the at least one processor acquires the target image comprising the lane lines by:
obtaining an original image by capturing a scene in front of the vehicle (Zou, Page 2, Step S1, “Step S1: Collect the original road image through the vehicle-mounted camera”);
extracting the lane lines from the original image (Zou, page 2, step S3, “the lane line pixels are extracted based on the sliding window”);
converting the extracted lane lines as a binarized image (Zou, page 2, Step S2. Image preprocessing: preprocessing the original road image based on the multi-threshold filtering method… and finally a binary image after multi-threshold filtering; the binary image of the road obtained in step S2”); and
obtaining the target image by superimposing pixel points in the binarized image (Zou, page 2, step S3, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation, and obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view. When the sliding window moves along the lane line, the lane line pixels are extracted based on the sliding window”; Zou, page 4, S32, “The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”).
Regarding Claim 10, Wu in view of Zou teaches the electronic device according to claim 8, wherein the at least one processor converts the target image into the aerial view of the lane lines by:
determining non-zero pixel points from the target image and taking each of the non-zero pixel points as a target point, the non-zero pixel point being a pixel point of which at least one of an abscissa and an ordinate is not zero (Zou, page 4, step S32, “sliding windows, the height of the sliding window and the width of the sliding window according to the image size, and then compress the image by row to obtain the pixel statistics of the binary image in the bird's-eye view In the figure, the abscissa corresponding to the highest point in the left half of the statistical chart is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half is determined in the same way. Then determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”);
obtaining an inverse perspective transformation matrix according to a coordinate transformation formula and coordinates of each target point in the target image (Zou, page 3, S31, “S31 converts the binary image of the road into a bird's-eye view through inverse perspective transformation using the formula”; page 6, last para, “the positions of the optical center of the camera in the pixel coordinate system, corresponding to the center coordinates of the image matrix…. The parameters of the internal and external parameter matrix of the above-mentioned camera can be obtained through camera calibration. The bird's-eye view after inverse perspective transformation is shown in Figure 3.); and
obtaining the aerial view based on the inverse perspective transformation matrix (Zou, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”).
Regarding Claim 15, Wu teaches:
A non-transitory storage medium having instructions stored thereon, when the instructions are executed by a processor of an electronic device, the processor is caused to perform a lane line recognition method (Wu, page 7, last para, “the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media)”), wherein the method comprises:
obtaining a target image comprising lane lines, which comprises a left lane line and a right lane line (Wu, page 2, summary, “Use the lane line fitting model to perform curve fitting on the data of the lane semantic map to obtain candidate lane lines, and select the left and right lane lines that satisfy the screening strategy from the candidate lane lines and the default lane lines as the target of the current frame Left and right lane lines; among”);
determining whether a recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve (Wu, page 4, Step S50, “Lane lines, select the left and right lane lines… the target left lane line and the target right lane line of the current frame are updated according to the variable weight fitting lane line fitting coefficient of the current frame, and the updated target left lane line The lane between the target right lane line and the current frame is the target lane. The above lane recognition method improves the accuracy and reliability of lane recognition”).
While Wu teaches obtaining an aerial view of the original image (Wu, page 7, para 11, “Camera view to bird's eye view conversion, binarization, dilation processing”) and performing curve fitting on the lane lines (Wu, abstract, “performing curve fitting on the data of the lane semantic map by using a lane line fitting model to obtain candidate lane lines, and selecting left and right lane lines meeting a screening strategy as target left and right lane lines of the current frame”), it fails to explicitly teach:
converting the target image into an aerial view of the lane lines; and
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view;
In the same field of endeavor, Zou teaches:
converting the target image into an aerial view of the lane lines (Zou, page 2, step 3, “convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”);
obtaining a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view (Zou, page 2, step S4, “Step S4. Lane line fitting: select a parabola as the target model for lane line fitting, and use the random sampling consistency algorithm to fit the effective pixels of the lane line extracted in step S3”); and
Wu and Zou are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu with the method of Zou to make the invention that converts the target image into an aerial view of the lane lines and obtains a left lane line curve and a right lane line curve by performing a curve fitting on the lane lines according to pixel points of the aerial view ; doing so can accurately identify lane lines with high precession detection (Zou, page 2, Summary); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 16, Wu in view of Zou teaches the non-transitory storage medium according to claim 15, wherein the acquiring of the target image comprising the lane lines comprises:
obtaining an original image by capturing a scene in front of the vehicle (Zou, Page 2, Step S1, “Step S1: Collect the original road image through the vehicle-mounted camera”);
extracting the lane lines from the original image (Zou, page 2, step S3, “the lane line pixels are extracted based on the sliding window”);
converting the extracted lane lines as a binarized image (Zou, page 2, Step S2. Image preprocessing: preprocessing the original road image based on the multi-threshold filtering method… and finally a binary image after multi-threshold filtering; the binary image of the road obtained in step S2”); and
obtaining the target image by superimposing pixel points in the binarized image (Zou, page 2, step S3, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation, and obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view. When the sliding window moves along the lane line, the lane line pixels are extracted based on the sliding window”; Zou, page 4, S32, “The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”).
Regarding Claim 17, Wu in view of Zou teaches the non-transitory storage medium according to claim 15, wherein the converting of the target image into the aerial view of the lane lines comprises:
determining non-zero pixel points from the target image and taking each of the non-zero pixel points as a target point, the non-zero pixel point being a pixel point of which at least one of an abscissa and an ordinate is not zero (Zou, page 4, step S32, “sliding windows, the height of the sliding window and the width of the sliding window according to the image size, and then compress the image by row to obtain the pixel statistics of the binary image in the bird's-eye view In the figure, the abscissa corresponding to the highest point in the left half of the statistical chart is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half is determined in the same way. Then determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view”);
obtaining an inverse perspective transformation matrix according to a coordinate transformation formula and coordinates of each target point in the target image (Zou, page 3, S31, “S31 converts the binary image of the road into a bird's-eye view through inverse perspective transformation using the formula”; page 6, last para, “the positions of the optical center of the camera in the pixel coordinate system, corresponding to the center coordinates of the image matrix…. The parameters of the internal and external parameter matrix of the above-mentioned camera can be obtained through camera calibration. The bird's-eye view after inverse perspective transformation is shown in Figure 3.); and
obtaining the aerial view based on the inverse perspective transformation matrix (Zou, “Step S3, lane line pixel extraction: convert the binary image of the road obtained in step S2 into a bird's-eye view through inverse perspective transformation”).
Claims 4 – 6, 11 – 13 and 18 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (See Machine Translation for CN115116017A; hereafter referred to as Wu) in view of Zou et al. (See Machine Translation for CN111444778A; hereafter referred to as Zou).
Regarding Claim 4, Wu in view of Zou teaches the lane line recognition method according to claim 1, further comprising:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, (Zou, page 2, step S3, “obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view”; page 4, S32, “determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view);
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Zou, page 10, S32, “For the statistical graph, the abscissa corresponding to the highest point in the left half of the statistical graph is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half of the area is determined in the same way”);
Wu in view of Zou fails to explicitly teach:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value;
determining a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value;
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position;
fitting all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve; and
fitting all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve.
In the same field of endeavor, Luo teaches:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value (Luo, page 3, para 2, “an integral map constructed by using the first image , and determine a plurality of first regions in the first image; wherein, the abscissa of the integral map is the number of pixel columns of the image, and the ordinate is the number of pixels of the image in the vertical axis direction...the first region is determined according to the third lane line and the maximum value of the integral map, wherein the position of the maximum value of the integral map may be a position where the pixels of the lane line are concentrated, so that the maximum value is determined at the maximum value”);
determining a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value (Luo, page 3, para 8, “lane lines that satisfy the constraint conditions are determined in the first area N times, and multiple lane lines are obtained; wherein, N is a non-zero natural number; pixels are determined in the multiple lane lines The one with the largest number of lane lines gets the second lane line. This embodiment of the present application constrains the relationship between the first lane lines according to the rules followed by the lane lines, and selects a lane line with the largest number of pixels among the first lane lines that satisfy the constraint conditions as the second lane line”);
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Luo, page 11, para 2, “determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”);
fitting all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve (Luo, page 14, para 5, “the lane line fitting process, the vehicle driving system uses sliding window, Hough transform and other algorithms to fit the image containing the lane line pixel information to obtain multiple lane lines. According to the multiple lane lines obtained by fitting, the third lane line is determined”); and
fitting all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve (Luo, page 14, para 7, “After determining the width and height of the initial sliding window, the vehicle driving system determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that generates a non-zero pixel point distribution map; generates a non-zero pixel point distribution map; and does fitting of all non-zero pixel points included in the first sliding window and second sliding window as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 5, Wu in view of Zou teaches the lane line recognition method according to claim 1, but fails to explicitly teach:
wherein the determining of whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve comprises:
determining whether there is an intersection point between the left lane line curve and the right lane line curve;
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve;
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve; and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different.
In the same field of endeavor, Luo teaches:
determining whether there is an intersection point between the left lane line curve and the right lane line curve (Luo, page 10, S502, para 3, “The coordinate value is transformed into a curve in the parameter space, and the intersection of the curves is obtained in the parameter space, thereby determining at least one first lane line”);
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”);
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve (Luo, S503, “When the curvatures corresponding to the two adjacent first lane lines are equal, it can be judged that the detection result of the lane lines is accurate, and a second lane line that conforms to the law followed by the lane lines is obtained”); and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that determines whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 6, Wu in view of Zou further in view of Luo teaches the lane line recognition method according to claim 5, wherein the determining of whether there is the intersection point between the left lane line curve and the right lane line curve comprises:
acquiring a plurality of pixel points on the left lane line curve, acquiring a plurality of pixel points on the right lane line curve (Luo, page 3, para 9, “pixels are determined in the multiple lane lines”); and
determining whether there is the intersection between the left lane line curve and the right lane line curve according to the plurality of pixel points on the left lane line curve and the plurality of pixel points on the right lane line curve (Luo, page 11, S503, “If the same coordinates are found in the coordinate values corresponding to the lane lines, it may be that two adjacent first lane lines intersect”; page 17, last para, “when the width between the pixels with the same ordinate in two adjacent first lane lines does not satisfy the first range, it may be that the distance between the two adjacent first lane lines is too close or intersects, etc”; Luo, page 18, para 3, “when the curvature difference between two adjacent first lane lines does not satisfy the fourth range, it may be that the distance between the two adjacent first lane lines is too close or intersects”).
Regarding Claim 11, Wu in view of Zou teaches the electronic device according to claim 8, wherein the at least one processor is further caused to:
generate a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, (Zou, page 2, step S3, “obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view”; page 4, S32, “determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view);
slide a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Zou, page 10, S32, “For the statistical graph, the abscissa corresponding to the highest point in the left half of the statistical graph is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half of the area is determined in the same way”);
Wu in view of Zou fails to explicitly teach:
generate a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value;
determine a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value;
slide a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position;
fit all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve; and
fit all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve.
In the same field of endeavor, Luo teaches:
generate a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value (Luo, page 3, para 2, “an integral map constructed by using the first image , and determine a plurality of first regions in the first image; wherein, the abscissa of the integral map is the number of pixel columns of the image, and the ordinate is the number of pixels of the image in the vertical axis direction...the first region is determined according to the third lane line and the maximum value of the integral map, wherein the position of the maximum value of the integral map may be a position where the pixels of the lane line are concentrated, so that the maximum value is determined at the maximum value”);
determine a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value (Luo, page 3, para 8, “lane lines that satisfy the constraint conditions are determined in the first area N times, and multiple lane lines are obtained; wherein, N is a non-zero natural number; pixels are determined in the multiple lane lines The one with the largest number of lane lines gets the second lane line. This embodiment of the present application constrains the relationship between the first lane lines according to the rules followed by the lane lines, and selects a lane line with the largest number of pixels among the first lane lines that satisfy the constraint conditions as the second lane line”);
slide a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Luo, page 11, para 2, “determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”);
fit all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve (Luo, page 14, para 5, “the lane line fitting process, the vehicle driving system uses sliding window, Hough transform and other algorithms to fit the image containing the lane line pixel information to obtain multiple lane lines. According to the multiple lane lines obtained by fitting, the third lane line is determined”); and
fit all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve (Luo, page 14, para 7, “After determining the width and height of the initial sliding window, the vehicle driving system determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that generates a non-zero pixel point distribution map; generates a non-zero pixel point distribution map; and does fitting of all non-zero pixel points included in the first sliding window and second sliding window as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 12, Wu in view of Zou teaches the electronic device according to claim 8, but fails to explicitly teach:
wherein the at least one processor determines whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve by:
determining whether there is an intersection point between the left lane line curve and the right lane line curve;
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve;
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve; and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different.
In the same field of endeavor, Luo teaches:
determining whether there is an intersection point between the left lane line curve and the right lane line curve (Luo, page 10, S502, para 3, “The coordinate value is transformed into a curve in the parameter space, and the intersection of the curves is obtained in the parameter space, thereby determining at least one first lane line”);
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”);
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve (Luo, S503, “When the curvatures corresponding to the two adjacent first lane lines are equal, it can be judged that the detection result of the lane lines is accurate, and a second lane line that conforms to the law followed by the lane lines is obtained”); and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that determines whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 13, Wu in view of Zou further in view of Luo teaches the electronic device according to claim 12, wherein the at least one processor determines whether there is the intersection point between the left lane line curve and the right lane line curve by:
acquiring a plurality of pixel points on the left lane line curve, acquiring a plurality of pixel points on the right lane line curve (Luo, page 3, para 9, “pixels are determined in the multiple lane lines”); and
determining whether there is the intersection between the left lane line curve and the right lane line curve according to the plurality of pixel points on the left lane line curve and the plurality of pixel points on the right lane line curve (Luo, page 11, S503, “If the same coordinates are found in the coordinate values corresponding to the lane lines, it may be that two adjacent first lane lines intersect”; page 17, last para, “when the width between the pixels with the same ordinate in two adjacent first lane lines does not satisfy the first range, it may be that the distance between the two adjacent first lane lines is too close or intersects, etc”; Luo, page 18, para 3, “when the curvature difference between two adjacent first lane lines does not satisfy the fourth range, it may be that the distance between the two adjacent first lane lines is too close or intersects”).
Regarding Claim 18, Wu in view of Zou teaches the non-transitory storage medium according to claim 15, wherein the method further comprises:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, (Zou, page 2, step S3, “obtain a pixel statistical map of the binary image in the bird's-eye view based on the sliding window in the bird's-eye view”; page 4, S32, “determine the sliding window boundary according to the base point coordinates, sliding window width, and sliding window height, and count the coordinates of all non-zero pixels in the sliding window area, and use the average of the abscissas of these points as the starting base point of the next sliding window The coordinates are sequentially superimposed on the last sliding window to obtain the effective pixels of the lane line in the overlook view);
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Zou, page 10, S32, “For the statistical graph, the abscissa corresponding to the highest point in the left half of the statistical graph is used as the initial abscissa of the base point of the left sliding window, and the initial abscissa of the base point of the right half of the area is determined in the same way”);
Wu in view of Zou fails to explicitly teach:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value;
determining a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value;
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position;
fitting all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve; and
fitting all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve.
In the same field of endeavor, Luo teaches:
generating a non-zero pixel point distribution map a number of non-zero pixel points in each column in the aerial view, the non-zero pixel point distribution map comprising a first peak value and a second peak value, and the first peak value being located on a left of the second peak value (Luo, page 3, para 2, “an integral map constructed by using the first image , and determine a plurality of first regions in the first image; wherein, the abscissa of the integral map is the number of pixel columns of the image, and the ordinate is the number of pixels of the image in the vertical axis direction...the first region is determined according to the third lane line and the maximum value of the integral map, wherein the position of the maximum value of the integral map may be a position where the pixels of the lane line are concentrated, so that the maximum value is determined at the maximum value”);
determining a first initial position of the left lane line in the aerial view according to the first peak value, and determining a second initial position of the right lane line in the aerial view according to the second peak value (Luo, page 3, para 8, “lane lines that satisfy the constraint conditions are determined in the first area N times, and multiple lane lines are obtained; wherein, N is a non-zero natural number; pixels are determined in the multiple lane lines The one with the largest number of lane lines gets the second lane line. This embodiment of the present application constrains the relationship between the first lane lines according to the rules followed by the lane lines, and selects a lane line with the largest number of pixels among the first lane lines that satisfy the constraint conditions as the second lane line”);
sliding a first sliding window based on the first initial position, and sliding a second sliding window based on the second initial position (Luo, page 11, para 2, “determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”);
fitting all non-zero pixel points included in the first sliding window to be a first curve during a sliding of the first sliding window in the aerial view, and taking the first curve as the left lane line curve (Luo, page 14, para 5, “the lane line fitting process, the vehicle driving system uses sliding window, Hough transform and other algorithms to fit the image containing the lane line pixel information to obtain multiple lane lines. According to the multiple lane lines obtained by fitting, the third lane line is determined”); and
fitting all non-zero pixel points included in the second sliding window to be a second curve during a sliding of the second sliding window in the aerial view, and taking the second curve as the right lane line curve, wherein a sliding of the second sliding window is adjusted according to the first curve (Luo, page 14, para 7, “After determining the width and height of the initial sliding window, the vehicle driving system determines the center of the next sliding window according to the mean value of the pixel coordinates of the lane lines in the initial sliding window, and then repeats this operation, that is, the position of a sliding window in each search is determined by the center within the next window until the sliding window covers the lane line pixels in the image”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that generates a non-zero pixel point distribution map; generates a non-zero pixel point distribution map; and does fitting of all non-zero pixel points included in the first sliding window and second sliding window as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 19, Wu in view of Zou teaches the non-transitory storage medium according to claim 15, but fails to explicitly teach:
wherein the determining of whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve comprises:
determining whether there is an intersection point between the left lane line curve and the right lane line curve;
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve;
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve; and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different.
In the same field of endeavor, Luo teaches:
determining whether there is an intersection point between the left lane line curve and the right lane line curve (Luo, page 10, S502, para 3, “The coordinate value is transformed into a curve in the parameter space, and the intersection of the curves is obtained in the parameter space, thereby determining at least one first lane line”);
determining that the identification of the lane lines is inaccurate in response that there is the intersection point between the left lane line curve and the right lane line curve (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”);
determining whether a curvature of the left lane line curve and a curvature of the right lane line curve are the same in response that there is no intersection point between the left lane line curve and the right lane line curve (Luo, S503, “When the curvatures corresponding to the two adjacent first lane lines are equal, it can be judged that the detection result of the lane lines is accurate, and a second lane line that conforms to the law followed by the lane lines is obtained”); and
determining that the recognition of the lane lines is inaccurate in response that the curvature of the left lane line curve and the curvature of the right lane line curve are different (Luo, page 11, S503, para 4, “When the curvatures corresponding to the two adjacent first lane lines are not equal, it may be that the distance between the two adjacent first lane lines is too close or the intersection, etc., resulting in non-parallel two adjacent lane lines, and the actual On the road, the lane line usually conforms to the driving rules of the vehicle, and there is no situation such as too close or crossing, so it can be judged that the obtained lane line detection result is inaccurate”).
Wu, Zao and Luo are considered analogous art as they are reasonably pertinent to the same field of endeavor of image processing. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wu in view of Zou with the method of Luo to make the invention that determines whether the recognition of the lane lines is accurate according to the left lane line curve and the right lane line curve as taught by Luo; doing so can improve the accuracy of lane detection and further improve automatic driving or assisted driving (Luo, abstract); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 20, Wu in view of Zou further in view of Luo teaches the non-transitory storage medium according to claim 17, wherein the determining of whether there is the intersection point between the left lane line curve and the right lane line curve comprises:
acquiring a plurality of pixel points on the left lane line curve, acquiring a plurality of pixel points on the right lane line curve (Luo, page 3, para 9, “pixels are determined in the multiple lane lines”); and
determining whether there is the intersection between the left lane line curve and the right lane line curve according to the plurality of pixel points on the left lane line curve and the plurality of pixel points on the right lane line curve (Luo, page 11, S503, “If the same coordinates are found in the coordinate values corresponding to the lane lines, it may be that two adjacent first lane lines intersect”; page 17, last para, “when the width between the pixels with the same ordinate in two adjacent first lane lines does not satisfy the first range, it may be that the distance between the two adjacent first lane lines is too close or intersects, etc”; Luo, page 18, para 3, “when the curvature difference between two adjacent first lane lines does not satisfy the fourth range, it may be that the distance between the two adjacent first lane lines is too close or intersects”).
Allowable Subject Matter
Claims 7 and 14 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and overcoming claim objections.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
CN 115731525 A Vehicle Lane Line Identification Method, Device, Electronic Device And Computer Readable Medium
CN 111126306 A A Lane Detection Method Based On Edge Feature And A Sliding Window Of
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAISALI RAO KOPPOLU whose telephone number is (571)270-0273. The examiner can normally be reached Monday - Friday 8:30 - 5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format.
For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
VAISALI RAO. KOPPOLU
Examiner
Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664