Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 7-8, 11-16, 20 are rejected under 35 USC 102 as being anticipated by Real-time detection and classification of road lane markings, de Paula, cited by applicant.
1. An apparatus for processing image data, the apparatus comprising:
a memory for storing the image data; and
processing circuitry in communication with the memory, wherein the processing circuitry is configured to: (de Paula p 89 discloses a computer vision method, thus it has memory and processors)
obtain a current set of one or more camera images of the image data from a current time; (de Paula p 87 video, note video is a set of images)
calculate respective lane marking confidence values for two or more lane marking types at a plurality of positions in a scene captured by the current set of one or more camera images; (de Paula fig 3 lane markers p 85-86 f2(t) distinguishes the various classes)
determine a lane marking type for each of the plurality of positions based on a comparison of the respective lane marking confidence values with previously-stored lane marking confidence values associated with the plurality of positions in the scene; and (de Paula p 87 determine classes)
output the lane marking type for each of the plurality of positions in the scene. (de Paula p 89 outputs the type of lane markings)
Claims 14 and 20 are rejected under a similar rationale
2. The apparatus of claim 1, wherein the processing circuitry is further configured to: determine a location of a lane marking change from a first lane marking type to a second lane marking type; and output the location of the lane marking change. (de Paula p 88 transitions among different lane types)
Claim 15 is rejected under a similar rationale
3. The apparatus of claim 2, wherein to output the location of the lane marking change, the processing circuitry is configured to: output the location of the lane marking change location indicating a first change type from solid lane markings to dashed lane markings; or output the location of the lane marking change indicating a second change type from dashed lane markings to solid lane markings. (de Paula p 88 transitions among different lane types)
Claim 16 is rejected under a similar rationale.
7. The apparatus of claim 1, wherein the processing circuitry is further configured to:
update position information of a vehicle relative to the lane marking type output for each of the plurality of positions in the scene; and
discard previously-stored lane marking confidence values associated with the plurality of positions in the scene determined to be located behind the vehicle based on the position information as updated for the vehicle. (de Paula p 89 detect and identify lane markers in a vehicle as it is moving and thus it updates the vehicle position with respect to the markers and no longer uses the previous markings as the vehicle moves)
8. The apparatus of claim 1, wherein the processing circuitry is further configured to: match the respective lane marking confidence values calculated for the two or more lane marking types at the plurality of positions in the scene captured by the current set of one or more camera images with the previously-stored lane marking confidence values associated with the plurality of positions in the scene using ego motion of a vehicle that captured the set of one or more camera images. (de Paula p 88 confidence and transitions among different lane types)
11. The apparatus of claim 1, wherein the processing circuitry and the memory are part of an advanced driver assistance system (ADAS). (de Paula p 84 ADAS)
12. The apparatus of claim 1, wherein the processing circuitry is configured to use the lane marking type output for each of the plurality of positions in the scene to control a vehicle. (de Paula p 83 vehicle control; note this is intended use and given little patentable weight)
13. The apparatus of claim 1, wherein the apparatus further comprises: one or more cameras affixed to a vehicle configured to capture the current set of one or more camera images from the current time; and wherein the one or more cameras affixed to the vehicle capture a forward view of an environment surrounding the vehicle. (de Paula p 84 cameras; see also figures 2, 8, 10, 11)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 10 is rejected under 35 USC 103 as being unpatentable over de Paula.
10. The apparatus of claim 1, wherein the processing circuitry is further configured to: detect a vehicle initiating a maneuver to change lanes or overtake; and
output a determination whether the maneuver is permissible based on the lane marking type output for at least one of the plurality of positions in the scene.
Although de Paula does not explicitly disclose detect a vehicle initiating a maneuver to change lanes or overtake; and output a determination whether the maneuver is permissible based on the lane marking type output for at least one of the plurality of positions in the scene, de Paula page 83 discusses the movement of the vehicle outside its lane and determine if a warning is necessary as a possible use case for the technology and therefore it would have been obvious to utilize the system of de Paula for the identified problem for the purposes of determining when lane changes are safe as discussed by de Paula.
Claims 4-6, 17-19 are rejected under 35 USC 103 as being unpatentable over de Paula in view of Adaptative Road Lanes Detection and Classification, Collado, cited by applicant.
4. The apparatus of claim 1, wherein to calculate the respective lane marking confidence values for the two or more lane marking types at the plurality of positions in the scene captured by the current set of one or more camera images, the processing circuitry is further configured to:
a lane marking object located at each of the plurality of positions in the scene captured by the current set of one or more camera images; (de Palma p 85-86 function is interpreted as a marking object)
a first confidence value indicating a probability the lane marking object located at each of the plurality of positions in the scene corresponds to a first one of the two or more lane marking types; and
a second confidence value indicating the probability the lane marking object located at each of the plurality of positions in the scene corresponds to a second one of the two or more lane marking types;
and wherein to determine the lane marking type for each of the plurality of positions, the processing circuitry is further configured to: determine the lane marking type for each of the plurality of positions based on a comparison of the first confidence value and the second confidence value with the previously-stored lane marking confidence values associated with the plurality of positions in the scene. (de Paula p 87 determine classes)
de Paula does not disclose apply semantic segmentation to a single current image corresponding to the current set of one or more camera images from the current time to generate:
Collado p. 3 discloses segmentation)
apply semantic segmentation to a single current image corresponding to the current set of one or more camera images from the current time to generate:
It would have been obvious to modify the system of de Paula to include segmentation for the purposes identifying the lane markings as taught by Collado.
Claim 17 is rejected under a similar rationale
5. The apparatus of claim 1, wherein the processing circuitry is further configured to:
wherein to calculate the respective lane marking confidence values for the two or more lane marking types at the plurality of positions in the scene captured by the current set of one or more camera images, the processing circuitry is configured to:
de Paula does not disclose generate camera features from the current set of one or more camera images corresponding to the two or more lane marking types at the plurality of positions in the scene captured by the current set of one or more camera images; and
project the camera features into a birds-eye-view (BEV) image space; apply semantic segmentation to the BEV image space to generate the respective lane marking confidence values for the two or more lane marking types.
Collando p 3 discloses taking a car view image and transforming into a BEV image as well as utilizing image segmentation to identify road markings. It would have been obvious to modify the system of de Paula to transform the image into a BEV image for the purposes of making the images easier to process as taught by Collando.
Claim 18 is rejected under a similar rationale.
6. The apparatus of claim 1:
de Palma does not disclose
wherein the plurality of positions in the scene captured by the current set of one or more camera images includes one or more occluded lane markings; and
wherein the processing circuitry is further configured to: output a predicted lane marking type for the one or more occluded lane markings based on the comparison of the respective lane marking confidence values corresponding to the one or more occluded lane markings with previously-stored lane marking confidence values associated with the plurality of positions in the scene.
Collando p 5-6 discloses occluded lane markings and outputting a predicted lane marking based on data about other lane markings in the scene. It would have been obvious to modify the system of de Palma to include the ability to process occluded lane markings for the purposes filling in missing areas in the image.
Claim 19 is rejected under a similar rationale.
Claims 9 is rejected under 35 USC 103 as being unpatentable over de Paula in view of Robust Road Marking Extraction in Urban Environments Using Stereo Images, Sebsadji, cited by applicant.
de Palma does not disclose
9. The apparatus of claim 1, wherein to output the lane marking type for each of the plurality of positions in the scene includes the processing circuitry configured to output one of: the lane marking type indicating a first change type to attention markings preceding a toll gate; the lane marking type indicating a second change type to crosswalk markings preceding a crosswalk; the lane marking type indicating a third change type to construction markings preceding a construction zone; or the lane marking type indicating a fourth change type to tunnel markings preceding a tunnel.
Sebsadji, abstract discloses the crosswalks and other markings. It would have been obvious to modify the system of de Palma to include crosswalks for the purposes of including other types of road markings.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
2024/0273920, Sakai, disclosing a lane detection device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ming Shui whose telephone number is (303)297-4247. The examiner can normally be reached on 7-5 Pacific Time, M-Th.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached on 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ming Shui/
Primary Examiner, Art Unit 2663
Prosecution Insights