DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Currently, claims 1-30 are pending in the application.
Response to Arguments / Amendments
Applicant’s arguments have been fully considered, but they are not persuasive, see discussion below.
Rejections under 35 U.S.C. § 103:
A. Independent Claims 1, 10, 19, and 25
The applicant argued that Park fails to disclose "calculating the distance between the first camera and the point in space comprises: estimating a disparity between the first image data and the second image data and calculating the distance between the first camera and the point in space based on the estimated disparity,"
As to the above argument, Heafitz discloses the distance between the first camera and the point in space recognizing and taking into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space ([0029]) and calculating a three-dimensional coordinate of object 302 from the telephoto image and the wide-angle image ([0061], FIG. 5).
Park also teaches determining a point in space based on a match between pixels of the first image data and pixels of the second image data using the large disparities between pixels in the two images may indicate objects or surfaces that are closer and small disparities may indicate objects or surfaces that are further away and generate dense pixel correspondences across left and right camera image data of the pairwise images ([0021]) and calculating the distance between the first camera and the point in space based on the estimated disparity by performing cross-camera OF tracking at least in the overlapping FOV region 210 of the cameras 202 and 204 and determine a distance to the particular location based on a disparity between pixels in the camera image data in the overlapping FOV region 210 for a particular location ( [0031], FIG. 2)
A. Independent Claims 5 and 6
The applicant argued that Heafitz and Park do not disclose that "a first position of the first camera is vertically offset from a second position of the second camera & the first position of the first camera is vertically offset from the second position of the second camera by a distance greater than one meter."
As to the above argument, Heafitz discloses the distance between the first camera and the point in space recognizing and taking into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space ([0029]) and calculating a three-dimensional coordinate of object 302 from the telephoto image and the wide-angle image ([0061], FIG. 5).
Heafitz also discloses the first position of the first camera is vertically offset from the second position of the second camera by a distance greater than one meter with the identifiable object 302 in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 and perform object tracking using the telephoto image and wide-angle image from stereo camera assembly 307 ([0061], FIG. 5).
Accordingly, Examiner maintains the rejection with regards to above arguments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-30 are rejected under 35 U.S.C. 103 as being unpatentable over Heafitz et al. (US 20220144186, hereinafter Heafitz) in view of Park et al. (US 20220250624, hereinafter Park).
Regarding Claim 1, Heafitz discloses a method for image processing for use in a vehicle assistance system ([0089], FIG. 8), comprising:
receiving first image data from a first image sensor of a first camera having a first lens type ([0040], [0041], FIG. 1, taking mages using wide- angle lens camera 102 and telephoto lens camera 104 are mounted at different heights; [0046], FIG. 2, field of view of telephoto image 204 corresponds to the field of view of the telephoto lens camera at the time of capturing telephoto image 204 & field of view of wide-angle image 202 corresponds to the field of view of the wide-angle lens camera at the time of capturing wide-angle image 202);
receiving second image data from a second image sensor of a second camera having a second lens type different from the first lens type ([0040], [0041], FIG. 1, taking mages using wide- angle lens camera 102 and telephoto lens camera 104 are mounted at different heights), wherein a first field of view of the second image data overlaps at least a portion of a second field of view of the first image data ([0047]-[0048], FIG. 3, a stereoscopic imaging system and an object in a tracking environment with imaging system 306 of stereo camera assembly 307 is connected to platform 304. Stereo camera assembly 307 comprises wide-angle lens camera 308 and telephoto lens camera 310 and the wide-angle lens camera 308 comprises a wide-angle lens and an image sensor. Telephoto lens camera 310 comprises a telephoto lens and an image sensor); and
calculating a distance between the first camera and the point in space based on the first lens type of the first camera and the second lens type of the second camera ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0037], set distance between wide-angle lens camera 102 and telephoto lens camera 104, and a mounting angle for each of wide-angle lens camera 102 and telephoto lens camera 104; [0038]; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from the telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307).
PNG
media_image1.png
388
328
media_image1.png
Greyscale
Heafitz further discloses that three-dimensional position 1254 is determined by performing triangulation on object 1202 identified in stereoscopic images 1214 that is performed using the positions of wide-angle lens camera 1222 and telephoto lens camera 1224 in stereoscopic imaging system 1210 and transformation 1242 ([0145])
However, Heafitz does not explicitly disclose determining a point in space based on a match between pixels of the first image data and pixels of the second image data and wherein calculating the distance between the first camera and the point in space comprises: estimating a disparity between the first image data and the second image data and calculating the distance between the first camera and the point in space based on the estimated disparity
Park teaches determining a point in space based on a match between pixels of the first image data and pixels of the second image data ([0021], large disparities between pixels in the two images may indicate objects or surfaces that are closer and small disparities may indicate objects or surfaces that are further away and generate dense pixel correspondences across left and right camera image data of the pairwise images);
and wherein calculating the distance between the first camera and the point in space comprises: estimating a disparity between the first image data ([0031], FIG. 2, perform cross-camera OF tracking at least in the overlapping FOV region 210 of the cameras 202 and 204 and determine a distance to the particular location based on a disparity between pixels in the camera image data in the overlapping FOV region 210 for a particular location)and the second image data and calculating the distance between the first camera and the point in space based on the estimated disparity ([0030], Based on a disparity between pixels in left and right camera image data in the overlapping FOV for a particular location, the optical flow tracker 120 may determine a distance to the particular location; [0031], FIG. 2, perform cross-camera OF tracking at least in the overlapping FOV region 210 of the cameras 202 and 204 and determine a distance to the particular location based on a disparity between pixels in the camera image data in the overlapping FOV region 210 for a particular location. This depth information may then be used to generate dense pixel correspondences across camera image data (e.g., the pairwise images) from the cameras 202 and 204).
PNG
media_image2.png
192
466
media_image2.png
Greyscale
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of determining a point in space based on a match between pixels of the first image data and pixels of the second image data as taught by Park ([0121]) into the imaging system of Heafitz in order to provide systems for adequate hazard detection system to be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow ego-vehicle enough time to avoid the hazard in an efficient manner and generate the depth map by carving or cropping out regions of interest such as regions that correspond to a driving surface in order to minimize search space (Park, [0002]).
Regarding Claim 2, Heafitz in view of Park discloses the method of claim 1, Heafitz discloses wherein the first lens type of the first camera is a fisheye lens type and wherein the second lens type of the second camera is a projective lens type ([0040], [0041], FIG. 1, taking mages using wide- angle lens camera 102 and telephoto lens camera 104 are mounted at different heights; [0046], FIG. 2).
Regarding Claim 3, Heafitz in view of Park discloses the method of claim 1, Heafitz discloses wherein calculating a distance between the first camera and the point in space comprises: rectifying the first image data; rotating the first image data; rotating the second image data ([0046], FIG. 2, field of view of telephoto image 204 corresponds to the field of view of the telephoto lens camera at the time of capturing telephoto image 204 & field of view of wide-angle image 202 corresponds to the field of view of the wide-angle lens camera at the time of capturing wide-angle image 202);
Regarding Claim 4, Heafitz in view of Park discloses the method of claim 3, Heafitz discloses wherein rectifying the first image data comprises performing a cylindrical correction on the first image data ([0046], FIG. 2, field of view of telephoto image 204 corresponds to the field of view of the telephoto lens camera at the time of capturing telephoto image 204 & field of view of wide-angle image 202 corresponds to the field of view of the wide-angle lens camera at the time of capturing wide-angle image 202).
Regarding Claim 5, Heafitz in view of Park discloses the method of claim 1, Heafitz discloses wherein a first position of the first camera is vertically offset from a second position of the second camera ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from the telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307).
Regarding Claim 6, Heafitz in view of Park discloses the method of claim 5, Heafitz discloses wherein the first position of the first camera is vertically offset from the second position of the second camera by a distance greater than one meter ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307)
Regarding Claim 7, Heafitz in view of Park discloses the method of claim 5, Heafitz discloses wherein calculating the distance between the first camera and the point in space is further based on the vertical offset between the first position and the second position ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from the telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307)
Regarding Claim 8, Heafitz in view of Park discloses the method of claim 7, Heafitz discloses wherein the first position of the first camera is horizontally offset from the second position of the second camera, and wherein calculating the distance between the first camera and the point in space is further based on the horizontal offset between the first position and the second position ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from the telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307).
Regarding Claim 9, Heafitz in view of Park discloses the method of claim 1, Heafitz discloses further comprising calculating a distance between the second camera and the point in space based on the first lens type and the second lens type ([0029], recognize and take into account that stereoscopic vision utilizes two cameras with stereo imaging an object is identified in two images to determine depth, location in three-dimensional space; [0061], FIG. 5, when object 302 is identifiably present in both a telephoto image and a wide-angle image, stereoscopic information used to calculate a three-dimensional coordinate of object 302 can be determined from the telephoto image and the wide-angle image. When object 302 is identifiably present in both a telephoto image and a wide-angle image, such as when object is at position 502, object tracking can be performed using the telephoto image and wide-angle image from stereo camera assembly 307)
Regarding Claims 10-18, Apparatus claims 10-18 of using the corresponding method claimed in claims 1-9, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claims 19-24, Computer-readable medium claims 19-24 of using the corresponding method claimed in claims 1-8, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claims 25-30, Apparatus claims 25-30 of using the corresponding method claimed in claims 1-8, and the rejections of which are incorporated herein for the same reasons as used above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samuel D Fereja whose telephone number is (469)295-9243. The examiner can normally be reached 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DAVID CZEKAJ can be reached at (571) 272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMUEL D FEREJA/Primary Examiner, Art Unit 2487
Prosecution Insights