Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 10-16 and 19-20 are rejected under 35 U.S.C. 102(a)(1)(2)as being anticipated by Micovic et al. (US 2016/0093029 A1) (herein after referred to as Micovic).
Regarding claim 1, Micovic teaches an apparatus for detecting objects, the apparatus comprising: at least one memory (Micovic, Fig 2, image buffers 208A, 208B, 208C); and
at least one processor (Micovic, Fig 2, paragraph [0020]) coupled to the at least one memory (Micovic, Fig 2, image buffers 208A, 208B, 208C) and configured to:
generate first image features based on a first image (Micovic, Fig 2, paragraph [0021-0022],..three images come into respective buffers 208A, 208B, 208C from a camera pipeline 204…);
generate second image features based on a second image (Micovic, Fig 2, paragraph [0021-0022],...three images come into respective buffers 208A, 208B, 208C from a camera pipeline 204…);
align the first image features and the second image features to generate aligned
features (Micovic, Fig 2, paragraph [0024-0029] {these paragraphs describe the entire process of alignment]);
estimate motion parameters based on the aligned features (Micovic, Fig 2, Multi-Resolution Alignment Estimations Modules 210A, 210B, paragraph [0025-0026], …the alignment estimation module, calculates motion between the reference image and the other two or more images….);
adjust the aligned features based on the motion parameters to generate adjusted
aligned image features (Micovic, Fig 2, Multi-Resolution Alignment Estimations Modules 210A, 210B, Warping engines 212A, 212B, paragraph [0025-0026], …the alignment estimation module, calculates motion between the reference image and the other two or more images….;; and
detect an object based on the adjusted aligned image features (Micovic, Fig 2, paragraph [0027-0028], …as shown in Fig 1, the subject 114 shown in the foreground has moved ….De-ghosting, shifts the subject position in the over exposed image to match the position of the subject in the reference image.).
Regarding claim 2, Micovic teaches the apparatus of claim 1, wherein the at least one processor is further configured to vectorize and pool the first image features and the second image features (Micovic, paragraph [0026], …A warping and scaling matrix may be used).
Regarding claim 3, Micovic teaches the apparatus of claim 1, wherein the at least one processor is further configured to apply transport optimization to determine correspondence between the first image features and the second image features (Micovic, paragraphs [0040-0049], Histogram mapping and matching are performed then passed to an exclusion mask remove erosion, dilation and smooth the edges for seamless blending.).
Regarding claim 4, Micovic teaches the apparatus of claim 1, wherein the at least one processor is further configured to determine translation-and-rotation parameters based on the motion parameters, wherein the first image features are adjusted based on the translation-and-rotation parameters (Micovic, Fig 2, Multi-Resolution Alignment Estimations Modules 210A, 210B, paragraph [0025-0026], …the alignment estimation module, calculates motion between the reference image and the other two or more images…. and paragraph [0052]).
Regarding claim 10, Micovic teaches the apparatus of claim 1, wherein the first image and the second image comprise sequential images captured by a rolling-shutter camera (Micovic, paragraph [0019]).
Regarding claim 11, Micovic teaches the apparatus of claim 1, wherein the at least one processor is further configured to, based on the detected object, adjust an operational parameter of a vehicle (Micovic, paragraph [0018 and 0025]).
Regarding claim 12, Micovic teaches the apparatus of claim 11, wherein the detected object comprises at least one of: a road feature; a street sign; debris; a vehicle; a pedestrian; an animal; a plant; or a structure (Micovic, paragraph [0018 and 0025], fixed mountain back ground).
Claim 13 is rejected for the same reasons as claim1.
Claim 14 is rejected for the same reasons as claim 2.
Claim 15 is rejected for the same reasons as claim 3.
Claim 16 is rejected for the same reasons as claim 4.
Regarding claim 19, Micovic teaches the method of claim 13, further comprising, based on the detected object, adjusting an operational parameter of a vehicle (Micovic, paragraph [0018 and 0025]).
Regarding claim 20, Micovic teaches the method of claim 19, wherein the detected object comprises at least one of: a road feature; a street sign; debris; a vehicle; a pedestrian; an animal;
a plant; or a structure (Micovic, paragraph [0018], …mountain back ground…snowboard rider).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5-9 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Micovic et al. (US 2016/0093029 A1) (herein after referred to as Micovic) in view of Wong et al. (US 2021/0004933 A1) (herein after referred to as Wong).
Regarding claim 5, Micovic teaches the apparatus of claim 1, but does not explicitly teach wherein the at least one processor is further configured to: determine depth information based on the first image;
determine translation-and-rotation parameters based on the motion parameters; and
convert the first image features into a displacement vector based on the depth information and the translation-and-rotation parameters.
Wong teaches a system for generating a panorama image that discloses wherein the at least one processor is further configured to: determine depth information based on the first image (Wong, paragraphs [0034];
determine translation-and-rotation parameters based on the motion parameters; and
convert the first image features into a displacement vector based on the depth information and the translation-and-rotation parameters (Wong, paragraphs [0061 and 0076 – 0080]).
These arts are analogous since they are both related to imaging devices that align images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Micovic with the at least one processor is further configured to: determine a point cloud based on the second image; and project the point cloud onto the first image using the motion parameters as seen in Wong to align images taken for a clearer composite image as seen in (Wong Abstract, among other citations in the specification of Wong).
Regarding claim 6, Micovic teaches the apparatus of claim 1, however, Micovic does not explicitly teach wherein the at least one processor is further configured to: determine a point cloud based on the second image; and project the point cloud onto the first image using the motion parameters.
Wong teaches a system for generating a panorama image that discloses wherein the at least one processor is further configured to: determine a point cloud based on the second image; and project the point cloud onto the first image using the motion parameters (Wong, paragraph [0122-0123 and 0141].
These arts are analogous since they are both related to imaging devices that align images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Micovic with the at least one processor is further configured to: determine a point cloud based on the second image; and project the point cloud onto the first image using the motion parameters as seen in Wong to align images taken for a clearer composite image as seen in (Wong Abstract, among other citations in the specification of Wong).
Regarding claim 7, Micovic teaches the apparatus of claim 1, however, Micovic does not explicitly teach wherein, to detect the object based on the adjusted aligned image features, the at least one processor is configured to process the adjusted aligned image features using a three-dimensional object-detection decoder to generate a bounding box of the object.
Wong teaches a system for generating a panorama image that discloses wherein to detect the object based on the adjusted aligned image features, the at least one processor is configured to process the adjusted aligned image features using a three-dimensional object-detection decoder to generate a bounding box of the object (Wong paragraph [0010, 0061 and 0076 – 0080]).
These arts are analogous since they are both related to imaging devices that align images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Micovic with the detect the object based on the adjusted aligned image features, the at least one processor is configured to process the adjusted aligned image features using a three-dimensional object-detection decoder to generate a bounding box of the object as seen in (Wong Abstract, among other citations in the specification of Wong).
Regarding claim 8, Micovic teaches the apparatus of claim 1, however, Micovic does not explicitly teach wherein the motion parameters are estimated using an optical flow technique.
Wong teaches a system for generating a panorama image that discloses wherein the motion parameters are estimated using an optical flow technique (Wong paragraph [0061 and 0076 – 0080]).
These arts are analogous since they are both related to imaging devices that align images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Micovic with wherein the motion parameters are estimated using an optical flow technique as seen in (Wong Abstract, among other citations in the specification of Wong).
Regarding claim 9, Micovic teaches the apparatus of claim 1, however, Micovic does not explicitly teach wherein the motion parameters are estimated using a recurrent all-pairs field transforms (RAFT) optical-flow technique.
Wong teaches a system for generating a panorama image that discloses wherein the motion parameters are estimated using a recurrent all-pairs field transforms (RAFT) optical-flow technique (Wong paragraph [0061 and 0074]).
These arts are analogous since they are both related to imaging devices that align images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Micovic with wherein the motion parameters are estimated using an optical flow technique as seen in (Wong Abstract, among other citations in the specification of Wong).
Claim 17 is rejected for the same reasons as claim 5.
Claim 18 is rejected for the same reasons as claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the Supervisory Patent Examiner should be directed to TWYLER L. HASKINS whose telephone number is (571)272-7406. The Supervisory Patent Examiner can normally be reached Mon-Thurs: 7:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Supervisory Patent Examiner by telephone are unsuccessful, the Supervisory Patent Examiner’s Group Director, Greg Toatley can be reached at (571) 272-4650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TWYLER L HASKINS/ Supervisory Patent Examiner, Art Unit 2639