DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 4-7, 10-13 and 16-19 are pending.
Claims 2-3, 8-9 and 14-15 are canceled.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1, 4, 7, 10, 13, 16 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakinami et al (US20080137940) in view of Barath et al (Homography, 2019 (IDS)).
Regarding claims 1, 7 and 13, Kakinami teaches a fundamental matrix generation apparatus comprising: at least one memory that is configured to store instructions; and at least one processor that is configured to execute the instructions to:
detect, from a first image and a second image, three or more feature point pairs that pairs of feature points corresponding to each other;
(Kakinami, “a feature point detection means detecting feature points of an object image in the first image plane and the second image plane; a fundamental matrix determination means determining a fundamental matrix expressing a geometrically corresponding relationship, based on translational camera movement, between the first image plane and the second image plane, from not less than two pairs of feature points corresponding between the first image plane and the second image plane, on the basis of calculation of epipoles through auto-epipolar property”, [claim 1], “the fundamental matrix determination means generates temporary fundamental matrices”, [claim 5]; the system first detects feature points in two image planes (from different viewpoints); it then repeatedly selects at least two pairs of corresponding feature points (e.g., 4 pairs), and using assumptions such as pure translational motion and the auto-epipolar property to generate multiple temporary fundamental matrices (e.g., generated 2 temporary fundamental matrices from two differently selected 4-pair sets); these are not fully determined fundamental matrices constrained by motion characteristics)
detect, for each of the feature point pairs, a derived point pair that is a pair of a point separated by a first distance in a first direction from a point on the first image included in the feature point pair and a point separated by a second distance in a second direction from a point on the second image included in the feature point pair; and
...
wherein the first direction and the first distance are each determined based on a feature value computed for the point on the first image included in the feature point pair,
wherein the second direction and the second distance are determined based on a feature value computed for the point on the second image included in the feature point pair,
(Kakinami, “takes feature points in which the distance between a straight line connecting the feature points corresponding to one another in the first image plane and the second image plane and the epipole is less than or equal to a predetermined value as properly-corresponding feature points”, [claim 5]; for each generated temporary fundamental matrix (of 2 generated ones), the system evaluates all other detected feature points to identify/derive “properly-corresponding feature points”, defined as pairs whose epipolar lines (derived from the matrix) pass within a certain distance of the epipoles; the system counts the number of properly-corresponding feature points for each matrix; obviously, the other detected feature points may generally have different locations (distances) and directions than the two 4-pair feature points selected for generating temporary fundamental matrices)
Kakinami further teaches:
generate a fundamental matrix representing an epipolar constraint between a point on the first image and a point on the second image using each of the detected feature point pairs and the detected derived point pairs,
(Kakinami, “determines the temporary fundamental matrix in which the number of properly-corresponding feature points is greatest as an official fundamental matrix”, [claim 5]; one of the temporary matrices with the highest number of properly-corresponding feature points is selected as the final (official) fundamental matrix; thus the final fundamental matrix is determined based on a particular set of 4 pairs of feature points and other feature points qualified as properly-corresponding feature points selected above)
Kakinami does not expressly disclose but Barath teaches:
wherein the first direction is determined based on a main-axis direction of a scale-invariant feature value computed for a point on the first image,
(Barath, Fig. 1, “The rotation of the feature in the i-th image is αi and the size is qi (i ∈ {1,2})”; “In case of having orientation- and scale-covariant features, e.g. SIFT, the known parameters are the rotation αi of the feature in the i-th image and a uniform scale qi”, “Reflecting the fact that we are given a scale qi ∈ ℝ and rotation αi ∈ [0,2π) independently in each image (i ∈ {1,2}; see Fig. 1)”, sec. 3.1, p3:c1; “A technique ... approximates the epipolar geometry from one or two affine correspondences by converting them to point pairs”, sec. 1, p1:c1; using orientation- and scale-covariant feature detectors, e.g. SIFT which provide a rotation or angle"; this rotation may correspond to the "main-axis direction" of the scale-invariant feature (SIFT); converting the affine correspondences into point pairs, and determining directional components for point generation based on the feature's rotation)
wherein the first distance is determined based on a length of a scale of the scale-invariant feature value computed for the point on the first image,
(Barath, Fig. 1, “The rotation of the feature in the i-th image is αi and the size is qi (i ∈ {1,2})”; “In case of having orientation- and scale-covariant features, e.g. SIFT, the known parameters are the rotation αi of the feature in the i-th image and a uniform scale qi”, sec. 3.1, p3:c1; also, “When using covariant features, e.g. SIFT, the information about the scale and orientation is given at no cost”, [abstract]; obtaining the size or scale (qi) from the scale-invariant feature (SIFT); this corresponds to the length of a scale used to define the feature's geometry)
wherein the second direction is determined based on a main-axis direction of a scale of a scale-invariant feature value computed for a point on the second image, and
(Barath, Fig. 1, “The rotation of the feature in the i-th image is αi and the size is qi (i ∈ {1,2})”; “Reflecting the fact that we are given a scale qi ∈ ℝ and rotation αi ∈ [0,2π) independently in each image (i ∈ {1,2}; see Fig. 1)”, sec. 3.1, p3:c1; the rotation/orientation (direction) is determined independently in each image, thus teaching the determination for the second image)
wherein the second distance is determined based on a length of a scale of the scale-invariant feature value computed for the point on the second image.
(Barath, Fig. 1, “The rotation of the feature in the i-th image is αi and the size is qi (i ∈ {1,2}). The scaling from the 1st to the 2nd image is calculated as q=q2/q1”; determining the scale/size (qi) independently for the point on the second image)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Barath’s feature-based estimation method into Kakinami’s vehicle object recognition system to improve the stability and efficiency of the fundamental matrix calculation. Kakinami aims to recognize objects using epipolar geometry with low computational load but faces challenges with stability when tracking points; Barath provides a direct solution by demonstrating that utilizing the scale and orientation data from feature detectors (like SIFT) allows for robust geometric estimation using fewer point pairs. Combining these would allow Kakinami to achieve its goal of stable, real-time object detection with significantly reduced data requirements. The combination of Kakinami and Barath also teaches other enhanced capabilities.
Regarding claims 4, 10 and 16, the combination of Kakinami and Barath teaches its/their respective base claim(s).
The combination further teaches the fundamental matrix generation apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions further to:
generating the fundamental matrix while changing the feature point pair used to detect the derived point pair; and
outputting a fundamental matrix with highest accuracy among a plurality of the generated fundamental matrices.
(Kakinami, see comments on claim 1; obviously, the feature points used to transform the temporary fundamental matrices (e.g., 2) into the final fundamental matrices may be the best feature points which do not necessarily include any of the 2 sets of the 4 pairs of feature points used for generating the 2 temporary fundamental matrices)
Regarding claim 19, Kakinami teaches a fundamental matrix generation apparatus comprising: at least one memory that is configured to store instructions; and at least one processor that is configured to execute the instructions to:
detect, from a first image and a second image, three or more feature point pairs that pairs of feature points corresponding to each other;
detect, for each of the feature point pairs, a derived point pair that is a pair of a point separated by a first distance in a first direction from a point on the first image included in the feature point pair and a point separated by a second distance in a second direction from a point on the second image included in the feature point pair; and
generate a fundamental matrix representing an epipolar constraint between a point on the first image and a point on the second image using each of the detected feature point pairs and the detected derived point pairs,
wherein the first direction and the first distance are determined based on a feature value computed for the point on the first image included in the feature point pair,
wherein the second direction and the second distance are determined based on a feature value computed for the point on the second image included in the feature point pair,
(Kakinami, see comments on claim 1)
Kakinami does not expressly disclose but Barath teaches:
wherein the first direction is determined based on a specific-axis direction of an affine-invariant feature value computed for a point on the first image,
(Barath, “For the i-th Jacobian, the following is a possible decomposition: Ji = RiUi = ... (eq. (5)) ... where angle αi is the rotation in the i-th image”, sec. 3.1, p3:c1; “Most of the widely-used feature detectors provide parts of the affine feature. For instance, there are detectors obtaining oriented features, e.g. ORB... or there are ones providing also the scales, e.g. SIFT... or SURF”, sec. 1, p2:c1; decomposing the Jacobian (representing the affine feature) into components including a rotation angle αi, which corresponds to the specific-axis direction of the feature in the first image (i=1))
wherein the first distance is determined based on a length of that axis of the affine-invariant feature value computed for the point on the first image,
(Barath, “For the i-th Jacobian, the following is a possible decomposition: Ji = RiUi = ... (eq. (5)) ... where ... q{u,i} and q{v,i} are the scales along axes u and v...”, sec. 3.1, p3:c1; “Affine correspondences encode higher-order information about the scene geometry”, sec. 1, p1:c2; the affine feature decomposition includes scales along axes u and v (q{u,i}, q{v,i}). These scales correspond to the length of that axis used to define the geometry of the feature for the first image (i=1))
wherein the second direction is determined based on a specific-axis direction of an affine-invariant feature value computed for a point on the second image, and
(Barath, “For the i-th Jacobian, the following is a possible decomposition: Ji = RiUi = ... (eq. (5)) ... where angle αi is the rotation in the i-th image”, sec. 3.1, p3:c1; computing the affine decomposition, including the rotation/direction αi for the second image (i=2))
wherein the second distance is determined based on a length of that axis of the affine-invariant feature value computed for the point on the second image.
(Barath, “For the i-th Jacobian, the following is a possible decomposition: Ji = RiUi = ... (eq. (5)) ... where ... q{u,i} and q{v,i} are the scales along axes u and v...”, sec. 3.1, p3:c1; computing the affine decomposition, including the scales/lengths q{u,i} and q{v,i} along the axes, for the second image (i=2))
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Barath’s affine-invariant feature processing into the system or method of Kakinami’s object recognition system to improve the stability and efficiency of the fundamental matrix calculation. Kakinami aims to achieve robust detection with a low computational load but struggles with point tracking; Barath provides a solution by demonstrating that utilizing the specific axis direction and length from affine features allows for accurate geometric estimation using significantly fewer correspondences. Combining these would allow Kakinami to achieve its goal of stable, real-time object detection with reduced data requirements by leveraging the richer geometric constraints taught by Barath. The combination of Kakinami, Barath and Whitehead also teaches other enhanced capabilities.
Claim(s) 6, 12 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakinami et al (US20080137940) in view of Barath et al (Homography, 2019 (IDS)) and further in view of Whitehead et al (Intrinsic Camera Parameters, 2004).
Regarding claims 6, 12 and 18, the combination of Kakinami and Barath teaches its/their respective base claim(s).
The combination does not expressly disclose but Whitehead teaches the fundamental matrix generation apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions further to
estimate internal parameters of a camera that has generated the first image and a camera that has generated the second image by using each of the detected feature point pairs and derived point pairs.
(Whitehead, essential matrix E is the calibrated form of the fundamental matrix and may be determined by eq. (5) about the epipolar constraint and eq. (6) about camera extrinsic parameters, sec. 2.3; “the calibration matrix (K), containing the internal parameters of this camera (focal length, pixel dimensions, etc.)”, sec. 2.2; “The essential matrix can be considered as the calibrated version of the fundamental matrix”, eq. (8); when the fundamental matrix F and essential matrix E are known, the camera calibration matrix K (camera internal parameters) may be obtained from eq. (8); since the fundamental matrix F are determined from feature point pairs such as those of Kakinami which may include the initial feature points used for determining the temporary fundamental matrix and the additional feature points together with the initial feature points for generating the official fundamental matrix (see comments on claim 1))
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Whitehead into the modified system or method of Kakinami and Barath in order to provide the camera internal parameters (focal length, etc.) needed for Kakinami's 3D reconstruction. Barath provides a method to derive these parameters directly from the fundamental matrix (which R0/R1 already computes), eliminating the need for separate manual calibration and thereby making the vehicle recognition system more versatile and easier to deploy. The combination of Kakinami, Barath and Whitehead also teaches other enhanced capabilities.
Allowable Subject Matter
Claim(s) 5, 11 and 17 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening Claim(s).
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 5, 11 and 17 recite(s) the limitation(s) directed to computing signed areas of point pairs to determine whether or not to generate the fundamental matrix. There are no explicit teachings are found in the prior art cited in this office action and from the prior art search.
Response to Arguments
Applicant's arguments filed on 12/8/2026 with respect to one or more of the pending claims have been fully considered but are moot in view of the new ground(s) of rejection using a prior art reference filed in the IDS on 9/22/2025. Per MPEP 609.04(b).II.A.2. “The information submitted with a statement under 37 CFR 1.97(e) can be used in a new ground of rejection and the next Office action can be made final, if the new ground of rejection was necessitated by amendment of the application by applicant...”, the office action is made final.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center. for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 1/22/2026