DETAILED ACTION
Introduction
Claims 1-21 have been examined in this application. Claims 1, 4, 5, 9-11, 14, 15, 19, and 20 are amended. Claims 2, 3, 6-8, 12, 13, 16-18, and 21 are original. This is a final office action in response to the arguments and amendments filed 2/5/2026. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Office Action Formatting
The following is an explanation of the formatting used in the instant Office Action:
• [0001] – Indicates a paragraph number in the most recent, previously cited source;
• [0001, 0010] – Indicates multiple paragraphs (in example: paragraphs 1 and 10) in the most recent, previously cited source;
• [0001-0010] – Indicates a range of paragraphs (in example: paragraphs 1 through 10) in the most recent, previously cited source;
• 1:1 – Indicates a column number and a line number (in example: column 1, line 1) in the most recent, previously cited source;
• 1:1, 2:1 – Indicates multiple column and line numbers (in example, column 1, line 1 and column 2, line 2) in the most recent, previously cited source;
• 1:1-10 – Indicates a range of lines within one column (in example: all lines spanning, and including, lines 1 and 10 in column 1) in the most recent, previously cited source;
• 1:1-2:1 – Indicates a range of lines spanning several columns (in example: column 1, line 1 to column 2, line 1 and including all intervening lines) in the most recent, previously cited source;
• p. 1, ln. 1 – Indicates a page and line number in the most recent, previously cited source;
• ¶1 – The paragraph symbol is used solely to refer to Applicant's own specification (further example: p. 1, ¶1 indicates first paragraph of page 1); and
• BRI – the broadest reasonable interpretation.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on application KR1020230010992 filed in The Republic of Korea on 01/27/2023. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Arguments
Applicant's arguments, filed 2/5/2026, have been fully considered.
Regarding the remarks pertaining to the claim objections (presented on p. 9), the amendments are acceptable. Therefore, the objections have been withdrawn.
Regarding the arguments pertaining to the claim rejections under 112 (presented on p. 9), the arguments and amendments are persuasive. Therefore, the rejections have been withdrawn.
Regarding the arguments pertaining to the claim rejections under 102 and 103 (presented on p. 9-12), the arguments and amendments are persuasive. Therefore, the rejections have been withdrawn. However, upon further consideration, a new grounds of rejection is made in view of the additional prior art of Publication US2022/0155075A1 (Jeon et al.) as well as the previously relied upon prior art of US2023/0406209A1 (Park et al.), KR20170055738A (Lee), KR102174011B1 (Kim), and NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding Claims 1 and 14, the claims recite a limitation to determine “an inaccuracy of the camera and the IMU device.” The disclosure as originally filed (e.g. original claims 9-11 and 19-21, specification p. 3, 5, 9, 13) appears to only recite determinations that an estimated position is inaccurate. Although these estimated positions may be based on camera data or IMU data, the inaccuracy is of the position and not the camera or the IMU device. Therefore, the subject matter was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 2-13 and 15-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as being dependent on rejected Claim 1 or 14 and for failing to cure the deficiencies listed above.
Regarding Claims 9 and 19, the claims recite a limitation to compare an average value, of a differences between the plurality of first estimated vehicle positions and the plurality of second estimated vehicle positions, to a threshold value (where, as previously recited in the claim, the plurality of second estimated vehicle positions have been determined by adjusting the plurality of first estimated vehicle positions). The disclosure as originally filed, in regards to an average value of plural errors / differences (e.g. specification p. 13, 14, with respect to Equation 1, and p. 21) recites the average of differences between vehicle position estimated based on the measurement of the IMU, and vehicle position estimated based on the image obtained from the camera. However, this is not consistent with the claim language, which specifies the difference is between first positions estimated from the camera and second positions using the IMU to adjust the first, camera-based positions (as opposed to second positions which are only estimated from the IMU, as disclosed). Therefore, the subject matter was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 1 and 14, the issues under 112(a) also render the claims indefinite. It is not clear whether the inaccuracy should be understood to be that of the camera and IMU themselves, or alternatively of the vehicle position data or vehicle position difference data, or of the first and second estimated vehicle positions, or something else. The scope of the claims is therefore indefinite. For the purposes of examination, the inaccuracy is interpreted as referring to the position estimate by the camera or IMU as opposed to the camera or IMU device itself.
Claims 2-13 and 15-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being dependent on rejected Claim 1 or 14 and for failing to cure the deficiencies listed above.
Regarding Claims 9 and 19, the issues under 112(a) also render the claims indefinite. It is not clear, in view of the discrepancy between the claims and specification, whether the vehicle positions used in the average calculation should be understood to be based on positions using only camera data and positions using only IMU data, or alternatively using some adjusted position that relies on both camera and IMU data (it is noted that in the specification on p. 13-15, the adjustment occurs as a result of the evaluation of the average, as opposed to the claim where the average calculation is based on already adjusted position data). The scope of the claims is therefore indefinite. For the purposes of examination, the claims are interpreted as referring to a first plurality of positions based on the camera and a second plurality of positions based on the IMU, as best understood in view of the specification.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 9-11 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Publication US2022/0155075A1 (Jeon et al.).
Regarding Claim 1, Jeon et al. discloses a vehicle (see [0057-0058]) comprising:
a camera mounted on the vehicle to obtain an image outside the vehicle (see [0059, 0080]);
an inertial measurement unit (IMU) device (see [0059] IMU as position sensor); and
a processor (see [0100] navigation apparatus implemented on processor) configured to:
estimate, based on the image obtained from the camera, a vehicle position of the vehicle to yield a first estimated vehicle position (see Figure 2, [0063] determine image-based position);
adjust, based on a measurement of the IMU device (see Figure 2, [0064] fusion in 240 based on sensor-based estimated position, which [0062] may be IMU measurement), the first estimated vehicle position to yield a second estimated vehicle position (see [0064] derive final position (second estimated) based on the image-based estimated position);
determine an inaccuracy of the camera and the IMU device (see [0069] image-based estimated position applied in deriving final position (i.e. neither camera and IMU position inaccurate) or not utilized (i.e. camera inaccurate and IMU accurate)), based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera (see [0068] changes in heading angles, of image-based positions in consecutive time steps), and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] compared to outputs of gyroscopes ([0059] of IMU)); and
cause, based on the at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy (see [0068-0069] camera and IMU positions and inaccuracy to determine final position), the vehicle to adjust at least one of: acceleration, steering, or braking (see [0025] control vehicle in response to position, [0108] acceleration, steering, and braking).
Regarding Claim 9, Jeon et al. discloses the vehicle of claim 1, wherein the processor is further configured to:
determine, using the camera over a period of time, a plurality of first estimated vehicle positions of the vehicle (see [0068] image-based estimated positions in consecutive time steps);
determine, using the IMU device over the period of time, a plurality of second estimated vehicle positions by adjusting the plurality of first estimated vehicle positions (see [0068] outputs of gyroscope ([0059] of IMU) comparable to image based data i.e. for the time steps);
compare an average value, of differences between the plurality of first estimated vehicle positions and the plurality of second estimated vehicle positions (see [0068] changes of positions calculated from image-based position and outputs of gyroscope, changes compared, and “average” for the case of only two time steps), to a threshold value (see [0068] determine whether exceeds threshold); and
determine, based on the comparison, whether the plurality of first estimated vehicle positions or the plurality of second estimated vehicle positions are inaccurate (see [0069] based on whether difference exceeds threshold, image-based estimated position utilized (i.e. neither first and second inaccurate) or not utilized (image based inaccurate and IMU accurate)).
Regarding Claim 10, Jeon et al. discloses the vehicle of claim 1,
wherein the processor is further configured to:
determine:
the first vehicle position difference, between the current frame and the previous frame, as measured by the camera (see [0068] changes in calculated heading from image-based position), and
the second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] as compared to gyroscope output, i.e. changes compared requiring change in heading from gyroscope ([0059] IMU)) ; and
based on a comparison between the first vehicle position difference and the second vehicle position difference (see [0068] determine whether difference exceeds threshold), determine that one of the first estimated vehicle position or the second estimated vehicle position is inaccurate (see [0069] image-based position not utilized (image based inaccurate and IMU accurate)).
Examiner's note: since the claim uses the conjunction "or," only one of the recited alternatives is necessary in the prior art to read on this claim.
Regarding Claim 11, Jeon et al. discloses the vehicle of claim 10, wherein the processor is further configured to perform one of:
based on a determination that the first estimated vehicle position is inaccurate (see [0069] image-based estimated position considered a fault), initializing the first estimated vehicle position with the second estimated vehicle position (see [0069] image-based position not utilized when deriving final, while [0064] IMU is used (see also applicant’s specification, initializing including adjusting or replacing)); or
based on a determination that the second estimated vehicle position is inaccurate, initializing the second estimated vehicle position with the first estimated vehicle position.
Examiner's note: since the claim uses the conjunction "or," only one of the recited alternatives is necessary in the prior art to read on this claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 (in the alternative) is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.).
Regarding Claim 1, Park et al. discloses a vehicle (see [0046]) comprising:
a camera mounted on the vehicle to obtain an image outside the vehicle (see [0047], Figure 2);
a processor (see [0047]) configured to:
estimate, based on the image obtained from the camera, a vehicle position of the vehicle to yield a first estimated vehicle position (see Figure 4, [0077-0080] operation 450, update e.g. pitch information of vehicle, based on images acquired in operation 410);
cause, based on the at least one of the first estimated vehicle position or the second estimated vehicle position, driving assistance (see Figure 4, [0083] operation 460 visualize blind spot portion of environment, [0050] for driving assistance)
Park et al. further discloses the use of current and previous frames (see [0063, 0066]) does not explicitly recite the vehicle comprising:
an inertial measurement unit (IMU) device; and
the processor to:
adjust, based on a measurement of the IMU device, the first estimated vehicle position to yield a second estimated vehicle position;
determine an inaccuracy of the camera and the IMU device, based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera, and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device; and
cause, based on the at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy, the vehicle to adjust at least one of: acceleration, steering, or braking.
However, Jeon et al. teaches a vehicle (see [0057-0058]) comprising:
an inertial measurement unit (IMU) device (see [0059] IMU as position sensor); and
the processor (see [0100] navigation apparatus implemented on processor) to:
adjust, based on a measurement of the IMU device (see Figure 2, [0064] fusion in 240 based on sensor-based estimated position, which [0062] may be IMU measurement), the first estimated vehicle position to yield a second estimated vehicle position (see [0064] derive final position (second estimated) based on the image-based estimated position);
determine an inaccuracy of the camera and the IMU device (see [0069] image-based estimated position applied in deriving final position (i.e. neither camera and IMU position inaccurate) or not utilized (i.e. camera inaccurate and IMU accurate)), based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera (see [0068] changes in heading angles, of image-based positions in consecutive time steps), and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] compared to outputs of gyroscopes ([0059] of IMU)); and
cause, based on the at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy (see [0068-0069] camera and IMU positions and inaccuracy to determine final position), the vehicle to adjust at least one of: acceleration, steering, or braking (see [0025] control vehicle in response to position, [0108] acceleration, steering, and braking).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the vehicle and detection of environment of Park et al. to further correct position based on an IMU and providing control as taught by Jeon et al., with a reasonable expectation of success, with the motivation of improving accuracy for uses in vehicle control and user guidance (see Jeon et al., [0058]).
Claims 2-4 and 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.), further in view of Publication KR20170055738A (Lee) (English description relied upon for citations).
Regarding Claim 2, Park et al. discloses wherein the processor is further configured to:
designate a feature point in the image (see [0067]).
Park et al. does not explicitly recite the vehicle of claim 1, wherein the processor is further configured to:
remove image noise from the image obtained from the camera; and
designate a region of interest in the image.
However, Lee teaches a technique for image analysis for a vehicle (see e.g. [0004]), including to:
remove image noise from the image obtained from the camera (see [0028]); and
designate a region of interest in the image (see [0028]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the extraction of a feature point of Park et al. to include extracting of lane features via the technique taught by Lee, with a reasonable expectation of success, with the motivation of improving accuracy by improving detection of road markings (see Lee, [0003-0004]).
Regarding Claim 3, Park et al. discloses wherein the processor is further configured to extract, from the image a feature point (see [0067]).
Park et al. does not explicitly recite the vehicle of claim 2, wherein the processor is further configured to extract, from the region of interest, a feature point.
However, Lee teaches the technique as above,
wherein the processor is further configured to extract, from the region of interest, a feature point (see [0028] extract lane component feature (points) from region of interest).
The motivation to combine Park et al. and Lee was provided in the rejection of Claim 2.
Regarding Claim 4, Park et al. discloses the vehicle of claim 3,
wherein the processor is further configured to:
track a change in position of the feature point across the previous frame and the current frame (see [0067]); and
perform matching of the feature point between the previous frame and the current frame (see [0068]).
Regarding Claim 6, Park et al. discloses the vehicle of claim 4, wherein the processor is further configured to estimate an essential matrix using a geometric relationship, in a normalized image plane, of the feature point between the previous frame and the current frame (see [0069]).
Regarding Claim 7, Park et al. discloses the vehicle of claim 6, wherein the processor is further configured to:
estimate, based on the essential matrix, a rotation matrix and a transformation matrix (see [0071]), and
wherein the processor is configured to estimate the vehicle position of the vehicle by estimating the vehicle position further based on the rotation matrix and the transformation matrix (see Figure 4, estimation of position in operation 450 based on matrices as part of operation 430).
Regarding Claim 8, Park et al. discloses the vehicle of claim 7, wherein the processor is further configured to estimate the vehicle position of the vehicle as facing up or down by estimating a pitch value based on the rotation matrix (see [0080] pitch information updated in operation 450 based on rotation matrix in 430).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.), further in view of Publication KR20170055738A (Lee) (English description relied upon for citations), further in view of Publication KR102174011B1 (Kim) (English description relied upon for citations).
Regarding Claim 5, Park et al. discloses wherein the processor is further configured to:
track a change in position of the feature point across the previous frame and the current frame(see [0067]).
Park et al. does not explicitly recite the vehicle of claim 3,
wherein the processor is further configured to:
based on a mismatch of the feature point between the previous frame and the current frame, remove the feature point.
However, Kim teaches a technique in navigation (see [0007]), configured to:
based on a mismatch of the feature point between the previous frame and the current frame, remove the feature point (see [0008]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the matching of points of Park et al. to remove point as taught by Kim, with a reasonable expectation of success, with the motivation of improving accuracy by preventing errors in matching (see Kim, [0004] ).
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0155075A1 (Jeon et al.) in view of NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.).
Regarding Claim 12, Jeon et al. does not explicitly recite the vehicle of claim 1, wherein the processor is further configured to:
adjust a point of a three-dimensional (3D) coordinate system of an actual ground area according to a difference between the first estimated vehicle position and the second estimated vehicle position;
obtain a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area; and
estimate, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area by a perspective transform matrix.
However, Zhang et al. teaches a vehicle operating in accordance to position updates (see p. 1164, Section 3.6 acting during pitch and traveling), configured to:
adjust a point of a three-dimensional (3D) coordinate system of an actual ground area (see p. 1164 road surface determined, Figure 6(b) with coordinate system);
obtain a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area (see p. 1164, section 3.6 camera coordinate system associated with ground area translation); and
estimate, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area by a perspective transform matrix (see Figure 6(a), 6(b), based on image plane intersect road ground surface (2D line of intersection), p. 1164, estimating road ground surface by matrix transformation of camera pitch/rotation).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the image-based position updates of Jeon et al. to be used to update camera parameters as taught by Zhang et al. with a reasonable expectation of success, with the motivation of improving accuracy of target detection and enabling ADAS systems (see Zhang et al., Abstract).
Regarding Claim 13, Jeon et al. does not explicitly recite the vehicle of claim 12, wherein the processor is further configured to estimate, based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image.
However, Zhang et al. teaches the vehicle as above,
wherein the processor is further configured to estimate,
based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image (see p. 1161, 1166, distance to targets using camera, and see Figure 2, based on optical axis reflecting actual ground area even with vehicle pitch).
The motivation to combine Jeon et al. and Zhang et al. was provided above in the rejection of Claim 12.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.), further in view of NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.).
Regarding Claim 14, Park et al. discloses a method comprising:
estimating, based on an image obtained via a camera (see [0047], Figure 2), a position of a vehicle to yield a first estimated vehicle position (see Figure 4, [0077-0080] operation 450, update e.g. pitch information of vehicle, based on images acquired in operation 410), and
causing, based on the at least one of the first estimated vehicle position or the second estimated vehicle position, driving assistance (see Figure 4, [0083] operation 460 visualize blind spot portion of environment, [0050] for driving assistance).
Park et al. does not explicitly recite:
adjusting, based on a measurement by an inertial measurement unit (IMU) device, the first estimated vehicle position to yield a second estimated vehicle position;
determining an inaccuracy of the camera and the IMU device, based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera, and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device; and
causing, based on at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy, the vehicle to adjust at least one of:
acceleration, steering, or braking.
However, Jeon et al. teaches a vehicle (see [0057-0058]) comprising:
adjusting, based on a measurement by an inertial measurement unit (IMU) device (see Figure 2, [0064] fusion in 240 based on sensor-based estimated position, which [0062] may be IMU measurement), the first estimated vehicle position to yield a second estimated vehicle position (see [0064] derive final position (second estimated) based on the image-based estimated position);
determining an inaccuracy of the camera and the IMU device (see [0069] image-based estimated position applied in deriving final position (i.e. neither camera and IMU position inaccurate) or not utilized (i.e. camera inaccurate and IMU accurate)), based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera (see [0068] changes in heading angles, of image-based positions in consecutive time steps), and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] compared to outputs of gyroscopes ([0059] of IMU)); and
causing, based on at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy (see [0068-0069] camera and IMU positions and inaccuracy to determine final position), the vehicle to adjust at least one of:
acceleration, steering, or braking (see [0025] control vehicle in response to position, [0108] acceleration, steering, and braking).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the vehicle and detection of environment of Park et al. to further correct position based on an IMU and providing control as taught by Jeon et al., with a reasonable expectation of success, with the motivation of improving accuracy for uses in vehicle control and user guidance (see Jeon et al., [0058]).
Park et al. does not explicitly recite:
based on a determination that the first estimated vehicle position is different from the second estimated vehicle position, adjusting a point of a three-dimensional (3D) coordinate system of an actual ground area according to a difference between the first estimated vehicle position and the second estimated vehicle position;
obtaining a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area;
estimating, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area; and
estimating, based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image.
However, Zhang et al. teaches a technique in a vehicle,
based on a determination that the first estimated vehicle position is different from the second estimated vehicle position (see p. 1164, Section 3.6 acting during pitch and traveling i.e. varying position), adjusting a point of a three-dimensional (3D) coordinate system of an actual ground area according to a difference between the first estimated vehicle position and the second estimated vehicle position (see p. 1164 road surface determined, Figure 6(b) with coordinate system);
obtaining a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area (see p. 1164, section 3.6 camera coordinate system associated with ground area translation);
estimating, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area (see Figure 6(a), 6(b), based on image plane intersect road ground surface (2D line of intersection), p. 1164, estimating road ground surface by matrix transformation of camera pitch/rotation);
estimating, based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image (see p. 1161, 1166, distance to targets using camera, and see Figure 2, based on optical axis reflecting actual ground area even with vehicle pitch).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the position updates of Park et al. to be used to update camera parameters as taught by Zhang et al. with a reasonable expectation of success, with the motivation of improving accuracy of target detection and enabling ADAS systems (see Zhang et al., Abstract).
Claims 15, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.), further in view of NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.), further in view of Publication KR20170055738A (Lee) (English description relied upon for citations).
Regarding Claim 15, Park et al. discloses
wherein the estimating of the position of the vehicle based on the image obtained via the camera comprises:
extracting, from the image, a feature point (see [0067]);
tracking a change in position of the feature point across the previous frame and the current frame (see [0067]); and
performing matching of the feature point between the previous frame and the current frame (see [0063, 0066, 0068]).
Park et al. does not explicitly recite the method of claim 14,
wherein the estimating of the position of the vehicle based on the image obtained via the camera comprises:
removing image noise from the image;
designating a region of interest in the image;
extracting, from the region of interest, a feature point.
However, Lee teaches a technique for image analysis for a vehicle (see e.g. [0004]), including to:
removing image noise from the image (see [0028]);
designating a region of interest in the image (see [0028]);
extracting, from the region of interest, a feature point (see [0028] extract lane component feature (points) from region of interest).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the extraction of a feature point of Park et al. to include extracting of lane features via the technique taught by Lee, with a reasonable expectation of success, with the motivation of improving accuracy by improving detection of road markings (see Lee, [0003-0004]).
Regarding Claim 17, Park et al. discloses the method of claim 15, further comprising:
estimating an essential matrix using a geometric relationship, in a normalized image plane, of the feature point between the previous frame and the current frame (see [0069]); and
estimating, based on the essential matrix, a rotation matrix and a transformation matrix (see [0071]),
wherein the estimating the position of the vehicle comprises estimating the position of the vehicle further based on the rotation matrix and the transformation matrix (see Figure 4, estimation of position in operation 450 based on matrices as part of operation 430).
Regarding Claim 18, Park et al. discloses the method of claim 17, wherein the estimating of the position of the vehicle comprises estimating the position of the vehicle as facing up or down by estimating a pitch value based on the rotation matrix (see [0080] pitch information updated in operation 450 based on rotation matrix in 430).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2023/0406209A1 (Park et al.) in view of Publication US2022/0155075A1 (Jeon et al.), further in view of NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.), further in view of Publication KR20170055738A (Lee) (English description relied upon for citations), further in view of Publication KR102174011B1 (Kim) (English description relied upon for citations).
Regarding Claim 16, Park et al. does not explicitly recite the method of claim 15, further comprising:
filtering to remove a mismatched feature point that does not match between the previous frame and the current frame.
However, Kim teaches a technique in navigation (see [0007]), configured to:
filtering to remove a mismatched feature point that does not match between the previous frame and the current frame (see [0008]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the matching of points of Park et al. to remove point as taught by Kim, with a reasonable expectation of success, with the motivation of improving accuracy by preventing errors in matching (see Kim, [0004] ).
Claim 14 (in the alternative) and Claims 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0155075A1 (Jeon et al.) in view of NPL Publication “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement” (Zhang et al.).
Regarding Claim 14, Jeon et al. discloses a method comprising:
estimating, based on an image obtained via a camera, a position of a vehicle to yield a first estimated vehicle position (see Figure 2, [0063] determine image-based position);
adjusting, based on a measurement by an inertial measurement unit (IMU) device (see Figure 2, [0064] fusion in 240 based on sensor-based estimated position, which [0062] may be IMU measurement), the first estimated vehicle position to yield a second estimated vehicle position (see [0064] derive final position (second estimated) based on the image-based estimated position);
determining an inaccuracy of the camera and the IMU device (see [0069] image-based estimated position applied in deriving final position (i.e. neither camera and IMU position inaccurate) or not utilized (i.e. camera inaccurate and IMU accurate)), based on a first vehicle position difference, between a current frame and a previous frame that are separated by a time interval, as measured by the camera (see [0068] changes in heading angles, of image-based positions in consecutive time steps), and based on a second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] compared to outputs of gyroscopes ([0059] of IMU));
causing, based on at least one of the first estimated vehicle position or the second estimated vehicle position, and based on the inaccuracy (see [0068-0069] camera and IMU positions and inaccuracy to determine final position), the vehicle to adjust at least one of:
acceleration, steering, or braking (see [0025] control vehicle in response to position, [0108] acceleration, steering, and braking).
Jeon et al. does not explicitly recite the method comprising:
based on a determination that the first estimated vehicle position is different from the second estimated vehicle position, adjusting a point of a three-dimensional (3D) coordinate system of an actual ground area according to a difference between the first estimated vehicle position and the second estimated vehicle position;
obtaining a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area;
estimating, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area; and
estimating, based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image.
However, Zhang et al. teaches a technique in a vehicle, including:
based on a determination that the first estimated vehicle position is different from the second estimated vehicle position (see p. 1164, Section 3.6 acting during pitch and traveling), adjusting a point of a three-dimensional (3D) coordinate system of an actual ground area according to a difference between the first estimated vehicle position and the second estimated vehicle position (see p. 1164 road surface determined, Figure 6(b) with coordinate system);;
obtaining a point of a camera coordinate system corresponding to the adjusted point of the 3D coordinate system of the actual ground area (see p. 1164, section 3.6 camera coordinate system associated with ground area translation);
estimating, based on a two-dimensional (2D) coordinate system of a ground area in the image, the 3D coordinate system of the actual ground area (see Figure 6(a), 6(b), based on image plane intersect road ground surface (2D line of intersection), p. 1164, estimating road ground surface by matrix transformation of camera pitch/rotation).
estimating, based on the estimated 3D coordinate system of the actual ground area, a distance to an actual object that corresponds to a position of an object in the image (see p. 1161, 1166, distance to targets using camera, and see Figure 2, based on optical axis reflecting actual ground area even with vehicle pitch).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the position updates of Jeon et al. to be used to update camera parameters as taught by Zhang et al. with a reasonable expectation of success, with the motivation of improving accuracy of target detection and enabling ADAS systems (see Zhang et al., Abstract).
Regarding Claim 19, Jeon et al. discloses the method of claim 14, wherein the adjusting of the first estimated vehicle position comprises:
determining, using the camera over a period of time, a plurality of first estimated vehicle positions of the vehicle (see [0068] image-based estimated positions in consecutive time steps);
determining, using the IMU device over the period of time, a plurality of second estimated vehicle positions by adjusting the plurality of first estimated vehicle positions (see [0068] outputs of gyroscope ([0059] of IMU) comparable to image based data i.e. for the time steps);
comparing an average value of differences between the plurality of first estimated vehicle positions and the plurality of second estimated vehicle positions (see [0068] changes of positions calculated from image-based position and outputs of gyroscope, changes compared, and “average” for the case of only two time steps), to a threshold value (see [0068] determine whether exceeds threshold); and
determining, based on the comparison, whether the plurality of first estimated vehicle positions or the plurality of second estimated vehicle positions are inaccurate (see [0069] based on whether difference exceeds threshold, image-based estimated position utilized (i.e. neither first and second inaccurate) or not utilized (image based inaccurate and IMU accurate)).
Regarding Claim 20, Jeon et al. discloses the method of claim 14 , wherein the method further comprises:
determining:
the first vehicle position difference, between the current frame and the previous frame, as measured by the camera (see [0068] changes in calculated heading from image-based position), and
the second vehicle position difference, between the current frame and the previous frame, as measured by the IMU device (see [0068] as compared to gyroscope output, i.e. changes compared requiring change in heading from gyroscope ([0059] IMU)); and
based on a comparison between the first vehicle position difference and the second vehicle position difference (see [0068] determine whether difference exceeds threshold), determining that one of the first estimated vehicle position or the second estimated vehicle position is inaccurate (see [0069] image-based position not utilized (image based inaccurate and IMU accurate)).
Examiner's note: since the claim uses the conjunction "or," only one of the recited alternatives is necessary in the prior art to read on this claim.
Regarding Claim 21, Jeon et al. discloses the method of claim 20, further comprising performing one of:
based on a determination that the first estimated vehicle position is inaccurate (see [0069] image-based estimated position considered a fault), initializing the first estimated vehicle position with the second estimated vehicle position (see [0069] image-based position not utilized when deriving final, while [0064] IMU is used (see also applicant’s specification, initializing including adjusting or replacing)); or based on a determination that the second estimated vehicle position is inaccurate, initializing the second estimated vehicle position with the first estimated vehicle position.
Examiner's note: since the claim uses the conjunction "or," only one of the recited alternatives is necessary in the prior art to read on this claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul Allen whose telephone number is (571) 272-4383. The examiner can normally be reached Monday - Friday from 9am to 5pm, Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at 571-270-7429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.A./Examiner, Art Unit 3669
/Erin M Piateski/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights