DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 1-20 are currently subject to non-statutory double patent rejections , but are otherwise not subject to any prior art rejections under either 35 U.S.C. § 102 or 35 U.S.C. § 103. Assuming that the foregoing shortcomings of these claims were rectified by the timely filing of a terminal disclaimer, these claims would be allowable.
The following is a statement of reasons for the indication of allowable subject matter:
Independent claims 1, 19 and 20, recite the same patentable features as were found allowable in parent application no. 18/241,113 which issued as United States Patent No. 12,181,888 on 31 December 2024.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 1 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 1
U.S. Patent No. 12,181,888
Claim 1
A method of guiding a mobile unit including a visual sensor having at least one camera of a first type light sensitivity and at least one auxiliary sensor, including one or more selected from (a multi-axis IMU, a wheel odometry sensor, a depth sensor, and a camera of a second type light sensitivity), the method including:
A method of guiding a mobile unit including a visual sensor having at least one operational camera and at least one auxiliary sensor, including one or more selected from a multi-axis IMU, a wheel odometry sensor and a depth sensor, the method including:
obtaining a location of the mobile unit and perspective, including view direction, of the at least one camera, referred to collectively as an initial pose;
obtaining a location of the mobile unit and perspective, including view direction, of a camera, referred to collectively as an initial pose;
obtaining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit;
obtaining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit;
obtaining a propagated pose from the initial pose using information indicating travel P l;
obtaining a propagated pose from the initial pose using information indicating travel Pl;
obtaining a corrected pose Pm using one or more features extracted from captured image frames and matched to retrieved feature points and computing at least a magnitude of translational portion of positions of matching features to determine a corrected pose; and
obtaining a corrected pose, the corrected pose being obtained from a captured image frame by: extracting features from the captured image frame; …; and calculating a corrected pose Pm using the scale factor applied to at least a magnitude of translational portion of positions of matching features to determine a corrected pose; and
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
Claim 2 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 2 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 2
U.S. Patent No. 12,181,888
Claim 2
The method of claim 1, wherein using one or more features extracted and matched to retrieved feature points includes: (1) reuse of matched features from a previous frame and (2) matching of features in a new frame with reprojected feature positions from a 3D map onto a 2D view from a perspective of the propagated pose.
The method of claim 1, wherein matching features extracted to retrieved feature points is based on (1) reuse of matched features from a previous frame and (2)matching of features in the captured image frame with reprojected feature positions from a 3D map onto a 2D view from a perspective of the propagated pose.
Claim 3 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 10 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 3
U.S. Patent No. 12,181,888
Claim 10
The method of claim 1, wherein a translational portion of positions of matching features
describes a difference in perspective from which a new frame was viewed by the camera relative
to a previous frame viewed by the camera.
The method of claim 1, wherein a translational portion of positions of matching features describes a difference in perspective from which the captured image frame was viewed by the camera relative to a previous frame viewed by the camera.
Claim 4 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 11 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 4
U.S. Patent No. 12,181,888
Claim 11
The method of claim 1, further including detecting that at least one camera is experiencing impaired visibility, and whenever at least one camera experiences impaired visibility, using information from an auxiliary sensor to determine the corrected pose whenever image information is unusable.
The method of claim 1, further including detecting that at least one camera is experiencing impaired visibility, and whenever at least one camera experiences impaired visibility, using information from an auxiliary sensor to determine the corrected pose whenever image information is unusable.
Claim 5 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 12 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 5
U.S. Patent No. 12,181,888
Claim 12
The method of claim 4, the method further including detecting that at least one camera is experiencing impaired visibility, and whenever at least one camera experiences impaired visibility, using information from the auxiliary sensor and previous images to determine the corrected pose.
The method of claim 11, the method further including detecting that at least one camera is experiencing impaired visibility, and whenever at least one camera experiences impaired visibility, using information from the auxiliary sensor and previous images to determine the corrected pose.
Claim 6 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 13 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 6
U.S. Patent No. 12,181,888
Claim 13
The method of claim 5, wherein impaired visibility includes one of: diminished light in a spectral area used by the camera, camera being obstructed.
The method of claim 12, wherein impaired visibility includes one of: diminished light in a spectral area used by the camera, camera being obstructed.
Claim 7 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 14 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 7
U.S. Patent No. 12,181,888
Claim 14
The method of claim 4, wherein the auxiliary sensor includes at least one of an inertial measurement unit (IMU) and a wheel odometry sensor, the method further including performing at least one of:
The method of claim 11, wherein the auxiliary sensor includes at least one of an inertial measurement unit (IMU) and a wheel odometry sensor, the method further including performing at least one of:
determining information indicating travel to Pl including a rotational portion and a translational portion of movement of the mobile unit from IMU information by integrating at least one of linear acceleration and angular velocity; and
determining information indicating travel to Pl including a rotational portion and a translational portion of movement of the mobile unit from IMU information by integrating at least one of linear acceleration and angular velocity; and
determining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit from odometry information of a rotary encoder coupled to a wheel, from a ratio of a circumference of the wheel to resolution of the rotary encoder.
determining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit from odometry information of a rotary encoder coupled to a wheel, from a ratio of a circumference of the wheel to resolution of the rotary encoder.
Claim 8 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 15 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 8
U.S. Patent No. 12,181,888
Claim 15
The method of claim 1, further including: examining a new frame using a criteria that includes: whether a count of 2D feature points of usable quality meets a threshold; whether a count of triangulated map points between a pairing of a propagated pose with a new image and a pairing of a current pose with an existing image meets a threshold; and including the new frame that meets the threshold.
The method of claim 1, further including: examining a new frame using a criteria that includes: whether a count of 2D feature points of usable quality meets a threshold; whether a count of triangulated map points between a pairing of a propagated pose with a new image and a pairing of a current pose with an existing image meets a threshold; and including the new frame that meets the threshold.
Claim 9 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 16 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 9
U.S. Patent No. 12,181,888
Claim 16
The method of claim 1, wherein the mobile unit including at least two auxiliary sensors, including a first auxiliary sensor comprising a multi-axis IMU and a second auxiliary sensor selected from (a wheel odometry sensor and a depth sensor), the method further including:
The method of claim 1, wherein the mobile unit including at least two auxiliary sensors, including a first auxiliary sensor comprising a multi-axis IMU and a second auxiliary sensor selected from (a wheel odometry sensor and a depth sensor), the method further including:
while waiting for a new frame, between successive camera frames, updating the initial pose using information from an IMU, to generate a first propagated pose;
while waiting for a new frame, between successive camera frames, updating the initial pose using information from an IMU, to generate a first propagated pose;
extracting new features from the new frame;
extracting new features from the new frame;
determining a second propagated pose, updating the initial pose using information from a wheel odometer;
determining a second propagated pose, updating the initial pose using information from a wheel odometer;
determining an error between the first propagated pose and the second propagated pose;
determining an error between the first propagated pose and the second propagated pose;
minimizing error of a translational portion of first and the second propagated poses and a rotational portion of first and the second propagated poses to find the corrected pose; and
minimizing error of a translational portion of first and the second propagated poses and a rotational portion of first and the second propagated poses to find the corrected pose; and
using the corrected pose for guiding a robot.
using the corrected pose for guiding a robot.
Claim 10 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 17 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 10
U.S. Patent No. 12,181,888
Claim 17
The method of claim 9, wherein the minimizing includes:
The method of claim 16, wherein the minimizing includes:
receiving an initial translational portion and rotational portion of first and second propagated poses having a first error;
receiving an initial translational portion and rotational portion of first and second propagated poses having a first error;
determining a new translational portion and rotational portion for first and second propagated poses having a second error;
determining a new translational portion and rotational portion for first and second propagated poses having a second error;
selecting one from the new translational portion and rotational portion and the initial translational portion and rotational portion based upon comparing the second error with the first error as a new initial translational portion and rotational portion of first and second propagated poses; and
selecting one from the new translational portion and rotational portion and the initial translational portion and rotational portion based upon comparing the second error with the first error as a new initial translational portion and rotational portion of first and second propagated poses; and
iteratively repeating the determining and selecting until a translational portion and rotational portion for first and second propagated poses achieves a minimum error criterion.
iteratively repeating the determining and selecting until a translational portion and rotational portion for first and second propagated poses achieves a minimum error criterion.
Claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 3 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 11
U.S. Patent No. 12,181,888
Claim 3
The method of claim 2, the visual sensor having at least one operational camera and a depth sensor, the method further including:
The method of claim 2, the visual sensor having at least one operational camera and a depth sensor, the method further including:
receiving depth information from a depth sensor disposed at a position to sense depth information for features within a field of view of a camera;
receiving depth information from a depth sensor disposed at a position to sense depth information for features within a field of view of a camera;
using the depth information, retrieving corresponding depth values for remaining two dimensional features;
using the depth information, retrieving corresponding depth values for remaining two dimensional features;
obtaining weighted average depth values for each remaining two-dimensional feature;
obtaining weighted average depth values for each remaining two-dimensional feature;
populating the 3D map with remaining two-dimensional features using the weighted average depth values; and
populating the 3D map with remaining two-dimensional features using the weighted average depth values; and
providing the 3D map as populated for guiding a mobile unit.
providing the 3D map as populated for guiding a mobile unit.
Claim 12 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 12
U.S. Patent No. 12,181,888
Claim 4
The method of claim 11, further including: tracking the mobile unit against the 3D map.
The method of claim 3, further including: tracking the mobile unit against the 3D map.
Claim 13 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 5 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 13
U.S. Patent No. 12,181,888
Claim 5
The method of claim 11, further including: adding additional poses by performing imu propagation to obtain a baseline between a latest to-be-added pose and a properly chosen previous pose; and using depth information from the depth sensor to correct depth information for triangulated map points.
The method of claim 3, further including: adding additional poses by performing imu propagation to obtain a baseline between a latest to-be-added pose and a properly chosen previous pose; and using depth information from the depth sensor to correct depth information for triangulated map points.
Claim 14 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 6 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 14
U.S. Patent No. 12,181,888
Claim 6
The method of claim 13, further including: for 2D feature points which are not triangulated, using the depth information to determine additional 3D map points.
The method of claim 5, further including: for 2D feature points which are not triangulated, using the depth information to determine additional 3D map points.
Claim 15 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 7 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 15
U.S. Patent No. 12,181,888
Claim 7
The method of claim 11, further including: removing from images two-dimensional features
having no corresponding readings from the depth sensor.
The method of claim 3, further including: removing from images two-dimensional features having no corresponding readings from the depth sensor.
Claim 16 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 8 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 16
U.S. Patent No. 12,181,888
Claim 8
The method of claim 11, further including: populating a table of depth values to pixel
coordinates to calibrate the depth sensor to the camera.
The method of claim 3, further including: populating a table of depth values to pixel coordinates to calibrate the depth sensor to the camera.
Claim 17 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 18 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 17
U.S. Patent No. 12,181,888
Claim 18
The method of claim 1, wherein obtaining a location of the mobile unit and perspective, including view direction, of a camera, further includes: receiving via an interface information from at least one selected from: (i) an inertial measurement unit (IMU) encoding information from at least one of a gyroscopic sensor, and an accelerometer; (ii) a sensor encoding odometry information, and (iii) a sensor encoding depth information for at least some pixels sensed by a camera.
The method of claim 1, wherein obtaining a location of the mobile unit and perspective, including view direction, of a camera, further includes: receiving via an interface information from at least one selected from: (i) an inertial measurement unit(IMU) encoding information from at least one of a gyroscopic sensor, and an accelerometer; (ii) a sensor encoding odometry information, and (iii) a sensor encoding depth information for at least some pixels sensed by a camera.
Claim 18 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 9 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 18
U.S. Patent No. 12,181,888
Claim 9
The method of claim 2, further including guiding the mobile unit as the mobile unit explores
while updating the 3D map with features.
The method of claim 2, further including guiding the mobile unit as the mobile unit explores while updating the 3D map with features.
Claim 19 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 19 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 19
U.S. Patent No. 12,181,888
Claim 19
A system including a processor and a computer readable storage unit storing instructions for
guiding a mobile unit including a visual sensor having at least one camera of a first type light
sensitivity and at least one auxiliary sensor, including one or more selected from (a multi-axis
IMU, a wheel odometry sensor, a depth sensor, and a camera of a second type light sensitivity),
which instructions, when executed by the processor perform:
A system including a processor and a computer readable storage unit storing instructions for guiding a mobile unit including a visual sensor having at least one operational camera and at least one auxiliary sensor, including one or more selected from a multi-axis IMU, a wheel odometry sensor and a depth sensor, which instructions, when executed by the processor perform:
obtaining a location of the mobile unit and perspective, including view direction, of the
at least one camera, referred to collectively as an initial pose;
obtaining a location of the mobile unit and perspective, including view direction, of a camera, referred to collectively as an initial pose;
obtaining information indicating travel Pl including a rotational portion and a
translational portion of movement of the mobile unit;
obtaining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit;
obtaining a propagated pose from the initial pose using information indicating travel Pl;
obtaining a propagated pose from the initial pose using information indicating travel Pl;
obtaining a corrected pose Pm using one or more features extracted from captured image
frames and matched to retrieved feature points and computing at least a magnitude of
translational portion of positions of matching features to determine a corrected pose; and
obtaining a corrected pose, the corrected pose being obtained from a captured image frame by: extracting features from the captured image frame; …; and calculating a corrected pose Pm using the scale factor applied to at least a magnitude of translational portion of positions of matching features to determine a corrected pose; and
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
Claim 20 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 20 of U.S. Patent No. 12,181,888. Although the claims at issue are not identical, they are not patentably distinct from each other as shown in the following table:
Present Application
Claim 20
U.S. Patent No. 12,181,888
Claim 20
A non-transitory computer readable medium storing instructions for guiding a mobile unit
including a visual sensor having at least one camera of a first type light sensitivity and at least one auxiliary sensor, including one or more selected from (a multi-axis IMU, a wheel odometry sensor, a depth sensor, and a camera of a second type light sensitivity), which instructions, when executed by a processor perform:
A non-transitory computer readable medium storing instructions for guiding a mobile unit including a visual sensor having at least one operational camera and at least one auxiliary sensor, including one or more selected from a multi-axis IMU, awheel odometry sensor and a depth sensor, which instructions, when executed by a processor perform:
obtaining a location of the mobile unit and perspective, including view direction, of a camera, referred to collectively as an initial pose;
obtaining a location of the mobile unit and perspective, including view direction, of a camera, referred to collectively as an initial pose;
obtaining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit;
obtaining information indicating travel Pl including a rotational portion and a translational portion of movement of the mobile unit;
obtaining a propagated pose from the initial pose using information indicating travel Pl;
obtaining a propagated pose from the initial pose using information indicating travel Pl;
obtaining a corrected pose Pm using one or more features extracted from captured image frames and matched to retrieved feature points and computing at least a magnitude of translational portion of positions of matching features to determine a corrected pose; and
obtaining a corrected pose, the corrected pose being obtained from a captured image frame by: extracting features from the captured image frame; …; and calculating a corrected pose Pm using the scale factor applied to at least a magnitude of the translational portion of positions of matching features to determine a corrected pose; and
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
guiding the mobile unit using one or both of the propagated pose and the corrected pose.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID F DUNPHY whose telephone number is (571)270-1230. The examiner can normally be reached 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached on 5712727332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID F DUNPHY/Primary Examiner, Art Unit 2673
Prosecution Insights