DETAILED ACTION
This is the first office action on the merits. Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 6/29/2023 and 7/31/2023 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 5, 7-8, 10, 12, and 17-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nehmadi et al., US 20160292905 A1 (“Nehmadi”).
Regarding claim 1, Nehmadi discloses a method comprising: receiving, from a camera (Fig. 2, passive sensor 230, Paragraph [0026]), a first image captured at a first location of an environment (Fig. 4, S410, Paragraph [0052]); receiving, by a three-dimensional (3D) coordinate measurement device (Fig. 2, active sensor 220, [0027]), first 3D coordinate data captured at the first location of the environment (Fig. 4, S460, Paragraph [0061]-[0062]); receiving, from the camera, a second image captured at a second location of the environment (Fig. 7, S720, Paragraph [0079]-[0080]; See also Paragraph [0071]-[0072]); detecting, by a processing system, first features of the first image and second features of the second image (Fig. 7, S710, Paragraph [0079]); determining, by the processing system, whether a correspondence exists between the first image and the second image (Fig. 7, S720, Paragraph [0080]); and responsive to determining that the correspondence exists between the first image and the second image, causing the 3D coordinate measurement device to capture, at the second location, second 3D coordinate data (Fig. 7, S740-S750, Paragraph [0082], Fig. 4, S460, Paragraph [0061]).
Regarding claim 3, Nehmadi discloses the method of claim 1, wherein the 3D coordinate measurement device is a laser scanner (Fig. 2, active sensor 220, [0027]).
Regarding claim 5, Nehmadi discloses the method of claim 1, wherein the second 3D coordinate data is captured subsequent to capturing the second image (Fig. 4, S460, Paragraph [0061]).
Regarding claim 7, Nehmadi discloses the method of claim 1, further comprising, subsequent to receiving the first 3D coordinate data and the second 3D coordinate data, performing registration on the first 3D coordinate data and the second 3D coordinate data (Fig. 4, S470, Paragraph [0066]).
Regarding claim 8, Nehmadi discloses the method of claim 1, wherein the 3D coordinate measurement device comprises the camera (Fig. 2, passive sensor 230, Paragraph [0026]).
Regarding claim 10, Nehmadi discloses the method of claim 1, wherein the first image and the second image are two-dimensional (2D) images (Fig. 4, S410, Paragraph [0052]).
Regarding claim 12, Nehmadi discloses the method of claim 1, wherein the camera and the 3D coordinate measurement device are disposed in a body (Fig. 2, apparatus 200, active sensor 220, passive sensor 230, Paragraph [0026]-[0027]).
Regarding claim 17, Nehmadi discloses the method of claim 1, wherein determining the correspondence is based at least in part on an angular difference between the first image and the second image (Paragraph [0056], Fig. 7, S720, Paragraph [0080]).
Regarding claim 18, Nehmadi discloses a method comprising: capturing a first image at first location of an environment (Fig. 4, S410, Paragraph [0052]); capturing a second image at second location of the environment (Fig. 7, S720, Paragraph [0079]-[0080]; See also Paragraph [0071]-[0072]); detecting, by a processing system, features of the first image and the second image (Fig. 7, S710, Paragraph [0079]); determining, by the processing system, whether a correspondence exists between the first image and the second image based at least in part on the features (Fig. 7, S720, Paragraph [0080]); and responsive to determining that the correspondence exists between the first image and the second image, capturing, at the first location, first three-dimensional (3D) coordinate data by a 3D coordinate measurement device and capturing, at the second location, second 3D coordinate data by the 3D coordinate measurement device (Fig. 7, S740-S750, Paragraph [0082], Fig. 4, S460, Paragraph [0061]).
Regarding claim 19, Nehmadi discloses the method of claim 18, wherein the first image and the second image are captured by a camera physically connected to the 3D coordinate measurement device (Fig. 2, apparatus 200, active sensor 220, passive sensor 230, Paragraph [0026]-[0027]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 4, 6, 9, 11, 13-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Nehmadi in view of Zweigle et al., US 20180321382 A1 (“Zweigle”).
Regarding claim 2, Nehmadi discloses the method of claim 1, further comprising: […]; detecting, by the processing system, third features of the third image (Fig. 7, S710, Paragraph [0079]); determining, by the processing system, whether a correspondence exists between the first image and the third image (Fig. 7, S720, Paragraph [0080]); and responsive to determining that the correspondence exists between the first image and the third image, causing the 3D coordinate measurement device to capture, at the third location, third 3D coordinate data (Fig. 7, S740-S750, Paragraph [0082], Fig. 4, S460, Paragraph [0061]).
Nehmadi does not teach: responsive to determining that the correspondence does not exist between the first image and the second image: receiving, from the camera, a third image captured at third location of the environment.
However, Zweigle teaches a laser scanner that takes a 2D scan at two registration points. If the two 2D scan data are not sufficient to determine a translation value, then the scanner is moved to an intermediate location (Fig. 11, first registration position 1112, second registration position 1114, intermediate scan locations 1120, Paragraph [0071].
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for determining if a 3D scan should be performed by adding a step to determine if the scan has sufficient correspondence, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to accurately determine translation and rotation values of the scanner positions, as suggested by Zweigle (Paragraph [0071]).
Regarding claim 4, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein the first 3D coordinate data is captured prior to capturing the first image.
However, Zweigle teaches a method for registering 3D coordinates where the 3D coordinate data determined before the 2D scan is taken (Fig. 15, element 1510 – 1515, Paragraph [0078]-[0079]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for determining if a 3D scan should be performed by determining 3D coordinate data before a 2D scan is performed, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to improve accuracy through multiple successive 2D scans, as suggested by Zweigle (Paragraph [0071]).
Regarding claim 6, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein the first 3D coordinate data is captured prior to capturing the first image and wherein the second 3D coordinate data is captured subsequent to capturing the second image.
However, Zweigle teaches a method for registering 3D coordinates where the 3D coordinate data determined before the first and second 2D scan is taken. After the 2D scans are performed, a second set of 3D coordinates is determined (Fig. 15, element 1510, 1515, 1525, Paragraph [0078]-[0079], [0081]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for determining if a 3D scan should be performed by performing successive 2D scans between 3D scans, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to improve accuracy through multiple successive 2D scans, as suggested by Zweigle (Paragraph [0071]).
Regarding claim 9, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein the camera is a panoramic camera.
However, Zweigle teaches a camera taking a panoramic image (Fig. 5, 3D scanned image 600, Paragraph [0048]-[0049]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have substituted Nehmadi’s passive sensor with a camera that can take panoramic images, which is disclosed by Zweigle. One of ordinary skill in the art could have substituted one camera for another and the results would have been predictable.
Regarding claim 11, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein the first image and the second image are 3D images.
However, Zweigle teaches determining correspondence between targets in two 3D collections of points (Fig. 15, element 1535, Paragraph [0082]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for determining if a 3D scan should be performed by using 3D images to determine correspondence, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to eliminate time consuming post-processing, as suggested by Zweigle (Paragraph [0085]).
Regarding claim 13, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein the camera is disposed in a first body and wherein the 3D coordinate measurement device is disposed in a second body separate from the first body.
However, Zweigle teaches a 2D scanner accessory that is mounted on the body of the 3D scanner (Fig. 8, 2D scanner accessory 810, 3D scanner 20, Paragraph [0057]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s apparatus by separating the bodies of the active sensor and passive sensor, which is disclosed by Zweigle. One of ordinary skill in the art could have implemented the known technique for housing both a 2D and 3D sensor, and the results would have been predictable.
Regarding claim 14, Nehmadi, as modified in view of Zweigle discloses the method of claim 13, wherein the first body is connected to the second body (Zweigle, Fig. 8, 2D scanner accessory 810, 3D scanner 20, Paragraph [0057]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s apparatus by separating the bodies of the active sensor and passive sensor, which is disclosed by Zweigle. One of ordinary skill in the art could have implemented the known technique for housing both a 2D and 3D sensor, and the results would have been predictable.
Regarding claim 15, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein determining the correspondence is based at least in part on a threshold number of detected features.
However, Zweigle teaches a quality threshold for determining a correspondence between scans. The correspondence through targets in the scan data (Paragraph [0093]; See also: Paragraph [0090]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for combining 3D scans by using a quality threshold for determining correspondence between objects, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to improve accuracy through multiple successive 2D scans, as suggested by Zweigle (Paragraph [0071]).
Regarding claim 16, Nehmadi discloses the method of claim 1.
Nehmadi does not teach: wherein determining the correspondence is based at least in part on a percentage of overlap between the first image and the second image.
However, Zweigle teaches a quality threshold for determining a correspondence between scans. The correspondence through targets in the scan data (Paragraph [0093]; See also: Paragraph [0090]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for combining 3D scans by using a quality threshold for determining correspondence between objects, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to improve accuracy through multiple successive 2D scans, as suggested by Zweigle (Paragraph [0071]).
Regarding claim 20, Nehmadi discloses the method of claim 18, further comprising: […]; detecting, by the processing system, third features of the third image (Fig. 7, S710, Paragraph [0079]); determining, by the processing system, whether a correspondence exists between the first image and the third image (Fig. 7, S720, Paragraph [0080]); and responsive to determining that the correspondence exists between the first image and the third image, capturing, at the first location, the first 3D coordinate data by the 3D coordinate measurement device and capturing, at the third location, third 3D coordinate data by the 3D coordinate measurement device (Fig. 7, S740-S750, Paragraph [0082], Fig. 4, S460, Paragraph [0061]).
Nehmadi does not teach: responsive to determining that the correspondence does not exist between the first image and the second image: capturing, at third location of the environment, a third image of the environment.
However, Zweigle teaches a laser scanner that takes a 2D scan at two registration points. If the two 2D scan data are not sufficient to determine a translation value, then the scanner is moved to an intermediate location (Fig. 11, first registration position 1112, second registration position 1114, intermediate scan locations 1120, Paragraph [0071].
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s method for determining if a 3D scan should be performed by adding a step to determine if the scan has sufficient correspondence, which is disclosed by Zweigle. One of ordinary skill in the art would have been motivated to make this modification in order to accurately determine translation and rotation values of the scanner positions, as suggested by Zweigle (Paragraph [0071]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rosengaus et al., US 20130096873 A1 discloses a scanner that is moved from one location to another in a construction site and takes panoramic images. The images are then used to update the site model.
Torok et al., US 20200249017 A1 teaches a surveying apparatus that is moved between locations and takes images. Features in each image are used to determine the camera position.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL N NGUYEN whose telephone number is (571)270-5405. The examiner can normally be reached Monday - Friday 8 am - 5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yuqing Xiao can be reached at (571) 270-3603. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL NGUYEN/Examiner, Art Unit 3645
/YUQING XIAO/Supervisory Patent Examiner, Art Unit 3645