DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim(s) 21-39 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-26 of U.S. Patent No.12,254,568 B2 either directly or in view of Barker et al. (US 2018/0089852 A1) in view of Se et al. (US 2006/0221072 A1). Although the claims at issue are not identical, they are not patentably distinct from each other because the claim of the instant application is either anticipated by, or the obvious variation of, some examples of the claims of U.S. Patent No. 12,254,568 B2, as shown in the table below.
Instant Application:
US 12,254,568 B2:
Claim 21. A system, comprising:
a range sensor to acquire depth data; and
an image sensor to acquire imaging data; and
a control circuit communicably couplable with the range sensor and the image sensor, the control circuit configured to:
generate a digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on the depth data;
identify, in the imaging data, a survey control point in the subterranean, GPS-denied environment captured by the image sensor, wherein the survey control point comprises a visually-identifiable mark, and wherein the survey control point has been associated with a coordinate location within a subterranean coordinate system;
generate a 3D digital anchor within the digital 3D model corresponding to the survey control point; and
apply one or more non-rigid transformations to the digital 3D model of the subterranean, GPS-denied environment to register the digital 3D model to the subterranean coordinate system based on the coordinate location of the survey control point.
Claim 1. A system for georeferencing three-dimensional (3D) geometric data, the system comprising:
an apparatus couplable to a mobile platform, the apparatus comprising:
a processing circuit;
a range sensor to acquire depth data of a subterranean, global positioning system (GPS)-denied environment, the range sensor communicably couplable with the processing circuit; and
a camera to acquire imaging data of the subterranean, GPS-denied environment, the camera communicably couplable with the processing circuit; and
a computing system communicably couplable with the apparatus, the computing system comprising:
a second processing circuit;
a 3D generator module communicably couplable with the second processing circuit and configured to generate a digital 3D model of the subterranean, GPS-denied environment based on the depth data;
a 3D survey control generator module communicably couplable with the second processing circuit and configured to:
identify, in the imaging data, a survey control point, wherein the survey control point comprises a visually-identifiable mark in the subterranean, GPS-denied environment captured by the camera, and wherein the survey control point has been associated with a coordinate location within a subterranean coordinate system; and
generate a 3D digital anchor within the digital 3D model of the subterranean, GPS-denied environment, wherein the 3D digital anchor corresponds to the identified survey control point; and
a georeferencing module communicably couplable with the second processing circuit and configured to apply one or more non-rigid transformations to the digital 3D model of the subterranean, GPS-denied environment to register the digital 3D model generated based on the depth data acquired by the range sensor to the subterranean coordinate system based on the coordinate location of the survey control point identified in the imaging data of the subterranean, GPS-denied environment captured by the camera.
Claim 28. The system of claim 21, wherein the digital 3D model comprises a model of a subterranean mine.
Claim 7. The system of claim 1, wherein the subterranean, GPS-denied environment comprises one of the following: a mine; a tunnel; a cave; a bunker; and a conduit.
Claim 29. The system of claim 28, wherein the model of the subterranean mine comprises 3D representations of identified geology along the subterranean mine.
Claim 1
Claim 30. The system of claim 28, wherein the subterranean coordinate system comprises a real-world coordinate system, and wherein the model of the subterranean mine is spatially located relative to locations on a GPS-accessible surface above the subterranean mine.
Claim 13. The system of claim 1, wherein the georeferencing module is to: spatially relocate the depth data to the subterranean coordinate system; and spatially relocate the imaging data to the subterranean coordinate system.
Claim 14. The system of claim 13, wherein the subterranean coordinate system comprises a real-world, geographic coordinate system.
Claim 31. A system, comprising:
a range sensor;
an image sensor; and
a control circuit communicably couplable with the range sensor and the image sensor, the control circuit configured to:
generate a first digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on first depth data acquired by the range sensor, wherein the first digital 3D model comprises a first face;
generate a second digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on the first digital 3D model and second depth data acquired by the range sensor, wherein the second digital 3D model comprises a second face spaced apart from the first face and a volumetric model between the first face and the second face;
identify a survey control point in the subterranean, GPS-denied environment captured by the image sensor, wherein the survey control point comprises a visually-identifiable mark, and wherein the survey control point has been associated with a coordinate location within a coordinate system;
generate a 3D digital anchor within the second digital 3D model corresponding to the survey control point; and
register the second digital 3D model of the subterranean, GPS-denied environment to the coordinate system based on the coordinate location of the survey control point.
Claim 23. A method for georeferencing three-dimensional (3D) geometric data, the method comprising:
generating a digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on a range data acquired by a ranging sensor;
identifying geology in an image data acquired by an image sensor;
generating 3D geometric data representative of the identified geology;
identifying a survey control point in the image data, wherein the survey control point comprises a visually-identifiable mark in the subterranean, GPS-denied environment;
generating a 3D digital anchor corresponding to the identified survey control point; and
applying one or more non-rigid transformations to the digital 3D model to georeference the 3D geometric data to a coordinate system of the subterranean, GPS-denied environment based on the coordinate location of the survey control point identified in the image data.
Claim 26. The method of claim 23, further comprising:
acquiring additional range data and image data associated with the subterranean, GPS-denied environment;
positioning the acquired additional range data and image data;
generating a second digital 3D model of the subterranean, GPS-denied environment associated with the acquired additional range data and image data; and
aligning the generated second digital 3D model of the subterranean, GPS-denied environment associated with the acquired additional range data and image data with the generated digital 3D model of the subterranean, GPS-denied environment to create an updated digital 3D model of the subterranean, GPS-denied environment.
Notes since the instant application and the above prior arts are in the same field of 3D model of a subterranean, a PHOSITA would have been motivated to incorporate the teachings of above priors to add the limitation in order to enhance the 3D model generating function.
Claim Objections
Applicant is advised that should claim 30 be found allowable, claim 33 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 21-30 and 33 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barker et al. (US 2018/0089852 A1).
Regarding Claim 21, Barker discloses a system, comprising:
a range sensor to acquire depth data (Fig.3: Range sensor 18. Barker does not explicitly recite the range sensor is used to acquire depth data. However Barker discloses the range sensor 18 includes a laser rangefinder such as a light detection and ranging (LIDAR) module, a radar module, an ultrasonic ranging module, a sonar module, a ranging module using triangulation or any other device able to acquire data representative of the geometry of the mine. The laser rangefinder emits a physical signal and receives a reflected physical signal. The emitted signals and the reflected signals can be, for example, light beams, electromagnetic waves, acoustic waves or the like. According to various aspects, by scanning the mine in a two-dimensional plane X, Y (where X is a first direction, Y is a second direction which is not collinear with X, and the directions X and Y form coordinate axes) while rotating around a third direction Z which is perpendicular to the first and the second directions, the laser rangefinder can acquire data which is representative of the three-dimensional geometry of the mine, see [0040]. Therefore it would have been obvious to a PHOSITA that the range sensor is to be used to acquire depth data since the LIDAR, radar etc. had been widely used before the effective filing date of the claimed invention to acquire depth data based on time of flight (ToF) which is the time between the laser rangefinder emits a physical signal and receives a reflected physical signal as described in [0040]); and
an image sensor to acquire imaging data (Fig.3: Camera 20); and
a control circuit communicably couplable with the range sensor and the image sensor (Fig.3: processor 22), the control circuit configured to:
generate a digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on the depth data ([0047]: In various aspects, the 3D generator module 30 is configured to utilize the acquired range sensor data and/or the acquired camera data to generate an electronic/digital three-dimensional model of the mine. According to various aspects, this electronic/digital three-dimensional model of the mine is generated in real-time, in near real-time, or at some point well after the acquisition of the range sensor date and/or camera data. Although Barker does not explicitly use the phrase GPS-denied environment, a skilled person would have recognized that an underground mine is a GPS-denied environment);
identify, in the imaging data, a survey control point in the subterranean, GPS-denied environment captured by the image sensor, wherein the survey control point comprises a visually-identifiable mark, and wherein the survey control point has been associated with a coordinate location within a subterranean coordinate system; generate a 3D digital anchor within the digital 3D model corresponding to the survey control point; and apply one or more non-rigid transformations to the digital 3D model of the subterranean, GPS-denied environment to register the digital 3D model to the subterranean coordinate system based on the coordinate location of the survey control point ([0049]: The electronic/digital anchors are feature vectors, and each electronic/digital anchor uniquely describes an oriented point within one or more of the electronic/digital three-dimensional models of the mine … According to various aspects, the one or more unique features may include, for example, image features, geometric features, topological features, symbolic features, or any combination of these features). [0050]: The electronic/digital anchors may be defined in any number of different ways. For example, according to various aspects, a given anchor point (an oriented X, Y, Z point) can be imported into one or more of the electronic three-dimensional models of the mine as the surveyed position of a real reference point in the mine (e.g. a distinctive outcrop of a sidewall of the mine). According to other aspects, a user can utilize the electronic anchor module 32 to create an electronic/digital anchor and insert the created electronic/digital anchor onto a three-dimensional surface of the mine displayed on the display device 38. According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine. [0081]: The system of Examples 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or 11, wherein at least one of the plurality of digital anchors comprises an oriented X, Y, Z point in a three-dimensional coordinate system. The anchor point is interpreted as the survey control point).
Regarding Claim 22, Barker teaches or suggests wherein the survey control point is automatically identified by the control circuit ([0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine. [0049]: each electronic/digital anchor is described uniquely in space and time (it can be tracked as the oriented X, Y, Z point moves through space and over time) with sufficient resolution as to allow it to be identified across multiple scans of the underground mine obtained over a sufficient period of time to monitor and measure ground movement).
Regarding Claim 23, Barker teaches or suggests wherein the survey control point is automatically recognized by the control circuit without manual designation of the survey control point ([0049]: each electronic/digital anchor is described uniquely in space and time (it can be tracked as the oriented X, Y, Z point moves through space and over time) with sufficient resolution as to allow it to be identified across multiple scans of the underground mine obtained over a sufficient period of time to monitor and measure ground movement. [0052]: The movement determination module 34 is configured to automatically determine an occurrence of movement within the mine based on the positions (location and orientation) of the electronic/digital anchors over time relative to the electronic/digital three-dimensional models of the mine. Barker indirectly discloses the anchors which are the survey control points are recognized automatically by the computer system 14 which includes anchor generator module and movement determination module as shown in Fig.4).
Regarding Claim 24, Barker further teaches or suggests wherein the control circuit is further configured to: identify a geological feature in the imaging data ([0049]: The electronic/digital anchors are feature. … According to various aspects, the one or more unique features may include, for example, image features, geometric features, topological features, symbolic features, or any combination of these features ); define a region of interest surrounding the geological feature; and generate a 3D polygon in the digital 3D model based on the region of interest ([0066]: Thus, it will be appreciated that at least some of the groupings/combinations of the electronic/digital anchors associated with the electronic/digital three-dimensional models of the mine can be utilized to “couple” at least some of the electronic/digital anchors to geological features of the mine. [0067]: In operation, as the mobile platform 26 makes subsequent passes through the mine, the system 10 acquires subsequent range sensor data and/or images of the interior of the mine (camera data) and/or other data representative of the mine and generates subsequent electronic three-dimensional models of the mine based on the subsequently acquired data).
Regarding Claim 25, Barker teaches or suggests wherein the region of interest is dynamically determined by the control circuit without manual designation of the region of interest ([0041]: According to various aspects, the range sensor 18 includes an internal coordinate frame which allows for subsequent scans to be easily aligned with or registered against the original scan of a given area of the mine. Additionally, the position of each range sensor 18 can be dynamically adjusted relative to its internal coordinate frame in order to assist in the scanning of a given area of the mine. [0055]: According to various aspects, the movement determination module 34 is also configured to automatically determine distances and angles between different tunnels (e.g., between adjacent or distant tunnels) of a mine and/or automatically determine the size/geometry of any pillars in the mine).
Regarding Claim 26, Barker teaches or suggests wherein the region of interest comprises a 2D region in a plane perpendicular to a lens of the image sensor ([0042]: … according to various aspects, the apparatus 12 includes a single camera 20 which includes a single lens (e.g., a single lens camera) and is configured to capture a two-dimensional image of the interior of the mine. … According to yet other aspects, the apparatus 12 includes multiple cameras 20 which are utilized to capture two-dimensional images of the interior of the mine from different positions/angles/orientations, thereby allowing for the captured images to be subsequently utilized to reconstruct the geometry of the mine … Also see [0050]: The electronic/digital anchors may be defined in any number of different ways. For example, according to various aspects, a given anchor point (an oriented X, Y, Z point) can be imported into one or more of the electronic three-dimensional models of the mine as the surveyed position of a real reference point in the mine (e.g. a distinctive outcrop of a sidewall of the mine). According to other aspects, a user can utilize the electronic anchor module 32 to create an electronic/digital anchor and insert the created electronic/digital anchor onto a three-dimensional surface of the mine displayed on the display device 38. According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine).
Regarding Claim 27, Barker further teaches or suggests wherein the depth data comprises: first depth data corresponding to a first face in the subterranean, GPS-denied environment; and second depth data corresponding to a second face in the subterranean, GPS-denied environment, wherein the digital 3D model comprises a volumetric model of a subterranean mine between the first face and the second face ([0039]: Although only one range sensor 18, one camera 20, one processor 22 and one memory device 24 are shown in FIG. 3 for purposes of simplicity, it will be appreciated that the apparatus 12 may include any number of these components. [0041]: For aspects which include more than one range sensor 18, each range sensor 18 may include a unique range sensor identification which can be stored in the memory device 24 along with distance and/or angle measurements associated with the given range sensor 18. [0040]: Examples of types of 3D scans which can be secured by the range sensor 18 include point clouds, wireframes, spatially registered image sequences (either absolute or relative registration), and/or textured or textured meshes. A skilled person would have recognized that more than one range sensors are to capture depth of different interior images which are different faces in the subterranean which could be underground mines. In addition, a skilled person would have known that point cloud model is a volumetric model due to its 3D nature).
Regarding Claim 28, Barker discloses wherein the digital 3D model comprises a model of a subterranean mine ([0047]: In various aspects, the 3D generator module 30 is configured to utilize the acquired range sensor data and/or the acquired camera data to generate an electronic/digital three-dimensional model of the mine. [0070]: the apparatus comprises a sensor configured to acquire data representative of a geometry of the subterranean structure and/or a camera configured to capture images of an interior of the subterranean structure, and a processing circuit communicably couplable to the sensor and/or the camera).
Regarding Claim 29, Barker further teaches or suggests wherein the model of the subterranean mine comprises 3D representations of identified geology along the subterranean mine ([0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine).
Regarding Claim 30, Barker teaches or suggests wherein the subterranean coordinate system comprises a real-world coordinate system ([0049]: In contrast to extensometers like the one shown in FIG. 1, the electronic/digital anchors can be placed where actual reference points might be subject to mechanical damage, such as low on a wall, at corners, or at other places where walls can suffer mechanical damage. The electronic/digital anchors are feature vectors, and each electronic/digital anchor uniquely describes an oriented point within one or more of the electronic/digital three-dimensional models of the mine. Stated differently, each electronic/digital anchor has a physical orientation with respect to the electronic/digital three-dimensional models of the mine. A given electronic/digital anchor includes at least one X, Y, Z point in relative or absolute coordinates, an orientation relative to a reference (e.g., angle relative to “vertical”, “horizontal”, etc.) and may also include one or more unique features which are in close proximity to the X, Y, Z point and are sufficient to uniquely describe the electronic/digital anchor), and wherein the model of the subterranean mine is spatially located relative to locations on a GPS-accessible surface above the subterranean mine ([0071]: The system of Example 1, wherein the subterranean structure comprises a mine, a tunnel, a cave, a bunker or a conduit. A skilled person would have known that a tunnel or a cave is spatially located relative to locations on a GPS-accessible surface which is above the tunnel or the cave since the surface above the tunnel or the cave is exposed to open air therefore GPS accessible).
Regarding Claim 33, Barker teaches or suggests wherein the coordinate system comprises a real-world coordinate system ([0049]: In contrast to extensometers like the one shown in FIG. 1, the electronic/digital anchors can be placed where actual reference points might be subject to mechanical damage, such as low on a wall, at corners, or at other places where walls can suffer mechanical damage. The electronic/digital anchors are feature vectors, and each electronic/digital anchor uniquely describes an oriented point within one or more of the electronic/digital three-dimensional models of the mine. Stated differently, each electronic/digital anchor has a physical orientation with respect to the electronic/digital three-dimensional models of the mine. A given electronic/digital anchor includes at least one X, Y, Z point in relative or absolute coordinates, an orientation relative to a reference (e.g., angle relative to “vertical”, “horizontal”, etc.) and may also include one or more unique features which are in close proximity to the X, Y, Z point and are sufficient to uniquely describe the electronic/digital anchor), and wherein the model of the subterranean mine is spatially located relative to locations on a GPS-accessible surface above the subterranean mine ([0071]: The system of Example 1, wherein the subterranean structure comprises a mine, a tunnel, a cave, a bunker or a conduit. A skilled person would have known that a tunnel or a cave is spatially located relative to locations on a GPS-accessible surface which is above the tunnel or the cave since the surface above the tunnel or the cave is exposed to open air therefore GPS accessible).
Claims 31-32 and 34-39 are rejected under 35 U.S.C. 103 as being unpatentable over Barker et al. (US 2018/0089852 A1) in view of Se et al. (US 2006/0221072 A1).
Regarding Claim 31, Barker discloses a system, comprising:
a range sensor (Fig.3: Range sensor 18. [0040]: the range sensor 18 includes a laser rangefinder such as a light detection and ranging (LIDAR) module, a radar module, an ultrasonic ranging module, a sonar module, a ranging module using triangulation or any other device able to acquire data representative of the geometry of the mine);
an image sensor (Fig.3: Camera 20); and
a control circuit communicably couplable with the range sensor and the image sensor (Fig.3: processor 22), the control circuit configured to:
generate a first digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on first depth data acquired by the range sensor ([0047]: In various aspects, the 3D generator module 30 is configured to utilize the acquired range sensor data and/or the acquired camera data to generate an electronic/digital three-dimensional model of the mine. According to various aspects, this electronic/digital three-dimensional model of the mine is generated in real-time, in near real-time, or at some point well after the acquisition of the range sensor date and/or camera data. Although Barker does not explicitly use the phrase GPS-denied environment, a skilled person would have recognized that an underground mine is a GPS-denied environment. Note Barker does not explicitly recite the range sensor is used to acquire depth data. However Barker discloses The laser rangefinder emits a physical signal and receives a reflected physical signal… by scanning the mine in a two-dimensional plane X, Y (where X is a first direction, Y is a second direction which is not collinear with X, and the directions X and Y form coordinate axes) while rotating around a third direction Z which is perpendicular to the first and the second directions, the laser rangefinder can acquire data which is representative of the three-dimensional geometry of the mine, see [0040]. A skilled person would have recognized that the range sensor is to obtain the depth information by the received reflected signal), wherein the first digital 3D model comprises a first face ([0042]: The apparatus 12 may include any number of cameras 20. In general, the camera 20 or cameras 20 operate to capture images of various parts of the interior of the mine. Each camera 20 may be any suitable type of camera. For example, according to various aspects, the apparatus 12 includes a single camera 20 which includes a single lens (e.g., a single lens camera) and is configured to capture a two-dimensional image of the interior of the mine. According to other aspects, the apparatus 12 includes two or more lenses (e.g., a stereo camera) and is configured to capture a three-dimensional image of the interior of the mine. According to yet other aspects, the apparatus 12 includes multiple cameras 20 which are utilized to capture two-dimensional images of the interior of the mine from different positions/angles/orientations, thereby allowing for the captured images to be subsequently utilized to reconstruct the geometry of the mine. [0047]: According to various aspects, for each time the apparatus 12 passes through the mine (or a portion thereof) and performs a mobile “scan” with the range sensor 16 and/or the camera 20, the 3D generator module 30 can generate another electronic/digital three-dimensional model of the mine based on the acquired range sensor data and/or camera data. Also see [0067]: In operation, as the mobile platform 26 makes subsequent passes through the mine, the system 10 acquires subsequent range sensor data and/or images of the interior of the mine (camera data) and/or other data representative of the mine and generates subsequent electronic three-dimensional models of the mine based on the subsequently acquired data. Any interior image can be interpreted as a face);
generate a second digital 3D model of a subterranean, global positioning system (GPS)-denied environment based on the first digital 3D model and second depth data acquired by the range sensor, ([0067]: In operation, as the mobile platform 26 makes subsequent passes through the mine, the system 10 acquires subsequent range sensor data and/or images of the interior of the mine (camera data) and/or other data representative of the mine and generates subsequent electronic three-dimensional models of the mine based on the subsequently acquired data. [0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine. A skilled person would have recognized that more than one range sensors are to capture depth of different interior images which are different faces in the subterranean which could be underground mines);
identify a survey control point in the subterranean, GPS-denied environment captured by the image sensor, wherein the survey control point comprises a visually-identifiable mark, and wherein the survey control point has been associated with a coordinate location within a coordinate system ([0047]: According to various aspects, for each time the apparatus 12 passes through the mine (or a portion thereof) and performs a mobile “scan” with the range sensor 16 and/or the camera 20, the 3D generator module 30 can generate another electronic/digital three-dimensional model of the mine based on the acquired range sensor data and/or camera data. Also see [0049]-[0050] and [0081]. The anchor point is interpreted as the survey control point);
generate a 3D digital anchor within the second digital 3D model corresponding to the survey control point ([0048]: The electronic anchor module 32 is configured to define a plurality of electronic/digital anchors (shown as “+” signs in FIGS. 6-11) and electronically (1) insert the electronic/digital anchors into, (2) position the electronic/digital anchors within or (3) overlay the electronic/digital anchors onto any number of the electronic/digital three-dimensional models of the mine generated by the 3D generator module 30. [0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine); and
register the second digital 3D model of the subterranean, GPS-denied environment to the coordinate system based on the coordinate location of the survey control point ([0041]: According to various aspects, the range sensor 18 includes an internal coordinate frame which allows for subsequent scans to be easily aligned with or registered against the original scan of a given area of the mine).
Barker does not explicitly recite wherein the second digital 3D model comprises a second face spaced apart from the first face and a volumetric model between the first face and the second face.
However Se discloses a 3D imaging system provides a method and apparatus for generating photo-realistic 3D models of mines (which also includes underground passages and caverns or other sub-terrain cavities) and automatically registering the models with a previously provided mine map ([0116]) and The present invention is related to an apparatus and a method for creating three-dimensional (3D) computer models (represented, for example, as point clouds, surfaces, or volumes) of environments and/or objects from a plurality of images from stereo and monocular cameras ([0002]). Se further discloses This 3D model creation and localization procedure is carried out repeatedly as the mine advances and the photo-realistic 3D models for each mine face are registered together to allow geologists to assess and predict the ore distribution in the mine. The advancing mine faces are labeled as 87, 88 and 89 in FIG. 8 ([0124]).
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Se into that of Barker and to include the limitation of wherein the second digital 3D model comprises a second face spaced apart from the first face (as shown in Fig.8 below) and a volumetric model between the first face and the second face ([0002]: three-dimensional (3D) computer models (represented, for example, as point clouds, surfaces, or volumes) of environments and/or objects) in order to register to the mine map accurately themselves as suggested by Se ([0007]).
PNG
media_image1.png
454
759
media_image1.png
Greyscale
Regarding Claim 32, Barker discloses wherein the second digital 3D model comprises a model of a subterranean mine ([0047]: In various aspects, the 3D generator module 30 is configured to utilize the acquired range sensor data and/or the acquired camera data to generate an electronic/digital three-dimensional model of the mine. [0070]: the apparatus comprises a sensor configured to acquire data representative of a geometry of the subterranean structure and/or a camera configured to capture images of an interior of the subterranean structure, and a processing circuit communicably couplable to the sensor and/or the camera).
Regarding Claim 34, Barker further teaches or suggests wherein the model of the subterranean mine comprises 3D representations of identified geological features along the subterranean mine ([0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine. [0066]: it will be appreciated that at least some of the groupings/combinations of the electronic/digital anchors associated with the electronic/digital three-dimensional models of the mine can be utilized to “couple” at least some of the electronic/digital anchors to geological features of the mine).
Regarding Claim 35, Barker further teaches or suggests wherein the control circuit is further configured to: identify a first geological feature in imaging data acquired by the image sensor ([0049]: The electronic/digital anchors are feature. … According to various aspects, the one or more unique features may include, for example, image features, geometric features, topological features, symbolic features, or any combination of these features ); define a region of interest surrounding the first geological feature; generate at least one 3D polygon in the second digital 3D model based on the region of interest ([0066]: Thus, it will be appreciated that at least some of the groupings/combinations of the electronic/digital anchors associated with the electronic/digital three-dimensional models of the mine can be utilized to “couple” at least some of the electronic/digital anchors to geological features of the mine. [0067]: In operation, as the mobile platform 26 makes subsequent passes through the mine, the system 10 acquires subsequent range sensor data and/or images of the interior of the mine (camera data) and/or other data representative of the mine and generates subsequent electronic three-dimensional models of the mine based on the subsequently acquired data. [0040]: Examples of types of 3D scans which can be secured by the range sensor 18 include point clouds, wireframes, spatially registered image sequences (either absolute or relative registration), and/or textured or textured meshes. A wire frame model is a visual representation of a 3D physical object based on a polygon mesh).
Regarding Claim 36, Barker teaches or suggests wherein the region of interest is dynamically determined by the control circuit without manual designation of the region of interest ([0041]: According to various aspects, the range sensor 18 includes an internal coordinate frame which allows for subsequent scans to be easily aligned with or registered against the original scan of a given area of the mine. Additionally, the position of each range sensor 18 can be dynamically adjusted relative to its internal coordinate frame in order to assist in the scanning of a given area of the mine. [0055]: According to various aspects, the movement determination module 34 is also configured to automatically determine distances and angles between different tunnels (e.g., between adjacent or distant tunnels) of a mine and/or automatically determine the size/geometry of any pillars in the mine).
Regarding Claim 37, Barker teaches or suggests wherein the region of interest comprises a 2D region in a plane perpendicular to a lens of the image sensor ([0042]: … according to various aspects, the apparatus 12 includes a single camera 20 which includes a single lens (e.g., a single lens camera) and is configured to capture a two-dimensional image of the interior of the mine. … According to yet other aspects, the apparatus 12 includes multiple cameras 20 which are utilized to capture two-dimensional images of the interior of the mine from different positions/angles/orientations, thereby allowing for the captured images to be subsequently utilized to reconstruct the geometry of the mine … Also see [0050]: The electronic/digital anchors may be defined in any number of different ways. For example, according to various aspects, a given anchor point (an oriented X, Y, Z point) can be imported into one or more of the electronic three-dimensional models of the mine as the surveyed position of a real reference point in the mine (e.g. a distinctive outcrop of a sidewall of the mine). According to other aspects, a user can utilize the electronic anchor module 32 to create an electronic/digital anchor and insert the created electronic/digital anchor onto a three-dimensional surface of the mine displayed on the display device 38. According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine).
Regarding Claim 38, Barker teaches or suggests wherein the survey control point is automatically identified by the control circuit ([0050]: According to yet other aspects, the electronic anchor module 32 can be configured to automatically create and position electronic/digital anchors every few meters along a tunnel of the mine. [0049]: each electronic/digital anchor is described uniquely in space and time (it can be tracked as the oriented X, Y, Z point moves through space and over time) with sufficient resolution as to allow it to be identified across multiple scans of the underground mine obtained over a sufficient period of time to monitor and measure ground movement).
Regarding Claim 39, Barker teaches or suggests wherein the survey control point is automatically recognized by the control circuit without manual designation of the survey control point ([0049]: each electronic/digital anchor is described uniquely in space and time (it can be tracked as the oriented X, Y, Z point moves through space and over time) with sufficient resolution as to allow it to be identified across multiple scans of the underground mine obtained over a sufficient period of time to monitor and measure ground movement. [0052]: The movement determination module 34 is configured to automatically determine an occurrence of movement within the mine based on the positions (location and orientation) of the electronic/digital anchors over time relative to the electronic/digital three-dimensional models of the mine. Barker indirectly discloses the anchors which are the survey control points are recognized automatically by the computer system 14 which includes anchor generator module and movement determination module as shown in Fig.4).
Conclusion
Any inquiry concerning this this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613
Prosecution Insights