DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Interpretation under 35 USC § 112
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
3. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
4. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: an image detecting unit, a correspondence relationship specifying unit, and an orientating unit in claim 1-4, and 8-9; an image data receiving unit, a laser scan data receiving unit, a feature obtaining unit, a feature extracting unit, and a correspondence relationship specifying unit in claim 5.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
5. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
6. Claims 1-4 and 6-9 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, and 6-7 each recites "wherein the first image and/or the second image is detected as an image having feature points of the maximal number and/or an image having feature points of number larger than a predetermined value, from objective images”. Regarding the term "and/or", One of ordinary skill in the art is unable to ascertain what the applicant regards as his invention since one would not be able to determine whether the limitations that occur after the "and or" are required in combination with the limitations before or merely optional. Therefore claims 1 and 6-7 and dependent claims 2-4, and 8-9 are deemed indefinite.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1,148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
6. Claims 1 and 4 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Kitahara et al. (English Translation of Japanese Publication JP2012225869 11-2012) in view of Sasaki et al. (US Publication 2016/0063703, hereinafter Sasaki), and further in view of Sasaki (US Publication 2022/0284608, hereinafter Sasaki-4608).
Regarding claim 1, Kitahara discloses an optical data processing device comprising:
an image detecting unit which detects a second image among images photographed by a second camera externally attached to a surveying device (Kitahara, para’s 0007-0010, a surveying/measuring device detects an image captured by a camera attached to the device).
Kitahara does not explicitly disclose but Sasaki discloses:
the image detecting unit detects a first image among images photographed by a first camera included in the surveying device, a correspondence relationship specifying unit which specifies a correspondence relationship between the first image and the second image, (Sasaki, fig. 1, para’s 0021-0027, a measuring system 110 in a vehicle 100 comprising a base on which a GNSS unit 111, an operating part 120, a camera 112, a camera 113, and an IMU 114 are mounted; the camera 112 is fixed to the vehicle 100 (IMU 114), and exterior orientation parameters thereof with respect to the vehicle 100 (IMU 114) are determined in advance. A panoramic camera can be used as the camera 112 to photograph conditions in 360 degrees, or a wide-angle camera, which can photograph over a wide-angle range; the camera 113 is capable of photographing moving images and still images and is mounted to the vehicle 100 at a position freely selected by a user. The camera 113 is arranged on the base of the measuring system 110 in this example, but it can be mounted at another location of the vehicle 100. However, the camera 113 must be arranged at a position in a particular attitude so as to be able to photograph the same object as the camera 112), and
a correspondence relationship specifying unit which specifies a correspondence relationship between the first image and the second image (Sasaki, para’s 0029-0037, the matched point selecting unit 123 selects common feature points, i.e., matched feature points, between the image photographed by the camera 112 and the image photographed by the camera 113. In this example, common feature points between the superposed reference image and the superposed comparative image are selected as the matched points), and
an orientating unit which calculates relationships of location and orientation between the first camera and the second camera based on the correspondence relationship, wherein the first image and/or the second image is detected as an image having feature points of the maximal number and/or an image having feature points of number larger than a predetermined value, from objective images (Sasaki, para’s 0029-0037, the exterior orientation parameter calculating unit 124 performs relative orientation based on the images photographed by the cameras 112 and 113 and calculates the exterior orientation parameters of the camera 113 by adjusting the scale with a predetermined scale. In this example, the exterior orientation parameters of the camera 113 are calculated by performing relative orientation based on the superposed reference image and the superposed comparative image and by adjusting the scale with a predetermined scale. After the matched points are selected, exterior orientation parameters of the camera 113 are calculated by performing relative orientation and by adjusting the scale with a predetermined scale (Step S404). This processing is performed by the exterior orientation parameter calculating unit 124. FIG. 5 is an explanatory drawing for explaining the principle of the relative orientation. In the processing of the relative orientation, at least six points (orientation parameters: pass points) are selected among the above matched points and correspond between the right and the left images, whereby a stereo model is generated. By obtaining the stereo model, the relative relationships of the position and the attitude between the two cameras, by which the right and the left images are obtained, are determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Sasaki’s features into Kitahara’s invention for generating a surveying device that can effectively and optically monitor a range of a moving target.
Regarding claim 4, Kitahara-Sasaki discloses the optical data processing device according to one of claim 1, wherein the surveying device includes a function to perform laser positioning, relationships of location and orientation between optical system performing the laser positioning and the first camera are known, location of feature of the first image is specified by the laser positioning, and based on the location of the feature of the first image, relationships of location and orientation between the first camera and the second camera are calculated (Kitahara, para. 0040, for measuring devices that do not have a built-in laser, look through the eyepiece of the measuring device to identify the center of the field of view and click to select that position. Therefore, Kitahara implicitly suggests that a laser device can be used to perform laser positioning; Sasaki, para’s 0029-0037, the exterior orientation parameter calculating unit 124 performs relative orientation based on the images photographed by the cameras 112 and 113 and calculates the exterior orientation parameters of the camera 113 by adjusting the scale with a predetermined scale. In this example, the exterior orientation parameters of the camera 113 are calculated by performing relative orientation based on the superposed reference image and the superposed comparative image and by adjusting the scale with a predetermined scale. After the matched points are selected, exterior orientation parameters of the camera 113 are calculated by performing relative orientation and by adjusting the scale with a predetermined scale (Step S404). This processing is performed by the exterior orientation parameter calculating unit 124. FIG. 5 is an explanatory drawing for explaining the principle of the relative orientation. In the processing of the relative orientation, at least six points (orientation parameters: pass points) are selected among the above matched points and correspond between the right and the left images, whereby a stereo model is generated. By obtaining the stereo model, the relative relationships of the position and the attitude between the two cameras, by which the right and the left images are obtained, are determined. As a result, relationships of location and orientation between optical system performing the laser positioning and the first camera are known, location of feature of the first image is specified by the laser positioning, and based on the location of the feature of the first image, relationships of location and orientation between the first camera and the second camera are calculated).
The motivation to combine the references and obviousness arguments are the same as claim 1.
Claims 6-7 are rejected for similar reasons as discussed in claim 1, Kitahara-Sasaki further discloses computer readable media (see Sasaki, para. 0028, external storage media).
7. Claim 5 is rejected under AIA 35 U.S.C. 103 as being unpatentable over Kitahara et al. (English Translation of Japanese Publication JP2012225869 11-2012) in view of Sasaki (US Publication 2022/0284608).
Regarding claim 5, Kitahara discloses an image detecting unit which detects a second image among images photographed by a second camera externally attached to a surveying device (Kitahara, para’s 0007-0010, a surveying/measuring device detects an image captured by a camera attached to the device).
Kitahara does not explicitly disclose but Sasaki discloses:
the image data receiving unit receives image data of images photographed by a camera externally attached to a surveying device having laser scanning function, a laser scan data receiving unit which receives laser scan data which is obtained by laser scanning using the laser scanning function (Sasaki, fig’s 1-2, para’s 0057-0058, 0062-0074, the vehicle 100 has mounted thereon the camera 101, the laser scanner 102, the GNSS position measuring unit 103, IMU (inertial measuring unit) 106, and operational unit 108. The digital still camera 101 repeatedly takes photographs of static images at a specific time interval. The laser scanner 102 obtains laser scan data by scanning a wide range or a specific range with laser light for distance measuring),
a feature obtaining unit from image, which obtains feature of photographing object based on the image data, a feature extracting unit from laser scan data, which extracts feature of the laser scanning object base on the laser scan data (Sasaki, fig’s 6-8, para’s 0062-0074, and 0111-0116, he point cloud feature point calculating part 304 calculates feature points of an object described by the laser scan point cloud based on the laser scan point cloud generated by the point cloud generating part 303. For example, in the case of FIG. 1, feature points of a shape of building 200 are calculated, based on the laser scan point cloud of the building 200. Point cloud feature point images are made by imaging this feature point. This point cloud feature point image is an example of the point cloud image, and it is an image of feature points of an object derived from the laser scan data. Two methods can be mentioned as a method to calculate feature points of the shape of the building 200. The first method is a method in which feature points of the building 200 are extracted from laser scan point clouds targeting the building 200. The second method is a method in which, based on the laser scan point cloud targeting the building 200, a three-dimensional model of the building 200 is made, and feature points of the building 200 are extracted from the three-dimensional model.
The delay time (Δt) obtaining part 305 obtains the delay time Δt which is a time from commanding the camera 101 to photograph (outputting the photographing command signal) to actually completion of photographing by the camera 101. The Δt is obtained by the methods explained in FIGS. 1 to 4; the point cloud feature point image projection part 308 projects the point cloud feature point image which is calculated by the point cloud feature point calculating part 304 based on the laser scan point cloud which is generated by the point cloud generating part 303, overlapping with the photographed image, so that a superposed projection image in which the two images are superposed is generated. FIGS. 2 and 4 show examples of superposed projection images. In the process in FIG. 4, the point cloud image based on the laser scan point cloud is projected onto the photographed image photographed by the camera, so that extent of superposition of the two images is studied. Here, the point cloud image itself is not used; instead, the point cloud feature point image in which an objective feature point is extracted from the laser scan point cloud is used as the point cloud image, and this image is projected onto the photographed image. It should be noted that as the photographed image which is the object of projection in this case, an image feature point image is used which is obtained by extracting the feature point from the photographed image),
a correspondence relationship specifying unit which specifies correspondence relationship between feature of the photographing object and feature of the scanning object, and an orientating unit which calculates relationships of location and orientation between the camera and the surveying device based on the correspondence relationship, wherein the feature of laser scanning object is a feature of image of which 3D model obtained by the laser scan data is viewed from location of the laser scanning (Sasaki, para’s 0095-0097, calculation of the exterior orientation elements of the camera 101 at the photographing time of the photographed image is performed. If the correspondence relationship of the point cloud image and the photographed image is obvious, each position in the absolute coordinate system of multiple points in the photographed image will also be obvious. Here, the multiple points in the photographed image of which the coordinates are obvious being reference points (orientation points), position of the camera 101 in the absolute coordinate system is calculated by the backward intersection method; In FIG. 6, the camera locates the position X, p1 to p6 are feature points in display of the photographed image photographed by the camera 101, and P1 to P6 are points of the laser scan point cloud corresponding to the points p1 to p6. It should be noted that the position of the camera X is unknown, and interior orientation elements of the camera are known. Furthermore, the position of the camera is the projection origin (optical origin) of the camera; para’s 0110-0113, after calculating Δt, the exterior orientation elements (position and orientation) of the camera 101 in the vehicle 100 at the photographing time T.sub.1=T+Δt of the photographed image which is focused on here is calculated. By calculating Δt, the actual photographing time T.sub.1=T+Δt of the camera 101 will be obvious. As a result, the position of the vehicle 100 at the time T.sub.1, that is, the position of the vehicle 100 at the time of photographing by the camera 101 can be known. In addition, the orientation of the vehicle 100 can be known based on the measured data by the IMU 106 at the time T.sub.1. Then, the position of the camera 101 in the vehicle 100 can be known based on the relationship of the position of the vehicle 100 at the time T.sub.1 and the position X.sub.1 of the camera 101 at the time T.sub.1. Furthermore, the orientation of the camera 101 in the absolute coordinate system at the time T.sub.1 is calculated in the step S106. Therefore, the orientation of the camera 101 in the vehicle 100 can be known based on relationships of the orientation of the vehicle 100 in the absolute coordinate system at the time T.sub.1 and the orientation of the camera 101 in the absolute coordinate system at the time T.sub.1. In this way, the exterior orientation elements (position and orientation) of the camera 101 in the vehicle 100 are calculated. These processes are performed in the camera position and orientation calculating part 307; para. 0113, with reference to FIG. 8, a process is explained in which the image photographed by the camera 101 and the point cloud image based on the laser scan data obtained by the laser scanner 102 are synchronized. It should be noted that the process of FIG. 8 is performed under conditions in which the exterior orientation elements of the camera 101 in the vehicle 100 are known; para’s 0067-0069, the point cloud generating part 303 generates the laser scan point cloud based on the laser scan data obtained by the laser scanner 102. The laser scanner 102 measures a direction to the reflection point of laser scan light (a direction viewed from the laser scanner) and a distance to the reflection point, and outputs data of the direction and the distance to the reflection point as laser scan data. Based on the direction and the distance, three-dimensional coordinates of the reflection point (laser scan point) are calculated. The point cloud feature point calculating part 304 calculates feature points of an object described by the laser scan point cloud based on the laser scan point cloud generated by the point cloud generating part 303).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Sasaki’s features into Kitahara’s invention for generating a surveying laser scanning device that can effectively and optically monitor a range of a moving target.
8. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. These include:
Palm, US Patent 5,699,444
Metzler et al., US Publication 2022/0333925
Examiner’s notes
Claims 2-3 comprise allowable subject matter, and are rejected as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all the limitations of the base claim and overcoming the U.S.C. 112(b) rejection.
The remaining claims 8-9 depending on claims 2-3 are rejected thereof.
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to LOI H TRAN whose telephone number is (571)270-5645. The examiner can normally be reached 8:00AM-5:00PM PST FIRST FRIDAY OF BIWEEK OFF.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LOI H TRAN/Primary Examiner, Art Unit 2484
Prosecution Insights