Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21 – 33 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 21 and 31, the phrase " are usable to" renders the claim indefinite because it is unclear whether the limitation(s) following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Regarding claims 22 – 30 and 32 – 33, they are dependent claims for claims 21 and 31, respectively, thus, they are rejected accordingly.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21 – 40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
In regarding claims 21, 31 and 34:
Step 1:
Claims 21, 31 and 34 are directed towards a process, machine, manufacture or composition of matter which is/are statutory subject matter.
Step 2A:
Prong 1:
Claims 21, 31 and 34 are directed an idea for determine a three-dimensional position of an object, which is an abstract idea.
Consideration of the claimed elements:
Regarding claims 21 and 31:
A computer device/computer readable media comprising:
receive one or more images captured by a camera;
identify an object on a ground surface in the one or more images;
select, based on a two-dimensional position of the object in the one or more images, one or more homographies from a plurality of homographies, wherein the plurality of homographies are usable to determine a three-dimensional position of the object in an environment of the one or more images; and
determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images.
For the limitation of “identify an object on a ground surface in the one or more images”, it can be interpreted as a person can identify an object on a ground surface in the one or more images. The claimed limitation can be broadly read as mental processes.
For the limitation of “select, based on a two-dimensional position of the object in the one or more images, one or more homographies from a plurality of homographies, wherein the plurality of homographies are usable to determine a three-dimensional position of the object in an environment of the one or more images”, it can be interpreted as the person can select one or more reference patterns (homographies) to determine object position. The claimed limitation can be broadly read as mental processes.
For the limitation of “determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images”, it can be interpreted as the person can determine/track, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images. The claimed limitation can be broadly read as mental processes.
Regarding claim 34:
A method for determining a three-dimensional (3D) position of a point in a two-dimensional (2D) image of an environment, the method comprising:
receiving the image of the environment, the image associated with a plurality of homographies mapping between points in an image plane and points in the environment;
identifying, within the image, a reference surface within the environment;
determining respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of homographies; and
determining a 3D position of the first point in the 2D image at least partly from an average of at least two of the 3D position estimates.
For the limitation of “identifying, within the image, a reference surface within the environment”, it can be interpreted as a person can identify a reference surface within the environment. The claimed limitation can be broadly read as mental processes.
For the limitation of “determining respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of homographies”, it can be interpreted as the person can determine/track respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of reference patterns (homographies). The claimed limitation can be broadly read as mental processes.
For the limitation of “determining a 3D position of the first point in the 2D image at least partly from an average of at least two of the 3D position estimates”,
it can be interpreted as the position is determined from an average of at least two of the 3D position estimates. The claimed limitation can be broadly read as mathematical concepts.
Prong 2:
The claims include additional elements of
a computer device, a processor, a memory; one or more non-transitory computer readable media;
receive one or more images captured by a camera;
receiving the image of the environment, the image associated with a plurality of homographies mapping between points in an image plane and points in the environment.
A computer device, a processor, a memory; one or more non-transitory computer readable media are general computer elements.
“[R]eceive one or more images” is considered as data gathering of adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)
Moreover, the claim limitations that are not indicative of integration into a practical application.
Thus, the above recited additional elements perform no more than their basic computer function. Generic computer-implementation of a method is not a meaningful limitation that alone can amount to significantly more than an abstract idea. Moreover, when viewed as a whole with such additional element considered as an ordered combination, claims modified by adding a generic memory and processor are nothing more than a purely conventional computerized implementation of an idea in the general field of computer processing and do not provide significantly more than an abstract idea.
Accordingly, the claims are directed to an idea of itself, and therefore not patent eligible.
Step 2B:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception such as improvements to another technology or technical field, or other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment.
Moreover, the claim language that may be separate from the abstract idea (i.e., additional elements) include general computer elements and data gathering. The additional elements (e.g., general computer elements and data gathering) simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC) - see MPEP 2106.05(d) and 2106.07(a)III.
Thus, the recited generic additional element (e.g., general computer elements and data gathering) perform no more than their basic computer function. Generic computer-implementation of a method is not a meaningful limitation that alone can amount to significantly more than an abstract idea. Moreover, when viewed as a whole with such additional element considered as an ordered combination, claims modified by adding a generic memory are nothing more than a purely conventional computerized implementation of an idea in the general field of computer processing and do not provide significantly more than an abstract idea.
Consequently, the identified additional elements taken into consideration individually or in combination fails to amount of significantly more than the abstract idea above.
Regarding claims 22 – 30, 32 – 33 and 35 – 40, the rejection is based on the same rationale described for claims 21, 31 and 34, because the claims include/inherit the same/similar type of problematic limitation(s) as claims 21, 31 and 34, wherein limitations regarding “determine …”, “identify …”, “cause …”, “receive …”, “calculate …”, correspond to …”, “comprising …”, “are associated …” and/or “captures …” is/are of sufficient breadth that it would be substantially directed to or reasonably interpreted as a part of the “mental processes” as the abstract idea (similar to claims 21, 31 and 34 as stated above). It is noted that further additional limitation is merely generic/conventional computer component/steps to implement the abstract idea, which is, individually or in combination, not sufficient to amount to significantly more than the judicial exception. Therefore, the claimed invention as a whole is directed to an ineligible subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 21, 28 – 31 and 33 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al. (WIPO Patent Application Publication WO 2020/114234), hereinafter referred as Wang.
Regarding claim 21, Wang discloses a compute device (Fig. 6) comprising:
a processor (Fig. 6, #602);
a memory (Fig. 6, #603) coupled to the processor; and
one or more non-transitory computer readable media (Fig. 6, #603, pages 10 – 11) comprising a plurality of instructions that, when executed by the processor, cause the processor to:
receive one or more images captured by a camera (Fig. 1, step S101, page 6, acquire images from camera);
identify an object on a ground surface in the one or more images (Fig. 3, page 8, identify target A on a ground surface);
select, based on a two-dimensional position of the object in the one or more images, one or more homographies from a plurality of homographies (Fig. 3, page 8, select 4 non-collinear points (one homography) in these 9 calibration points (a plurality of homographies)), wherein the plurality of homographies are usable to determine a three-dimensional position of the object in an environment of the one or more images (page 8, 9 calibration points (a plurality of homographies) are usable to determine a three-dimensional position of the target A); and
determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images (Fig. 1, step S103, pages 8 – 9, based on the one or more images and the one or more homographies, target GPS coordinates (three-dimensional position) can be determined).
Regarding claim 28 (depends on claim 21), Wang discloses the device wherein the plurality of instructions further cause the processor to: receive one or more calibration images captured by the camera, wherein each of the one or more calibration images include a calibration pattern on the ground surface (Fig. 2, page 8, 9 calibration points); and calculate the one or more homographies based on the one or more calibration images (page 8).
Regarding claim 29 (depends on claim 21), Wang discloses the device wherein each of the plurality of homographies correspond to a different position and orientation of the ground surface in a field of view of the camera (page 9, PTZ parameters for camera).
Regarding claim 30 (depends on claim 21), Wang discloses the device further comprising the camera (page 9, camera).
Regarding claim 31, it is corresponding to claim 21, thus, it is interpreted and rejected for the same reason set forth for claim 21.
Regarding claim 33 (depends on claim 31), Wang discloses the computer readable media wherein each of the plurality of homographies correspond to a different position and orientation in a field of view of the camera (page 9, PTZ (pan/tilt/zoom) of the camera).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 22, 32 and 34 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Wang et al. (Chinese Patent Application Publication CN 108154526), hereinafter referred as Wang’526.
Regarding claim 22 (depends on claim 21), Wang fails to explicitly disclose the device wherein the one or more homographies comprises two or more homographies, wherein to determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images comprises to: determine, for each of the two or more homographies, an estimated position of the object; and determine a final position of the object based on a weighted average of the estimated positions corresponding to the two or more homographies.
However, in a similar field of endeavor Wang’526 discloses a method for image processing (abstract). In addition, Wang’526 discloses the method the one or more homographies comprises two or more homographies, wherein to determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images comprises to: determine, for each of the two or more homographies, an estimated position of the object; and determine a final position of the object based on a weighted average of the estimated positions corresponding to the two or more homographies (page 15 – 16).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and the one or more homographies comprises two or more homographies, wherein to determine, based on the one or more images and the one or more homographies, a three-dimensional position of the object in the environment of the one or more images comprises to: determine, for each of the two or more homographies, an estimated position of the object; and determine a final position of the object based on a weighted average of the estimated positions corresponding to the two or more homographies. The motivation for doing this is to that the final position can be more accurate.
Regarding claim 32, it is corresponding to claim 22, thus, it is interpreted and rejected for the same reason set forth for claim 22.
Regarding claim 34, Wang discloses a method for determining a three-dimensional (3D) position of a point in a two-dimensional (2D) image of an environment (Fig. 1), the method comprising:
receiving the image of the environment (Fig. 1, step S101, page 6, acquire images from camera), the image associated with a plurality of homographies mapping between points in an image plane and points in the environment (Figs. 2 – 3, page 8);
identifying, within the image, a reference surface within the environment (Fig. 3, page 8, identify target A on a ground surface); and
determining respective 3D position estimates of a first point on the reference surface at least partly from one or more of the plurality of homographies (Fig. 1, step S103, pages 8 – 9, based on the one or more images and the one or more homographies, target GPS coordinates (three-dimensional position) can be determined).
However, Wang fails to explicitly disclose the method wherein determining respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of homographies; and determining a 3D position of the first point in the 2D image at least partly from an average of at least two of the 3D position estimates.
However, in a similar field of endeavor Wang’526 discloses a method for image processing (abstract). In addition, Wang’526 discloses the method determining respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of homographies; and determining a 3D position of the first point in the 2D image at least partly from an average of at least two of the 3D position estimates (page 15 – 16).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and determining respective 3D position estimates of a first point on the reference surface at least partly from two or more of the plurality of homographies; and determining a 3D position of the first point in the 2D image at least partly from an average of at least two of the 3D position estimates. The motivation for doing this is to that the final position can be more accurate.
Claim(s) 23 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Francois et al. (US Patent 10,284,794), hereinafter referred as Francois.
Regarding claim 23 (depends on claim 21), Wang discloses the device wherein the plurality of instructions further cause the processor to: identify a three-dimensional position of an object above the ground surface based on the one or more homographies, and the one or more images (Fig. 1, step S103, pages 8 – 9, based on the one or more images and the one or more homographies, target GPS coordinates (three-dimensional position) can be determined).
However, Wang fails to explicitly disclose the device further determine a gravity direction in the one or more images; and identify an additional object above the ground surface based on the gravity direction.
However, in a similar field of endeavor Francois discloses a method for image processing (abstract). In addition, Francois discloses the method determine a gravity direction in the one or more images; and identify an additional object above the ground surface based on the gravity direction (col. 6, lines 32 – 58, col. 14, lines 27 – 38).
There was some teaching, suggestion, or motivation, either in the references themselves or in the knowledge generally available to one of ordinary skill in the art, to modify the reference or to combine reference teachings to achieve the claimed limitation.
There was reasonable expectation of success to arrive at the claimed limitation (KSR scenario G).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and determine a gravity direction in the one or more images; and identify an additional object above the ground surface based on the gravity direction. The motivation for doing this is to produce a smooth, 360 degree, 3D composite image of an object from a plurality of images.
Regarding claim 24 (depends on claim 23), Francois discloses the device wherein to determine the gravity direction comprises to determine a gravity direction based on a vertical object identified in an image captured by the camera (col. 6, lines 32 – 58, col. 14, lines 27 – 38).
Claim(s) 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Francois, and in further view of Shimatani et al. (US Patent Application Publication 2018/0376068), hereinafter referred as Shimatani.
Regarding claim 25 (depends on claim 23), Wang in view of Francois fails to explicitly disclose the device wherein to determine the gravity direction comprises to determine a gravity direction based on a weight on a string identified in an image captured by the camera.
However, in a similar field of endeavor Shimatani discloses a method for image processing (abstract). In addition, Shimatani discloses the method determine the gravity direction comprises to determine a gravity direction based on a weight on a string identified in an image captured by the camera ([0067]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and determine the gravity direction comprises to determine a gravity direction based on a weight on a string identified in an image captured by the camera. The motivation for doing this is gravity direction can be visible so that it would be easy for image process.
Claim(s) 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Francois, and in further view of Wirola et al. (US Patent 10,659,923), hereinafter referred as Wirola.
Regarding claim 26 (depends on claim 23), Wang in view of Francois fails to explicitly disclose the device wherein to determine the gravity direction comprises to access an indication of the gravity direction stored on the compute device.
However, in a similar field of endeavor Wirola discloses a method for determining an indicator information indicative of one or more indicators of an object (abstract). In addition, Wirola discloses the method determine the gravity direction comprises to access an indication of the gravity direction stored on the compute device (claim 20).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Francois, and determine the gravity direction comprises to access an indication of the gravity direction stored on the compute device. The motivation for doing this is that gravity direction can be obtained from other resources so that application of Francois can be extended.
Claim(s) 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Liu et al. (Chinese Patent Application Publication CN 111339850), hereinafter referred as Liu.
Regarding claim 27 (depends on claim 21), Wang fails to explicitly disclose the device wherein the object is part of an animal, wherein the plurality of instructions further cause the processor to determine, based at least in part on the three-dimensional position of the object, whether the animal is lame.
However, in a similar field of endeavor Liu discloses a method for image processing (abstract). In addition, Liu discloses the method the object is part of cow, and determine, based at least in part on hoof movement in 3D space of cow, whether the animal is lame (page 4 – 5).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and the object is part of cow, and determine, based at least in part on the three-dimensional position of the object, whether the animal is lame. The motivation for doing this is to that the application of Wang can be extended to other field.
Claim(s) 35 and 40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Wang’526, and in further view of Lee et al. (US Patent Application Publication 2021/0374976), hereinafter referred as Lee.
Regarding claim 35 (depends on claim 34), Wang fails to explicitly disclose the method wherein at least some of the plurality of homographies are associated with a portion of the reference surface with varying slope.
However, in a similar field of endeavor Lee discloses a method for image processing (abstract). In addition, Lee discloses the method at least some of the plurality of homographies are associated with a portion of the reference surface with varying slope ([0038]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and at least some of the plurality of homographies are associated with a portion of the reference surface with varying slope. The motivation for doing this is to that more complicated situations can be handled so that the application of Wang can be extended.
Regarding claim 40 (depends on claim 35), Wang discloses the method wherein the image captures a plurality of calibration patterns on the reference surface (page 8).
Claim(s) 36 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Wang’526, and in further view of Farag et al. (US Patent 11,044,400), hereinafter referred as Farag.
Regarding claim 36 (depends on claim 34), Wang in view of Wang’526 fails to explicitly disclose the method determining the 3D position of the point in the 2D image comprises determining the 3D position at least partly from a weighted average of the at least two of the 3D position estimates, wherein a weighting assigned to one of the 3D position estimates from a homography closer to the first point is higher than a weighting assigned to one of the 3D position estimates from a homography further from the first point.
However, in a similar field of endeavor Farag discloses a method for image processing (abstract). In addition, Farag discloses the method wherein a weighting assigned to one of the 3D position estimates from a homography closer to the first point is higher than a weighting assigned to one of the 3D position estimates from a homography further from the first point (col. 4, lines 47 – 49, weight is assigned higher closer to centroid (first point) than further from the centroid).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and a weighting assigned to one of the 3D position estimates from a homography closer to the first point is higher than a weighting assigned to one of the 3D position estimates from a homography further from the first point. The motivation for doing this is to that important point can be focused.
Claim(s) 37 and 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Wang’526, and in further view of Kirsch et al. (European Patent Application Publication EP 3 573 024), hereinafter referred as Kirsch.
Regarding claim 37 (depends on claim 34), Wang in view of Wang’526 fails to explicitly disclose the method further comprising: determining a reference projection direction within the environment; and determining a 3D position of a second point in the image based at least partly from an intersection of a first ray substantially parallel to the reference projection direction extending through the first point and a second ray extending through both the second point and a camera optical centre.
However, in a similar field of endeavor Kirsch discloses a method for image processing (abstract). In addition, Kirsch discloses the method wherein determining a reference projection direction within the environment; and determining a 3D position of a second point in the image based at least partly from an intersection of a first ray substantially parallel to the reference projection direction extending through the first point and a second ray extending through both the second point and a camera optical centre (page 25, lines 9 - 17).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and determining a reference projection direction within the environment; and determining a 3D position of a second point in the image based at least partly from an intersection of a first ray substantially parallel to the reference projection direction extending through the first point and a second ray extending through both the second point and a camera optical centre. The motivation for doing this is to that external calibration can be performed properly.
Regarding claim 39 (depends on claim 37), Kirsch discloses the method wherein the first point is on a ray that extends through first points of other images with the same perspective of the environment; wherein each of the first points of other images correspond to different positions of a moving object on the reference surface (Fig. 13 – 14, [0091 – 0097]).
Claim(s) 38 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Wang’526, in further view of Kirsch and Francois.
Regarding claim 38 (depends on claim 37), Wang discloses the method wherein the reference surface comprises a ground surface within the environment (Fig. 4, a ground surface within the environment).
However, Wang fails to explicitly disclose the reference projection direction comprises a direction of gravity within the environment.
However, in a similar field of endeavor Francois discloses a method for image processing (abstract). In addition, Francois discloses the method determine a gravity direction in the one or more images (col. 6, lines 32 – 58, col. 14, lines 27 – 38).
There was some teaching, suggestion, or motivation, either in the references themselves or in the knowledge generally available to one of ordinary skill in the art, to modify the reference or to combine reference teachings to achieve the claimed limitation.
There was reasonable expectation of success to arrive at the claimed limitation (KSR scenario G).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Wang, and the reference projection direction comprises a direction of gravity within the environment. The motivation for doing this is to produce a smooth, 360 degree, 3D composite image of an object from a plurality of images.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QIAN YANG whose telephone number is (571)270-7239. The examiner can normally be reached on Monday-Thursday 8am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QIAN YANG/
Primary Examiner, Art Unit 2677