DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Pokale et al. (A Principled Formulation of Integrating Objects in Monocular SLAM”, Jan 2020, hereinafter “Pokale”)
Zhou et al. (CN 109887087 A, hereinafter “Zhou”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-28 are rejected under 35 U.S.C. 103 as being unpatentable over Pokale et al. (A Principled Formulation of Integrating Objects in Monocular SLAM”, Jan 2020, hereinafter “Pokale”) and in view of Zhou et al. (CN 109887087 A, hereinafter “Zhou”).
Regarding claims 1 and 21:
Pokale teaches an apparatus for tracking objects (Pokale Abstract, where Pokale discloses method and apparatus for localization and tracking of objects; we propose a novel approach in which objects are combined in the SLAM back-end to jointly optimize for the 3D structure, camera track as well as the object pose and shape), the apparatus comprising:
A system and apparatus for computer vision (Pokale Abstract and introduction) and configured to:
obtain a pose of an object in a world coordinate system (Pokale Section 3.2.2 Object Measurement using category models. Where Pokale teaches we assume that the SE3 pose(in the world frame) of each camera is denoted by Cj and from the jth pose, the robot observes M objects. The SE3 pose of each object observation m with respect to camera j is denoted by ˆTOm
j . The SE3 pose(in world frame) of each object observation m in the set is indexed as TOm. We denote the number of keypoints in the object category by K. The kth keypoint of the mth object observed are denoted by smk.);
obtain an image of the object from a camera position (Pokale section 3.3 paragraph 1 figs. 1 and 2);
obtain a world-to-camera transformation for relating the world coordinate
system to the camera position (Pokale section 3.2.2 where Pokale teaches We assume that the SE3 pose(in the world frame) of each camera is denoted by Cj and from the jth pose, the robot observes M objects. The SE3 pose of each object observation m with respect to camera j is denoted by ˆTom j . The SE3 pose(in world frame) of each object observation m in the set is indexed as Tom);
determine a reprojection error based on the pose of the object in the world coordinate system, the image, and the world-to-camera transformation ((Pokale section 3.2.3 and 3.3, where Pokal teaches When we run any SLAM, the reconstruction that we get is up to a scale. As mentioned in 3.2.2, we do the optimization of objects in Ceres and we need a good initialization for it converge to a better solution. We use the translation initialization method used in [9]. This gives us the initialization of the translation of the object with respect to the camera. In this method, the height and width of the bounding box are used to give an estimate of the translation. Let Tx, Ty, Tz be the translations in the x, y and z directions respectively with respect to the camera); and
adjust the pose of the object in the world coordinate system based on the reprojection error (Pokale section 3.2.3 and 3.3, where Pokal teaches optimizing for the global pose of the chair along with the other parameters. We show that solving for the global pose of chair along with shape aids in estimation of the Edge SLAM trajectory and 3D structure. The shape and pose in world frame of each object can then be computed by minimizing the following keypoint reprojection error).
Pokale fails to teach the apparatus comprising at least one memory and at least one processor coupled to the at least one memory.
However, the system of Pokale is directed to automatic use of Simultaneous Localization and Mapping (SLAM) from image captured by at least one camera for computer vision which inherently would use modern processor and memory to optimize the process. Nevertheless, Zhou teaches a SLAM mapping method and system of vehicle wherein the multi-frame based on the target image and the second camera module pose sequence determining the target camera module in the global SLAM map relocation pose. to make use of the target camera module the relocation pose the second vehicle pose sequence converted to the SLAM map coordinate system of the global SLAM map, converting the third vehicle pose sequence with the second camera module pose sequence to obtain the minimum error, the coordinate origin of the global SLAM map coordinate system is the target shot by the camera module light center position when the initial frame. optimizing unit for iteratively adjusting the value of the three-dimensional spatial position pose sequence of the target camera module included in each camera module pose of the value and the current SLAM map in the map point, until the target camera module sequence and the pose of the first pose sequence between the minimum projection error (Zhou [0025]-[0033], [0040]-[45], [0051]-[0056]).
Therefore, taking the teaching of Pokale and Zhou as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to use processor coupled with memory to implement the system, since it is a well known technique in the art to implement these kinds of systems.
Regarding claims 2 and 22:
Pokale in view of Zhou teaches wherein the at least one processor is further configured to determine that the object is stationary, wherein adjusting the pose of the object in the world coordinate system is based, at least in part, on determining that the object is stationary (Pokale section 3.2.2: Zhou [0025]-[0033]).
Regarding claims 3 and 23:
Pokale in view of Zhou teaches wherein the object is determined to be stationary based on one or more of:
respective locations of the object in a plurality of images of the object; a categorization of the object; or data from one or more other sensors (Pokale section 3.2.2: Zhou [0031]).
Regarding claims 4 and 24:
Pokale in view of Zhou teaches wherein: the image comprises a first image; the camera position comprises a first camera position;
the world-to-camera transformation comprises a first world-to-camera transformation;
the reprojection error comprises a first reprojection error; and the at least one processor is further configured to:
obtain a second image of the object from a second camera position;
obtain a second world-to-camera transformation for relating the world coordinate system to the second camera position;
determine a second reprojection error based on the pose of the object in the world coordinate system, the second image, and the second world-to-camera
transformation; and
adjust the pose of the object in the world coordinate system based on the
second reprojection error(Pokale section 3.2.3 and 3.3; Zhou [0025]-[0033).
Regarding claim 5:
Pokale in view of Zhou teaches wherein to determine the reprojection error, the at least one processor is further configured to:
obtain a model of the object;
obtain features of the object based the model of the object; and
determine the reprojection error based on comparing locations of the features of the
object in the image with locations of the features of the object as reprojected from the model
of the object in the pose of the object in the world coordinate system into an image plane
based on the camera position (Pokale section 3.2; Zhou [0025]-[0033).
Regarding claim 6:
Pokale in view of Zhou teaches wherein the model of the object comprises a point- cloud model of the object, a computer-aided design model of the object, or a neural radiance field (Pokale section 3.2; Zhou [0025]-[0033).
Regarding claim 7:
Pokale in view of Zhou teaches wherein to determine the reprojection error, the at
least one processor is further configured to:
obtain a model of the object;
obtain features of the object based the model of the object; and
determine the reprojection error based on comparing locations of the features of the
object in the image with locations of the features of the object as represented by the model
of the object with the pose of the object in the world coordinate system and world-to-camera
transformation applied to the model of the object (Pokale section 3.2; Zhou Abstract, [0004] and [0013]-[0015]).
Regarding claim 8:
Pokale in view of Zhou teaches wherein to determine the reprojection error, the at
least one processor is further configured to determine visual correspondences between the
image and a representation of the object in the world coordinate system with the world-to-
camera transformation applied thereunto (Pokale section 3.2; Zhou [0025]-[0033).
Regarding claim 9:
Pokale in view of Zhou teaches wherein the visual correspondences are determined
based on one or more of: a template-matching technique; a feature-matching technique; or
a deep-learning technique (Pokale section 3.2; Zhou Abstract, [0004] and [0013]-[0015]).
Regarding claim 10:
Pokale in view of Zhou teaches wherein to obtaining the pose of the object in the world coordinate system, the at least one processor is further configured to:
determine a pose of the object in a camera coordinate system based on a second
image of the object captured from a second camera position, wherein the pose of the object
in the camera coordinate system comprises a position of the object relative to the second
camera position and an orientation of the object; and
invert a second world-to-camera transformation and apply the inverted second world-
to-camera transformation to the pose of the object in the camera coordinate system to determine the pose of the object in the world coordinate system (Pokale section 3.1, 3.2; Zhou Abstract, [0004] and [0013]-[0015]).
Regarding claim 12:
Pokale in view of Zhou teaches wherein the world-to-camera transformation is
based on one or more of:
inertial data obtained from one or more inertial measurement units (IMUs); or
relative position data based on a simultaneous localization and mapping (SLAM)
technique (Pokale section 3.1; Zhou Abstract, [0004] and [0013]-[0015]).
Regarding claim 13:
Pokale in view of Zhou teaches wherein the at least one processor is further configured to determine to use the image from among a plurality of images (Pokale section 3.1; Zhou Abstract, [0004] and [0010]-[0015], [0027]).
Regarding claim 14:
Pokale in view of Zhou teaches wherein the image is determined to be used based on one or more of:
a difference between the camera position and another camera position;
a difference between a camera orientation and another camera orientation;
a difference between a timestamp of the image and a timestamp of another image; or
a quality of the image (Zhou [0075]-[0076]).
Regarding claims 15 and 25:
Pokale in view of Zhou teaches wherein the object comprises a first object and wherein the at least one processor is further configured to: determine that the first object is stationary; obtain a pose of a second object in the world coordinate system; determine that the second object is stationary; and adjust the pose of the second object in the world coordinate system based on the reprojection error (Pokale section 3.1; Zhou [0006]-[0011]).
Regarding claims 16 and 26:
Pokale in view of Zhou teaches wherein the at least one processor is further configured
to:
obtain a second image of the first object from a second camera position;
obtain a second world-to-camera transformation for relating the world coordinate
system to the second camera position;
determine a second reprojection error based on the pose of the first object in the
world coordinate system, the second image, and the second world-to-camera transformation;
adjust the pose of the first object in the world coordinate system based on the second
reprojection error; and adjust the pose of the second object in the world coordinate system based on the second reprojection error (Pokale section 3.2; Zhou [0006]-[0011], [0108]).
Regarding claims 17 and 27:
Pokale in view of Zhou teaches wherein the at least one processor is further configured
to:
associate the first object with the second object based on the first object and the second
object being stationary;
determine further reprojection errors based on further poses of the first object in the world coordinate system, further images, and further work-to-camera transformations;
adjust the pose of the first object in the world coordinate system and the pose of the second object in the world coordinate system based on the further reprojection errors based on the association between the first object and the second object (Pokale section 3.2; Zhou [0006]-[0011], [0108).
Regarding claims 18 and 28:
Pokale in view of Zhou teaches wherein the at least one processor is further configured to:
determine that the second object is not stationary; disassociate the first object and the second object; and adjust the pose of the first object in the world coordinate system based on the further reprojection errors (Zhou [0123]).
Regarding claim 19:
Pokale in view of Zhou teaches wherein the at least one processor is further configured to track the object (Pokale section 3.1; Zhou [0016]).
Regarding claim 20:
Pokale in view of Zhou teaches wherein the at least one processor is further configured to generate virtual content based on the pose of the object in the world coordinate system (Pokale section 3.1).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 March 19, 2026
Prosecution Insights