DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 9-11, and 14-16 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable by Mai (US 20250086816 A1).
Regarding Claim 1, representative of Claim 9 and 14, Mai teaches a motion system, comprising:
one or more processors;
a memory communicably coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:
acquire images depicting surrounding objects present in an environment ([0100] As described above, in the information processing apparatus of the embodiment, not only the motion of the stationary part such as the background but also the motion (three-dimensional motion) of the moving object in the image are modeled);
generate depth maps for the images according to a depth model that performs monocular depth estimation ([0060]: the estimation unit 122 may use the estimation model MA to perform the estimation process to estimate the depth of each input image);
generate, using a motion model, an indicator specifying a presence and a location of motion associated with the surrounding objects according to the depth maps ([0017]: motion information indicating the motion of each of a plurality of pixels in a three-dimensional space is output, [0041]: converts the per-pixel three-dimensional motion 312 into optical flow);
provide the indicator about motion ([0017]: motion information indicating the motion of each of a plurality of pixels in a three-dimensional space is output, [0059]: estimation unit performs an estimation process using the estimation models, [0058]: motion calculation unit 112 (estimation model MB, [0061]: the output control unit 102 displays a result of the estimation process by the estimation unit 122 on a display. Examiner notes the breadth of “providing an indicator”. The outputted 3D motion estimation is a provided indicator of motion); and
train the motion model using a heuristic to generate supervising annotations for the depth maps by comparing the depth maps to directly identify motion ([0028] Each of the estimation model MA and the estimation model MB may be a model of any structure, and is, for example, a neural network model (hereinafter, also simply referred to as a neural network) or a machine learning model such as random forest. Examiner notes random forest is primarily a supervised model. [0077] The loss function represented by Formula (3) indicates a difference between the depth information DA and depth information DB (depth D) and the depth training data (depth D.sup.T) that is the training data concerning the depth).
Regarding Claim 2, representative of Claims 10 and 15, Mai teaches the motion system of claim 1. In addition, Mai teaches wherein the instructions to generate the indicator include instructions to compare the depth maps on a per-pixel basis according to a heuristic ([0017] Estimation model MB: a model (second estimation model) into which two pieces of depth information obtained for two input images are input and from which motion information indicating the motion of each of a plurality of pixels in a three-dimensional space is output.)
Regarding Claim 3, representative of Claims 11 and 16, Mai teaches the motion system of claim 2. In addition, Mai teaches wherein the instructions to generate the indicator include instructions to generate the indicator according to the heuristic that compares depth values within the depth maps to identify pixels with values that have changed to indicate increasing or decreasing depths ([0017] Estimation model MB: a model (second estimation model) into which two pieces of depth information obtained for two input images are input and from which motion information indicating the motion of each of a plurality of pixels in a three-dimensional space is output, [0041]: converts the per-pixel three-dimensional motion 312 into optical flow. Examiner notes the 3D pixel motion estimation is taking the two depths to identify differences between them. Optical flow would indicate these changes).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6-8 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Mai (US 20250086816 A1) in view of Wang (US 10970856 B2).
Regarding Claim 6, representative of Claim 19, Mai teaches the motion system of claim 1. Mai does not explicitly teach the remaining limitations of Claim 6. However, Wang teaches, wherein the instructions to generate the indicator include instructions to compensate for motion of a platform on which a camera is mounted by generating a transformation that defines a change in position between poses of the camera when capturing the images ([0042]: A motion network predicts (315) a relative camera transformation T.sub.t.fwdarw.s based on the first and the second images, [0050]: it is necessary to distinguish between the motion from rigid background/camera motion and dynamic moving objects).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified Mai to include the teachings of Wang doing so would improve object motion estimation from depth by compensating for background motion.
Regarding Claim 7, Mai teaches the motion system of claim 1. Mai does not explicitly teach the remaining limitations of Claim 7. However, Wang teaches wherein the instructions to provide the indicator include instructions to associate the motion with an identified object of the surrounding objects according to a semantic model that identifies the surrounding objects and a pixel-wise association of the motion in relation to the identified object ([0036]: segment moving objects. Finally, since one or more methodology embodiments in this patent document decomposes static background and moving objects, the disclosed approach is also related to segmentation of moving objects from a given video. Current contemporary SOTA methods are dependent on supervision from human labels by adopting CNN image features or RNN temporal modeling).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified Mai to include the teachings of Wang by including a semantic model that identifies the identified and surrounding objects. Doing so would improve the accuracy of identifying motion of moving objects.
Regarding Claim 8, Mai teaches the motion system of claim 1. In addition, Mai teaches wherein the depth model performs monocular depth estimation ([0023]: a method using a system that estimates a depth from one image). Mai does not explicitly teach trained according to self-supervised structure-from-motion (SfM) training.
Wang explicitly teaches trained according to self-supervised structure-from-motion training ([0018]: recently, for unsupervised single image depth estimation, impressive progress has been made to train a deep network taking only unlabeled samples as input and using 3D reconstruction for supervision, yielding even better depth estimation results than those of supervised methods in outdoor scenarios. The core idea is to supervise depth estimation through view synthesis via rigid structure from motion (SfM)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Mai by substituting the depth model for Wang’s structure from motion model. Doing so would provide the predictable result of a depth estimate.
Regarding Claim 20, Mai teaches the method of claim 14. Mai does not explicitly teach the remaining limitations of Claim 20.
Wang teaches wherein providing the indicator includes associating the motion with an identified object of the surrounding objects according to a semantic model that identifies the surrounding objects and a pixel-wise association of the motion in relation to the identified object ([0036]: segment moving objects. Finally, since one or more methodology embodiments in this patent document decomposes static background and moving objects, the disclosed approach is also related to segmentation of moving objects from a given video. Current contemporary SOTA methods are dependent on supervision from human labels by adopting CNN image features or RNN temporal modeling),
wherein the depth model performs monocular depth estimation and is trained according to self-supervised structure-from-motion (SfM) training ([0018]: recently, for unsupervised single image depth estimation, impressive progress has been made to train a deep network taking only unlabeled samples as input and using 3D reconstruction for supervision, yielding even better depth estimation results than those of supervised methods in outdoor scenarios. The core idea is to supervise depth estimation through view synthesis via rigid structure from motion (SfM)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights