DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Invention I comprising claims 1-6 and 14-20 in the reply filed on 10/14/2025 is acknowledged. Therefore the restriction is made FINAL and Claims 7-13 are withdrawn from further consideration.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 07/18/2023 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) has/have been considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 6, 14, and 17-19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).).
Regarding claim 14, Ramamonjisoa discloses An apparatus comprising: at least one memory component; at least one processing device coupled to the at least one memory component, wherein the at least one processing device is configured to execute instructions stored in the at least one memory component; and (Ramamonjisoa Section 4.1 Implementation Details ¶1 -found on left hand column of p. 6; training and evaluation are performed on an NVIDIA GTX 1080 Ti GPU. Their code is also available on github (see footnote 1 on same page).) a monocular depth estimation (MDE) network configured to generate a depth map of an image, (Ramamonjisoa Section 5. Conclusion and Fig. 2; depth maps are created from a single RGB image. Fig. 2 shows the network architecture with depth in the center.) wherein the MDE network is trained using training data including edge data generated by a depth edge estimation (DEE) network. (Ramamonjisoa Fig. 2, Section 3.2. Loss Function -found on p. 4 lefthand column, and Section 3.3. Supervision Terms Ld, Lc and Ln – Occluding contours prediction loss Lc – found on lefthand column p. 5, and 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5; A network is used to estimate occluding contours (depth edges) as can be seen in Fig. 2 on the top. Occluding contours are used in a training loss.)
Regarding claim 17, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above. Ramamonjisoa further discloses wherein the apparatus further comprises an augmented reality (AR) unit configured to display an AR object based on the depth map. (Ramamonjisoa Fig.1, and Section 5. Conclusion – found on p. 8 righthand column; a virtual Stanford rabbit is inserted into an RGB image using the depth map – see “Ours” row. Integrating objects to an augmented reality setting in real time is disclosed.)
Regarding claim 18, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above. Ramamonjisoa further discloses wherein the apparatus further comprises the DEE network. (Ramamonjisoa Fig. 2; The occluding contours branch is part of the network architecture.)
Regarding claim 19, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above. Ramamonjisoa further discloses wherein the apparatus further comprises an edge detection block (EDB) configured to generate predicted edge data based on the depth map. (Ramamonjisoa Fig. 2, Section 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5, and Section 5. Conclusion – found on p. 8 righthand column; The occluding contours branch is part of the network architecture. A depth-contours consensus term is utilized so the contour (edge data) and depth maps are based on each other. Contour predictions (edge detections) are by-products of the algorithm.)
Regarding claims 1 and 3, they are the corresponding method claims to claims 14 and 16 respectively and are rejected for similar reasons.
Regarding claim 2, Ramamonjisoa discloses the claim limitations with regards to claim 1, as described above. Ramamonjisoa further discloses further comprising: displaying a boundary of an object based on the depth map. (Ramamonjisoa Fig. 7; contours can be seen with corresponding depth maps for each of the 3 example RGB images. Ramamonjisoa Fig. 2, Section 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5, and Section 5. Conclusion – found on p. 8 righthand column; These contours are based on depth maps.)
Regarding claim 4, Ramamonjisoa discloses the claim limitations with regards to claim 1, as described above. Ramamonjisoa further discloses further comprising: displaying an augmented reality (AR) object based on the depth map, wherein the AR object is partially occluded based on an object in the image. (Ramamonjisoa Fig.1, and Section 5. Conclusion – found on p. 8 righthand column; a virtual Stanford rabbit is inserted into an RGB image using the depth map – see “Ours” row. It can be seen that the rabbit is partially occluded by the doorway. Integrating objects to an augmented reality setting in real time is disclosed.)
Regarding claim 6, Ramamonjisoa discloses the claim limitations with regards to claim 1, as described above. Ramamonjisoa further discloses wherein: the depth map comprises a depth estimation for a pixel of the image, (Ramamonjisoa Fig. 7; depth maps can be seen for each of the 3 example RGB images.) and the edge data includes a probability of an edge for a pixel of a training image. (Ramamonjisoa Section 3. Method ¶1 – found on p. 3 righthand column; the network is trained to predict occluding contour probabilities.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5, and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).) in view of Mousavian (Pub. No. US20200193630 A1).
Regarding claim 5, Ramamonjisoa discloses the claim limitations with regards to claim 1, as described above.
Ramamonjisoa discloses indoor scenes obtained using a camera (see Fig. 2 input), but not explicitly further comprising: capturing the image using a camera located in a same device as the MDE network.
Mousavian, however, discloses further comprising: capturing the image using a camera located in a same device as the MDE network. (Mousavian ¶15; a system with a camera to provide depth is disclosed. ¶70 the system can be combined into a single hardware device.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the method of Ramamonjisoa with teachings of Mousavian by including a camera to provide depth on the same system as the depth network in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Regarding claim 15, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above.
Ramamonjisoa discloses indoor scenes obtained using a camera (see Fig. 2 input), but not explicitly wherein the apparatus further comprises a camera configured to obtain the image.
Mousavian, however, discloses wherein the apparatus further comprises a camera configured to obtain the image. (Mousavian ¶15; a system with a camera to provide depth is disclosed. ¶70 the system can be combined into a single hardware device.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of Ramamonjisoa with teachings of Mousavian by including a camera to provide depth on the same system as the depth network in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Regarding claim 16, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above.
Ramamonjisoa does not explicitly disclose wherein the apparatus further comprises a navigation unit configured to generate navigation information based on the depth map.
Mousavian, however, discloses wherein the apparatus further comprises a navigation unit configured to generate navigation information based on the depth map. (Mousavian ¶59; utilization of depth estimations in path planning or collision avoidance is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of Ramamonjisoa with teachings of Mousavian by utilizing the depth information to in path planning in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).) in view of Zhang (Y. Zhang, S. Song, E. Yumer, M. Savva, J.-Y. Lee, H. Jin, and T. Funkhouser. Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks. In Conference on Computer Vision and Pattern Recognition, 2017.).
Regarding claim 20, Ramamonjisoa discloses the claim limitations with regards to claim 14, as described above. Ramamonjisoa further discloses wherein the apparatus further comprises wherein the DEE network is trained based on the synthetic image.(Ramamonjisoa Section 4.1. Implementation Details – Datasets – found on p. 6 left hand column; initial training using the synthetic PBRS dataset is disclosed.)
Ramamonjisoa does not explicitly disclose an image generation component configured to generate a synthetic image.
Zhang, however, discloses an image generation component configured to generate a synthetic image. (Zhang Abstract; a synthetic data set is generated for training in computer vision tasks.
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of Ramamonjisoa with teachings of Zhang by including generation of synthetic training data in order to customize training data to specific desired tasks and environments.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights