DETAILED ACTION
Response to Arguments
Applicant has amended claims 1, and 14; claims 1-20 are currently pending with claims 7-13 withdrawn from further consideration.
Applicant’s arguments filed 7/7/2026, with respect to the rejection(s) of claim(s) 1-4, 6, 14, and 17-19 under 102(a)(1) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yang (Yang Z, Wang P, Wang Y, Xu W, Nevatia R. Lego: Learning edge with geometry all at once by watching videos. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018 Jun 18 (pp. 225-234). IEEE.). Therefore this action is made FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 14, and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).) in view of Yang (Yang Z, Wang P, Wang Y, Xu W, Nevatia R. Lego: Learning edge with geometry all at once by watching videos. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018 Jun 18 (pp. 225-234). IEEE.).
Regarding claim 14, Ramamonjisoa discloses An apparatus comprising: at least one memory component; at least one processing device coupled to the at least one memory component, wherein the at least one processing device is configured to execute instructions stored in the at least one memory component; and (Ramamonjisoa Section 4.1 Implementation Details ¶1 -found on left hand column of p. 6; training and evaluation are performed on an NVIDIA GTX 1080 Ti GPU. Their code is also available on github (see footnote 1 on same page).) a monocular depth estimation (MDE) network configured to generate a depth map of an image, (Ramamonjisoa Section 5. Conclusion and Fig. 2; depth maps are created from a single RGB image. Fig. 2 shows the network architecture with depth in the center.) wherein the MDE network is trained using training data including edge data generated by a depth edge estimation (DEE) network (Ramamonjisoa Fig. 2, Section 3.2. Loss Function -found on p. 4 lefthand column, and Section 3.3. Supervision Terms Ld, Lc and Ln – Occluding contours prediction loss Lc – found on lefthand column p. 5, and 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5; A network is used to estimate occluding contours (depth edges) as can be seen in Fig. 2 on the top. Occluding contours are used in a training loss.)
Ramamonjisoa does not explicitly disclose wherein the DEE network is separately trained from the MDE network.
Yang, however, discloses wherein the DEE network is separately trained from the MDE network (Yang Section 5.4 Edge experiments -Experiment setup - ¶2; utilization of a pretrained edge detection network in calculating monocular depth is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of Ramamonjisoa with teachings of Yang by including a pre-trained edge detection decoder in order to help training of the full model converge.
Regarding claim 17, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above. They further disclose wherein the apparatus further comprises an augmented reality (AR) unit configured to display an AR object based on the depth map. (Ramamonjisoa Fig.1, and Section 5. Conclusion – found on p. 8 righthand column; a virtual Stanford rabbit is inserted into an RGB image using the depth map – see “Ours” row. Integrating objects to an augmented reality setting in real time is disclosed.)
Regarding claim 18, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above. They further disclose wherein the apparatus further comprises the DEE network. (Ramamonjisoa Fig. 2; The occluding contours branch is part of the network architecture.)
Regarding claim 19, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above. They further disclose wherein the apparatus further comprises an edge detection block (EDB) configured to generate predicted edge data based on the depth map. (Ramamonjisoa Fig. 2, Section 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5, and Section 5. Conclusion – found on p. 8 righthand column; The occluding contours branch is part of the network architecture. A depth-contours consensus term is utilized so the contour (edge data) and depth maps are based on each other. Contour predictions (edge detections) are by-products of the algorithm.)
Regarding claims 1 and 3, they are the corresponding method claims to claims 14 and 16 respectively and are rejected for similar reasons.
Regarding claim 2, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 1, as described above. They further disclose further comprising: displaying a boundary of an object based on the depth map. (Ramamonjisoa Fig. 7; contours can be seen with corresponding depth maps for each of the 3 example RGB images. Ramamonjisoa Fig. 2, Section 3.4 Consensus Terms Ldc and Ldn – Depth-contours consensus term – found on right hand column on p. 5, and Section 5. Conclusion – found on p. 8 righthand column; These contours are based on depth maps.)
Regarding claim 4, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 1, as described above. They further disclose further comprising: displaying an augmented reality (AR) object based on the depth map, wherein the AR object is partially occluded based on an object in the image. (Ramamonjisoa Fig.1, and Section 5. Conclusion – found on p. 8 righthand column; a virtual Stanford rabbit is inserted into an RGB image using the depth map – see “Ours” row. It can be seen that the rabbit is partially occluded by the doorway. Integrating objects to an augmented reality setting in real time is disclosed.)
Regarding claim 6, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 1, as described above. They further disclose wherein: the depth map comprises a depth estimation for a pixel of the image, (Ramamonjisoa Fig. 7; depth maps can be seen for each of the 3 example RGB images.) and the edge data includes a probability of an edge for a pixel of a training image. (Ramamonjisoa Section 3. Method ¶1 – found on p. 3 righthand column; the network is trained to predict occluding contour probabilities.)
Claim(s) 5, and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).) in view of Yang (Yang Z, Wang P, Wang Y, Xu W, Nevatia R. Lego: Learning edge with geometry all at once by watching videos. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018 Jun 18 (pp. 225-234). IEEE.), and Mousavian (Pub. No. US20200193630 A1).
Regarding claim 5, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 1, as described above.
The combination of Ramamonjisoa and Yang discloses indoor scenes obtained using a camera (Ramamonjisoa see Fig. 2 input), but not explicitly further comprising: capturing the image using a camera located in a same device as the MDE network.
Mousavian, however, discloses further comprising: capturing the image using a camera located in a same device as the MDE network. (Mousavian ¶15; a system with a camera to provide depth is disclosed. ¶70 the system can be combined into a single hardware device.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the method of the combination of Ramamonjisoa and Yang with teachings of Mousavian by including a camera to provide depth on the same system as the depth network in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Regarding claim 15, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above.
The combination of Ramamonjisoa and Yang discloses indoor scenes obtained using a camera (Ramamonjisoa see Fig. 2 input), but not explicitly wherein the apparatus further comprises a camera configured to obtain the image.
Mousavian, however, discloses wherein the apparatus further comprises a camera configured to obtain the image. (Mousavian ¶15; a system with a camera to provide depth is disclosed. ¶70 the system can be combined into a single hardware device.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of the combination of Ramamonjisoa and Yang with teachings of Mousavian by including a camera to provide depth on the same system as the depth network in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Regarding claim 16, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above.
The combination of Ramamonjisoa and Yang does not explicitly disclose wherein the apparatus further comprises a navigation unit configured to generate navigation information based on the depth map.
Mousavian, however, discloses wherein the apparatus further comprises a navigation unit configured to generate navigation information based on the depth map. (Mousavian ¶59; utilization of depth estimations in path planning or collision avoidance is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of the combination of Ramamonjisoa and Yang with teachings of Mousavian by utilizing the depth information to in path planning in order to utilize the algorithm in machine vision systems (Mousavian ¶15).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramamonjisoa (Ramamonjisoa M, Lepetit V. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0).) in view of Yang (Yang Z, Wang P, Wang Y, Xu W, Nevatia R. Lego: Learning edge with geometry all at once by watching videos. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018 Jun 18 (pp. 225-234). IEEE.), and Zhang (Y. Zhang, S. Song, E. Yumer, M. Savva, J.-Y. Lee, H. Jin, and T. Funkhouser. Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks. In Conference on Computer Vision and Pattern Recognition, 2017.).
Regarding claim 20, the combination of Ramamonjisoa and Yang discloses the claim limitations with regards to claim 14, as described above. They further disclose wherein the apparatus further comprises wherein the DEE network is trained based on the synthetic image.(Ramamonjisoa Section 4.1. Implementation Details – Datasets – found on p. 6 left hand column; initial training using the synthetic PBRS dataset is disclosed.)
The combination of Ramamonjisoa and Yang does not explicitly disclose an image generation component configured to generate a synthetic image.
Zhang, however, discloses an image generation component configured to generate a synthetic image. (Zhang Abstract; a synthetic data set is generated for training in computer vision tasks.
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the apparatus of the combination of Ramamonjisoa and Yang with teachings of Zhang by including generation of synthetic training data in order to customize training data to specific desired tasks and environments.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
