Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on May 7, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed 6/23/2026 have been fully considered but they are not persuasive. Applicant argues that the prior art Gupta fails to disclose an error in a relative pose between images. However, Examiner disagrees because Gupta discloses a disparity map/depth calculation and also Step S505 may correspond to performing a cost volume calculation, wherein a matching cost volume is computed based on the differences in distances between at least some of the vectors generated for the at least two colour images (Gupta et. al. col 9, lines 5-10). This cost volume computation step is the foundation for computing error, which in stereo matching refers to the disparity error in relative pose between images. If the correct disparity is not the one with the lowest cost, the matching is wrong, which indicates an error. Thus, it would have been obvious to one skilled in the art to have derived the error in a relative pose between images from the cost volume computation step. Therefore, the prior art Gupta can still be used in the rejection of the claims. Claims 1-20 are still rejected with the same grounds of rejection as the prior non-final rejection given no amendments were filed in response to the office action.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 10, 17 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gupta et. al. (United Kingdom Patent Application GB 2585197 A).
Regarding claim 1, Gupta et. al. discloses a computer-implemented method comprising: receiving a pair of overlapping images of a scene (Gupta et. al. GB 2585197 A, Abstract, Figure 5, stereo-matching, overlapping viewpoint); calculating a relative pose between the pair of overlapping images (Gupta et. al., col. 4, lines 5-11, where at step S102, the extrinsics (position and/or orientation) and intrinsics (focal length) associated with the at least two images are obtained); applying a two-view geometry model to predict an error in the relative pose between the pair of overlapping images (Gupta et. al., col 10, lines 15-23, the disparity may be determined by calculating the difference in pixel positions for that pair of feature vectors. Then, the depth determined may be used to generate a point cloud which is then used to generate a triangular or polygonal mesh with the region of each image corresponding to the overlapping fields of view); and providing content for display at a client device accounting for the error (Gupta et. al. col 10, lines 20-29, a viewer may be able to change their perspective of the reconstructed objects, such that the objects are displayed from the viewer’s new perspective).
Regarding claim 10, which recites a computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor of a client device executed the method steps of claim 1. Thus, the rejection analysis of claim 1 is fully incorporated herein (Gupta et. al.; page 14, lines 14-16).
Regarding claim 17, which recites a processor based client device comprising computer instructions of claim 10 and when implemented carries out the method steps of claim 1. Thus, the rejection analysis of claim 1 is fully incorporated herein (Gupta et. al. Figures 5-6, page 16, lines 23-28, the client device is a “game machine”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-9, 11-16, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupta et. al. (United Kingdom Patent GB 2585197 A) in view of Sinha et. al. (United States Patent Application Publication US 2023/0154032 A1).
Regarding claims 2, 11, and 18, Gupta et. al. discloses the method of claim 1, the computer program product of claim 10, and the client device of claim 17. However, Gupta et. al. fails to disclose wherein the two-view geometry model uses an epipolar attention mechanism to predict the error in the relative pose between the pair of overlapping images.
Sinha et. al. teaches wherein the two-view geometry model uses an epipolar attention mechanism to predict the error in the relative pose between the pair of overlapping images (Sinha et. al. US 2023/0154032 A1, [0049]-[0050] two different machine learning models are used in parallel, where pairs of the predicted 2D locations and the predicted 3D bearing vectors which relate to a same one of the 3D scene landmarks are determined. [0088] The estimated pose was evaluated using the rotational and translational error as an equation).
PNG
media_image1.png
172
580
media_image1.png
Greyscale
This feature is a critical piece to the claimed invention because it allows for quantified pose estimation using rotational and translational errors. Thus, it would have been obvious for one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and Sinha et. al. so that the method includes the estimated pose calculation of Sinha et. al.
Regarding claims 3, 12, and 19, Gupta et. al. discloses the method of claim 1, the computer program product of claim 11, and the client device of claim 17. However, Gupta et. al. fails to disclose, further comprising: computing feature maps for each of the pair of overlapping images; and using a self-attention layer and a cross-attention layer to form transformed feature maps for each of the feature maps.
Sinha et. al. teaches further comprising: computing feature maps for each of the pair of overlapping images; and using a self-attention layer and a cross-attention layer to form transformed feature maps for each of the feature maps (Sinha et. al. US 2023/0154032 A1, [0070]-[0071], pixel likelihood maps or heatmaps are generated using a convolutional neural network (CNN) model). Feature maps are an important element to the claimed invention because it shows where the 2D images are projected for the 3D image in terms of landmarks. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and Sinha et. al. so that the method of Gupta et. al. includes the feature maps of Sinha et. al.
Regarding claims 4, and 13 Gupta et. al. discloses the method of claim 3, and the computer program product of claim 11. However, Gupta et. al. fails to disclose wherein the operations further comprising: applying cross-attention along epipolar lines to embed a fundamental matrix hypothesis into corresponding transformed feature maps to form final feature maps for the pair of overlapping images.
Sinha et. al. teaches wherein the operations further comprising: applying cross-attention along epipolar lines to embed a fundamental matrix hypothesis into corresponding transformed feature maps to form final feature maps for the pair of overlapping images (Sinha et. al. [0075], Figure 7 schematic diagram of a machine learning model for predicting 3D bearing vectors). This is an important component to the claimed invention because the fundamental matrix hypothesis provides an equation for the trained machine learning model. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and Sinha et. al. so that the method of Gupta et. al. includes the operations of Sinha et. al.
Regarding claims 5, and 14 Gupta et. al. discloses the method of claim 4, and the computer program product of claim 13. However, Gupta et. al. fails to disclose wherein applying cross-attention along the epipolar lines to embed the fundamental matrix hypothesis into corresponding transformed feature maps is performed such that a resolution of the final feature maps is less than a resolution of the transformed feature maps.
Sinha et. al. teaches wherein applying cross-attention along the epipolar lines to embed the fundamental matrix hypothesis into corresponding transformed feature maps is performed such that a resolution of the final feature maps is less than a resolution of the transformed feature maps (Sinha et. al. [0073] the training model generates heatmaps where the final image is a quarter and half of the resolution W and H of ground truth image). This is an important feature of the claimed invention because the lower resolution image produced saves computational energy. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and Sinha et. al. so that the method of Gupta et. al. includes the operations of Sinha et. al.
Regarding claims 6, 15, and 20, Gupta et. al. discloses the method of claim 4, the computer program product of claim 13, and the client device of claim 17. However, Gupta et. al. fails to disclose wherein applying the two-view geometry model to predict the error in the relative pose between the pair of overlapping images, comprises: predicting an angular translation error and a rotation error associated with the fundamental matrix hypothesis using the final feature maps.
Sinha et. al. teaches wherein applying the two-view geometry model to predict the error in the relative pose between the pair of overlapping images, comprises: predicting an angular translation error and a rotation error associated with the fundamental matrix hypothesis using the final feature maps (Sinha et. al. US 2023/0154032 A1, [0049]-[0050] two different machine learning models are used in parallel, where pairs of the predicted 2D locations and the predicted 3D bearing vectors which relate to a same one of the 3D scene landmarks are determined. [0088] The estimated pose was evaluated using the rotational and translational error as an equation). This is an important feature of the claimed invention because it quantifies the error in relative poses between each camera. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and the teachings of Sinha et. al. so that the method of Gupta et. al. includes the error calculations of Sinha et. al.
Regarding claims 7, and 16, Gupta et. al. discloses the method of claim 6, and the computer program product of claim 15. However, Gupta et. al. fails to disclose wherein the two-view geometry model does not use correspondences to predict the error in the relative pose between the pair of overlapping images.
Sinha et. al. teaches wherein the two-view geometry model does not use correspondences to predict the error in the relative pose between the pair of overlapping images (Sinha et. al. [0078] SLD does not require predict 3D from the image (does not use correspondences)). This is an important feature of the claimed invention because not using correspondences achieves higher generalization than directly predicting pose from an image. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and the teachings of Sinha et. al. so that the method does not use correspondences.
Regarding claim 8, Gupta et. al. discloses the method of claim 1. However, Gupta et. al. fails to disclose further comprising: identifying a number of correspondences between the pair of overlapping images; and electing to use the two-view geometry model responsive to the number of correspondences being below a threshold.
Sinha et. al. teaches further comprising: identifying a number of correspondences between the pair of overlapping images; and electing to use the two-view geometry model responsive to the number of correspondences being below a threshold (Sinha et. al. [0074] the detection of a landmark requires exceeding a threshold value, [0080]-[0082] Landmark Selection Algorithm). This is an important aspect of the claimed invention because the threshold determines which landmarks are used in generating the feature maps. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and the teachings of Sinha et. al. so that the method of Gupta et. al. includes the operations of Sinha et. al.
Regarding claim 9, Gupta et. al. discloses the method of claim 1. However, Gupta et. al. fails to disclose wherein applying the two-view geometry model comprises: generating a pool of fundamental matrix hypotheses; reducing the pool of fundamental matrix hypotheses based on rankings of the fundamental matrix hypotheses; applying the two-view geometry model to calculate a hypothesis error for each fundamental matrix hypothesis in the reduced pool of fundamental matrix hypotheses; and using a fundamental matrix hypothesis having a lowest hypothesis error to determine the relative pose between the pair of overlapping images, wherein the error in the lowest hypothesis error.
Sinha et. al. teaches wherein applying the two-view geometry model comprises: generating a pool of fundamental matrix hypotheses; reducing the pool of fundamental matrix hypotheses based on rankings of the fundamental matrix hypotheses; applying the two-view geometry model to calculate a hypothesis error for each fundamental matrix hypothesis in the reduced pool of fundamental matrix hypotheses; and using a fundamental matrix hypothesis having a lowest hypothesis error to determine the relative pose between the pair of overlapping images, wherein the error in the lowest hypothesis error (Sinha et. al. [0104]-[0108] method for camera localization within a scene using a machine learning model to output a plurality of predictions on 2D location in the image to depict a 3D scene landmark or a 3D bearing vector).
This is an important aspect of the claimed invention because a pool of functions is needed for the collection of images to be trained. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Gupta et. al. and the teachings of Sinha et. al. so that the method of Gupta et. al. includes the operations of Sinha et. al.
PNG
media_image2.png
439
568
media_image2.png
Greyscale
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 July 15, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights