CTFR 18/533,422 CTFR 99729 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. The Applicant’s Remarks filed 18 February 2026 have been received and considered. The 112(d) rejection in the non-final office action mailed 19 November 2025 has been withdrawn . Claims 1 – 20 are pending . Claims 1 – 20 , all of the pending claims in this application, have been rejected . Response to Applicant’s Remarks Applicant’s remarks were filed 18 February 2026 regarding the prior art the Examiner used to reject independent claims 1 . Applicant does not believe the art teaches the limitations of claim 1 and, therefore, the Applicant did not amend. On pages 9 – 10 of the Remarks, Applicant contends that the applied art does not teach or suggest determining correspondence based on the approximate pose, as claimed. In support of this argument, Applicant asserts that Shotton teaches the opposite – namely, predicting correspondences to determine camera pose. The Examiner respectfully disagrees with this characterization of the reference. As shown in the pseudocode of algorithm 1, Shotton discloses initial hypothesized poses “Hk”. Further, in line 9 of the pseudocode, Shotton discloses that the correspondence prediction mentioned by the Applicant is dependent on the initial hypothesized poses Hk. Thus, Shotton does indeed teach determining the correspondences based on the approximate pose, contrary to Applicant’s assertions. On pages 10 – 11 of the Remarks, Applicant argues that Shotton does not detect structural components. In support of this argument, Applicant alleges that Shotton operates at a pixel level, asserts that a pixel is not a structural component in the context of the present application, and provides examples of structural components such as “Shelf A” or “Light B”. The Examiner respectfully submits that Shotton teaches the broadest reasonable interpretation of the claimed structural components. Applicant has not provided any evidence to support the assertion that a pixel cannot be interpreted as a structural component. As shown in Fig. 1 of Shotton, the images that are analyzed are full of structural items, such as chairs, desks, and bookshelves. The pixels in the image are components that represent these structural items. As such, Shotton’s pixels of structural items reads on the broadest reasonable interpretation of the term in question. Nothing in the claim language precludes the above interpretation. If Applicant believes that the structural components of the present invention are different from Shotton’s pixels of structural items, the Examiner recommends amending the claim language to more clearly highlight this distinction . On page 11 of the Remarks, Applicant argues that an ordinarily skilled artisan would not have looked to Alkhatib’s sparse descriptor matching to modify Shotton’s dense regression forest, as doing so would allegedly undermine the very premis of Shotton’s coordinate regression approach. In support of this assertion, Applicant contends Shotton explicitly seeks to avoid feature matching, citing section 1 of the reference. The Examiner respectfully maintains that an ordinarily skilled artisan would indeed have combined the references in the manner proposed in the rejection, despite this teaching. Initially, the Examiner notes that Applicant has mischaracterized the proposed modification. The rejection does not rely on modifying Shotton’s regression forest itself, as Applicant implies. Rather, Alkhatib’s feature matching process for identifying point correspondences would have supplemented Shotton’s process for identifying point correspondences. Indeed, both references disclose identifying point correspondences in the service of pose estimation (i.e. Shotton’s Algorithm 1 Pseudocode and Section 3 of Alkhatib). Thus, if an ordinarily skilled artisan was interested in verifying predictions of Shotton’s regression forest, Alkhatib’s method would have provided a strong alternative method of doing so. Therefore, the Examiner maintains that the combined teaches of Shotton, in view of Alkhatib, do indeed teach the original features of the claim, as detailed below. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1 – 4 , 8 , 10 , 12 – 13 , and 15 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature " Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images" to Shotton et al. (hereinafter Shotton) in view of Non-Patent Literature “Camera pose estimation based on structure from motion" to Alkhatib et al. (hereinafter Alkhatib) . Claim 1 Regarding claim 1, Shotton teaches an apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: process an image captured by a camera device to determine one or more pixel coordinates of one or more structural components depicted in the image ("The regression forest described in the previous section is capable of associating scene coordinates with any 2D image pixel.", Section 3. Camera Pose Optimization) ; determine an approximate pose of the camera device at a time the image was captured (Figure 2 - Camera pose estimation; "The algorithm maintains a set of inlier pixels for each of several camera pose hypotheses. Bottom right: The hypothesis with the lowest energy (highest number of inliers) is chosen as the final inferred pose (shown as the blue frustum; the ground truth is shown in red).", Section 3. Camera Pose Optimization) ; PNG media_image1.png 397 339 media_image1.png Greyscale determine a correspondence of the one or more structural components to one or more known components based on the approximate pose (Figure 1; "A 3D representation of a scene’s shared world coordinate frame, with overlaid ground truth camera frusta for the images below. The color visualization maps scene coordinates to the RGB cube. A scene coordinate regression forest (SCoRe Forest) is trained to infer the scene coordinates at any image pixel. (Bottom) Three test frames: the input RGB and depth images; the ground truth scene coordinate pixel labels; and the inliers inferred by the SCoRe Forest after camera pose optimization. For this visualization we show all inlier pixels, but note that the optimization algorithm only actually evaluates the forest at a much sparser set of pixels. Fig. 7 shows example inferred camera poses.", Introduction) ; and PNG media_image2.png 395 345 media_image2.png Greyscale compute a pose estimation of the camera device based on the one or more location coordinates and the one or more pixel coordinates ("Our main contribution is the scene coordinate regression forest (SCoRe Forest). As illustrated in Fig. 1, the forest is trained to directly predict correspondences from any image pixel to points in the scene’s 3D world coordinate frame…The algorithm assumes only a single RGB-D image as input. We optimize an energy function that measures the number of pixels for which the SCoRe Forest predictions agree with a given pose hypothesis.", Introduction) . Shotton does not teach query a database for one or more location coordinates of the one or more known components based on the correspondence. However, Alkhatib teaches query a database for one or more location coordinates of the one or more known components based on the correspondence ("During this stage, the 3D model is constructed and stored appropriately in the database. Initially, several frames are collected from the camera so that these frames are overlapped, then features are extracted and described for each frame. Matching (2D / 2D) is applied between the features in each of the frames. Each point of matching pairs of points is projection to the same 3D point, thus the three coordinates of the location of each feature (point) is concluded. At the end of this stage, each 3D point is associate with the corresponding descriptor and store it in structure (3D points and descriptors). This structure expresses the 3D map of scene with its features. Navigation stage is performed inreal-time. Initially acquired query image, then the keypoints of this image are extracted and described. Then an operation 2D / 3D matching is performed between features in the query 2D image and features in 3D map structure.", Section 3 - Proposed method) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Shotton to incorporate query a database for the known coordinates of corresponding components, as disclosed by Alkhatib . The suggestion/motivation for doing so would have been to be able to compare detected objects from a recently captured image with the those of a previously image that was stored from which all objects had been verified to determine if the objects in both image correspond with each other so that certain objects may be avoided, like robots, stationary targets, etc.. Claim 2 Regarding claim 2, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton does not teach wherein the camera device is equipped on a mobile agent, and wherein the apparatus is further caused to perform: localize the mobile agent based on the pose estimation. However, Alkhatib further teaches wherein the camera device is equipped on a mobile agent, and wherein the apparatus is further caused to perform: localize the mobile agent based on the pose estimation ("This work is devoted to estimate the position and the orientation of the robot (this robot carries a camera) based on the images captured by this camera.", Introduction; "In this work, a new classification for vision-based localization methods is presented and a new algorithm is proposed to estimate location in indoor environment using SFM. This method approved to determine a location of the camera 6 DOF (three translations and three rotations).", Conclusion) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate the orientation of a robot based on images captured by a camera, as disclosed by Alkhatib . The suggestion/motivation for doing so would have been to be able to allow an autonomous robot to be able to navigate a facility based on camera information and images captured by said camera(s). Claim 3 Regarding claim 3, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton further teaches wherein the one or more location coordinates, the pose estimation, or a combination thereof are in a world frame of reference (Rejected as applied to claim 1) . Claim 4 Regarding claim 4, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton further teaches wherein the one or more structural components include one or more shelves, one or more containers, one or more door or window frames, one or more lights, one or more structural beams, or a combination thereof (Figure 1) , where shelves and other structures are present in the various images . Claim 8 Regarding claim 8, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton further teaches wherein the approximate pose is based on another localization modality (Rejected as applied to claim 1) , where SAVloc is not being used . Claim 10 Regarding claim 10, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton further teaches wherein the one or more structural components include one or more vertical structures, one or more horizontal structures, or a combination thereof (Figure 1) , where the shelves and desks are present in the images . Claim 12 Regarding claim 12, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton further teaches wherein the one or more known components, the one or more location coordinates, or a combination thereof are determined from blueprint data (Rejected as applied to claim 1). Shotton begins with a known 3D environment (Abstract, Introduction first paragraph.) Shotton does not say how the known data was acquired. In the context of the apparatus of Claim 12, this is simply a model of what the space is supposed to look like, and in terms of the data stored would be the same as if it originated from a blueprint. It would be data representing the 3D dimensions of the space, and the data itself would not be identifiably different if it originated from a blueprint or not. The claimed apparatus does not appear to have a feature the converts blueprints to data models; see paragraphs [0062]-[0066] and especially [0070] of the specification. Claim 13 Regarding claim 13, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Shotton does not teach wherein the estimation of the pose of the camera device, the computed pose of the camera device, or a combination thereof is determined with respect to six degrees of freedom However, Alkhatib further teaches wherein the estimation of the pose of the camera device, the computed pose of the camera device, or a combination thereof is determined with respect to six degrees of freedom ("In this work, a new classification for vision-based localization methods is presented and a new algorithm is proposed to estimate location in indoor environment using SFM. This method approved to determine a location of the camera 6 DOF (three translations and three rotations).", Section 5 - Conclusions and Future Work) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate a localization method determined with respect to the camera’s 6 DOF, as disclosed by Alkhatib . The suggestion/motivation for doing so would have been to be able to improve the accuracy of autonomous robot movement in an indoor environment, as disclosed by Alkhatib . Claim 15 , an independent method claim, is rejected for the same reasons as applied to claim 1 . Claims 16 and 17 , dependent on claim 15 , are rejected for the same reasons as applied to the above claims. Claim 18 , an independent non-transitory computer readable storage medium claim, is rejected for the same reasons as applied to claim 1 . Claims 19 and 20 , dependent on claim 18 , are rejected for the same reasons as applied to the above claims . 07-21-aia AIA Claim s 5 – 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature " Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images" to Shotton et al. (hereinafter Shotton) in view of Non-Patent Literature “Camera pose estimation based on structure from motion" to Alkhatib et al. (hereinafter Alkhatib) in further view of Non-Patent Literature “Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments” to Almalioglu et al. (hereinafter Almalioglu) . Claim 5 Regarding claim 5, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Neither Shotton , or Alkhatib , or the combination teach wherein the approximate pose is based on a prior pose coupled with a relative pose change up to the time the image was captured. However, Almalioglu teaches wherein the approximate pose is based on a prior pose coupled with a relative pose change up to the time the image was captured ("The second network shown in the bottom of Fig. 1 tries to estimate relative pose p ∈ SE(3) introduced by motion fields across frames.", Section II. UNSUPERVISED DEPTH AND POSE ESTIMATION: Part B - b) Pose Network) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate estimating poses based on the image frames captured during robot traversal, as disclosed by Almalioglu . The suggestion/motivation for doing so would have been to be able to improve the functionality of a robot as it traverses various extreme environments, as disclosed by Almalioglu . Claim 6 Regarding claim 6, dependent on claim 5 , Shotton , in view of Alkhatib and Almalioglu , teaches the invention as claimed in claim 5 . Neither Shotton , or Alkhatib , or the combination teach wherein the relative pose change is obtained from odometry. However, Almalioglu further teaches wherein the relative pose change is obtained from odometry (Rejected as applied to claim 5) , wherein image-based odometry (also known as visual odometry) in robots is based on analyzing sequences of images captured by an onboard camera (or cameras) as the robot traverses an environment . Claim 14 Regarding claim 14, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Neither Shotton , or Alkhatib , or the combination teach wherein the one or more structural components include one or more repetitive structures. However, Almalioglu teaches wherein the one or more structural components include one or more repetitive structures ("The proposed architecture is based on unsupervised deep learning to learn ego-motion and depth from monocular image sequences jointly. The raw RGB sequences, consisting of a target and source views, are stacked together to form an input batch to the multi-view pose estimation and depth recovery modules. The motion-prediction network predicts a motion of every pixel with respect to the background and a residual translation field to account for moving objects. In parallel, a second network generates a depth map of the target view. The view reconstruction module reconstructs the target image using the predicted depth map, estimated 6- DoF camera pose, and nearby colour values from source images. In this architecture, a) we impose scale-consistency across consecutive frames through a geometry consistent loss function; b) we estimate occlusions geometrically, based on the estimated depth maps to apply this loss only in non-occluded pixels; c) we regularize motion fields based on residual translations that indicate which pixels might belong to moving objects; and d) we include other stateof-the-art loss functions to handle dissimilarity or edgeaware smoothness in a total loss function.", Section II. UNSUPERVISED DEPTH AND POSE ESTIMATION: Part A - Architecture Overview) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate capturing consecutive images for continued scale consistency of detected targets while a robot traverses an environment, as disclosed by Almalioglu . The suggestion/motivation for doing so would have been to maintain an idea of potential obstacles while moving, as disclosed by Almalioglu . 07-21-aia AIA Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature " Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images" to Shotton et al. (hereinafter Shotton) in view of Non-Patent Literature “Camera pose estimation based on structure from motion" to Alkhatib et al. (hereinafter Alkhatib) in further view of Non-Patent Literature “Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments” to Almalioglu et al. (hereinafter Almalioglu) in further view of US Patent No. 11189049 B1 to Chakravarty et al. (hereinafter Chakravarty) . Claim 7 Regarding claim 7, dependent on claim 6 , Shotton , in view of Alkhatib and Almalioglu , teaches the invention as claimed in claim 6 . Neither Shotton , or Alkhatib , or Almalioglu , or the combination teach process another image captured at another time to determine one or more other pixel coordinates of the one or more structural components; wherein the odometry is based on a change between the one or more pixel coordinates and the one or more other pixel coordinates. However, Chakravarty teaches process another image captured at another time to determine one or more other pixel coordinates of the one or more structural components ("Visual odometry is a known technique for determining six DoF data from a sequential series of images. Visual odometery can be determined by training a VAE to input stereo pairs of images and outputting six DoF data. The VAE determines corresponding feature points in sequential images and calculates the change in location of the sensor between images. A six DoF pose for the camera can be determined by triangulating two or more sets of feature points to determine translation and rotation to determine a frame of reference for the sensor in global coordinates.", Column 10, lines 61 - 67; Column 11, lines 1 - 3) ; wherein the odometry is based on a change between the one or more pixel coordinates and the one or more other pixel coordinates ("FIG. 4. is a diagram of a stereo point cloud image 402 generated from a pair of stereo images 302, 304. While the pixels values in a pair of stereo images 302, 304 correspond to the amount of light received by the sensors, in a stereo point cloud image 402 the value of the pixels correspond to distances from the point corresponding to the pixel to the sensor. A stereo point cloud image 402 can be constructed from a pair of stereo images 302, 304 based on stereo disparity. Stereo disparity is defined as the difference in corresponding feature point locations in a pair of stereo images 302, 304. Corresponding feature points are defined as locations in the pair of stereo images 302, 304 that share similar pixel values including regions around the locations. For example, corners, edges and textures in the pair of stereo images 302, 304 can be corresponding feature points. The feature points can be determined by known machine vision techniques which determine feature points by processing regions in images to find pixel locations that can be defined by patterns of abrupt changes in pixel values, for example edges and corners and textures. Patterns of pixel values around feature points can be compared between pairs of stereo images to identify corresponding feature points that occur in both images. The difference in location with respect to the array of image points can be used to measure stereo disparity. ", Column 8, lines 19 - 43) . PNG media_image3.png 297 579 media_image3.png Greyscale It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib and Almalioglu , to incorporate comparing the pixel coordinates of corresponding objects taken from images with varying timestamps, as disclosed by Chakravarty . The suggestion/motivation for doing so would have been to determine object distance from sensors in the vicinity of host while traversing, as disclosed by Chakravarty . 07-21-aia AIA Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature " Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images" to Shotton et al. (hereinafter Shotton) in view of Non-Patent Literature “Camera pose estimation based on structure from motion" to Alkhatib et al. (hereinafter Alkhatib) in further view of Non-Patent Literature "Robust camera pose and scene structure analysis for service robotics" to Grigorescu et al. (hereinafter Grigorescu) in further view of US Patent No. 11189049 B1 to Chakravarty et al. (hereinafter Chakravarty) . Claim 9 Regarding claim 9, dependent on claim 1 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 1 . Neither Shotton , or Alkhatib , or the combination teach wherein the identification of the one or more structural components is determined by causing the apparatus to further perform: obtain one or more candidate objects from the database based on the approximate pose; cause a projection of the one or more candidate objects into an image plane of the image; wherein the identification of the one or more structural components is based on a closest object of the one or more candidate objects to the one or more pixel coordinates. However, Grigorescu teaches wherein the identification of the one or more structural components is determined by causing the apparatus to further perform: obtain one or more candidate objects from the database based on the approximate pose ("The evaluation of the overall machine visual system has been performed with respect to the real 3D poses of the objects of interest. The real 3D positions and orientations of the objects of interest were manually determined using the following setup. On the imaged scene, a visual marker, considered to be the ground truth information, was installed in such a way that the poses of the objects could be easily measured with respect to the marker. The 3D pose of the marker was detected using the ARToolKit library which provides subpixel accuracy estimation of the marker’s location with an average error of [28]. By calculating the marker’s 3D pose, a ground truth reference value for camera position and orientation estimation could be obtained using the inverse of the marker’s pose matrix. Further, the positions of the imaged objects, as well as the camera pose, were calculated using the proposed system which includes the feedback mechanisms for depth estimation. Both results, that is camera and objects poses, were compared to the ground truth data provided by the ARToolKit marker.", Section 6.2: Evaluation of the overall vision architecture) ; and cause a projection of the one or more candidate objects into an image plane of the image ("The model of the stereo camera used in sensing the robot’s environment is illustrated in Fig. 4 . A real world point represented in homogeneous coordinates projected onto the image planes of a stereo camera as the homogeneous 2D image points: where PL and PR have the 2D coordinates (XL,YL) and (XR,YR) projected onto the left and right images, respectively. The and 2D image positions are given by the intersection with the image plane of the line connecting point in world coordinates with the optical centers OL and OR of both cameras, as shown in Fig. 4 . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate obtaining targets from a database and projecting said targets into an image plane of the image, as disclosed by Grigorescu . The suggestion/motivation for doing so would have been to project known objects into a recently captured image for comparison purposes so that more accurate traversal of an environment can be accomplished. Neither Shotton , or Alkhatib , or Grigorescu , or the combination teach wherein the identification of the one or more structural components is based on a closest object of the one or more candidate objects to the one or more pixel coordinates. However, Chakravarty teaches wherein the identification of the one or more structural components is based on a closest object of the one or more candidate objects to the one or more pixel coordinates ("In addition to distances, the pixels of the stereo point cloud image 402 can be labeled according to the objects they correspond to. One or more of the RGB images included in the stereo image pairs 302, 304 can be input to a convolutional neural network (CNN) that has been trained to segment images. Image segmentation is a machine vision technique that labels objects in image data. The CNN can label objects in an input image and then the labels can be transferred to point cloud data. In stereo point cloud image 402, objects corresponding to a roadway, vehicles, trees and buildings adjacent to the roadway have been labeled to identify the regions of pixels in the stereo point cloud image 402 corresponding to the labeled objects. The CNN can be trained by labeling a plurality of RGB images manually to create ground truth images. The RGB images can be labeled by humans using image processing software to label regions of the images that correspond to objects as defined above. Point cloud image 402 has been processed using the node system 500 described in relation to FIG. 5 to form a semantic point cloud 512 where objects and regions are labeled. For example, point cloud image includes labeled regions corresponding to a roadways, sidewalks, vehicles, pedestrians, buildings, foliage, etc.", Column 9, lines 31 - 53) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib and Grigorescu , to incorporate labeling based on the pixels in the stereo point cloud image, as disclosed by Chakravarty . The suggestion/motivation for doing so would have been to properly identify objects while moving to avoid potential accidents or injuries to others . 07-21-aia AIA Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature " Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images" to Shotton et al. (hereinafter Shotton) in view of Non-Patent Literature “Camera pose estimation based on structure from motion" to Alkhatib et al. (hereinafter Alkhatib) in further view of Non-Patent Literature "Using vanishing points for camera calibration and coarse 3D reconstruction from a single image" to Guillou et al. (hereinafter Guillou) . Claim 11 Regarding claim 11, dependent on claim 10 , Shotton , in view of Alkhatib , teaches the invention as claimed in claim 10 . Neither Shotton , or Alkhatib , or the combination teach wherein the one or more vertical structures and the one or more horizontal structures intersect to form a rectangular area, and wherein the pose estimation is determined based on one or more vanishing points of the rectangular area in an image plane of the image. However, Guillou teaches wherein the one or more vertical structures and the one or more horizontal structures intersect to form a rectangular area, and wherein the pose estimation is determined based on one or more vanishing points of the rectangular area in an image plane of the image (Figures 9, 10, and 11; As the vanishing points are determined manually, one has to compute them with precision. For this reason, the resolution of the single image used must be high. Note that vanishing points can be extracted automatically [25, 26], and our method is not limited to pose estimation since it determines the focal length . Moreover, as we are interested in coarse 3D reconstruction for walkthrough, we can neglect camera distortions and assume that the principle point is close to the image center", Section 3.5 Discussion) . PNG media_image4.png 327 422 media_image4.png Greyscale PNG media_image5.png 304 652 media_image5.png Greyscale It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Shotton , in view of Alkhatib , to incorporate determining pose estimation based on vanishing points, as disclosed by Guillou . The suggestion/motivation for doing so would have been to utilize the vanishing points and pose estimation to generate a 3D reconstruction from a single image, as disclosed by Guillou . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Brahmbhatt, Samarth, et al. "Geometry-aware learning of maps for camera localization." Proceedings of the IEEE conference on computer vision and pattern recognition . 2018. https://openaccess.thecvf.com/content_cvpr_2018/papers/Brahmbhatt_Geometry-Aware_Learning_of_CVPR_2018_paper.pdf J. R. Rambach, A. Tewari, A. Pagani and D. Stricker, "Learning to Fuse: A Deep Learning Approach to Visual-Inertial Camera Pose Estimation," 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) , Merida, Mexico, 2016, pp. 71-76, doi: 10.1109/ISMAR.2016.19. https://ieeexplore.ieee.org/abstract/document/7781768 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7781768&tag=1 Kendall, Alex, Matthew Grimes, and Roberto Cipolla. "Posenet: A convolutional network for real-time 6-dof camera relocalization." Proceedings of the IEEE international conference on computer vision . 2015. https://openaccess.thecvf.com/content_iccv_2015/papers/Kendall_PoseNet_A_Convolutional_ICCV_2015_paper.pdf Klein, Georg, and David Murray. "Improving the agility of keyframe-based SLAM." European conference on computer vision . Berlin, Heidelberg: Springer Berlin Heidelberg, 2008. https://link.springer.com/chapter/10.1007/978-3-540-88688-4_59 Gee, Andrew P., and Walterio W. Mayol-Cuevas. "6D Relocalisation for RGBD Cameras Using Synthetic View Regression." BMVC . Vol. 1. 2012. https://www.researchgate.net/profile/WalterioMayolCuevas/publication/255565731_6D_Relocalisation_for_RGBD_Cameras_Using_Synthetic_View_Regression/links/606201f492851cd8ce760899/6D-Relocalisation-for-RGBD-Cameras-Using-Synthetic-View-Regression.pdf Oe, Motoko, Tomokazu Sato, and Naokazu Yokoya. "Estimating camera position and posture by using feature landmark database." Scandinavian Conference on Image Analysis . Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. https://link.springer.com/chapter/10.1007/11499145_19 Semaan, Bernard, et al. "Camera pose estimation using collaborative databases and single building image." Journal of Geographic Information System 12.06 (2020): 620-645. https://hal.science/hal-03060116/ Zeisl, Bernhard, Torsten Sattler, and Marc Pollefeys. "Camera pose voting for large-scale image-based localization." Proceedings of the IEEE International Conference on Computer Vision . 2015. https://www.cvfoundation.org/openaccess/content_iccv_2015/papers/Zeisl_Camera_Pose_Voting_ICCV_2015_paper.pdf T. Sato, Yoshiyuki Nishiumi, Mitsutaka Susuki, Tomoka Nakagawa and N. Yokoya, "Camera position and posture estimation from still image using feature landmark database," 2008 SICE Annual Conference , Chofu, Japan, 2008, pp. 1514-1519, doi: 10.1109/SICE.2008.4654900. https://ieeexplore.ieee.org/abstract/document/4654900 S. Hoque, M. Y. Arafat, S. Xu, A. Maiti and Y. Wei, "A Comprehensive Review on 3D Object Detection and 6D Pose Estimation With Deep Learning," in IEEE Access , vol. 9, pp. 143746-143770, 2021, doi: 10.1109/ACCESS.2021.3114399. https://ieeexplore.ieee.org/abstract/document/9543652 THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a) . A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ronde Miller whose telephone number is (703) 756-5686 The examiner can normally be reached Monday-Friday 8:00-4:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice . If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Gregory Morse can be reached on (571) 272-3838 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov . Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /RONDE LEE MILLER/Examiner, Art Unit 2663 /GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698 Application/Control Number: 18/533,422 Page 2 Art Unit: 2663 Application/Control Number: 18/533,422 Page 3 Art Unit: 2663 Application/Control Number: 18/533,422 Page 4 Art Unit: 2663 Application/Control Number: 18/533,422 Page 5 Art Unit: 2663 Application/Control Number: 18/533,422 Page 6 Art Unit: 2663 Application/Control Number: 18/533,422 Page 7 Art Unit: 2663 Application/Control Number: 18/533,422 Page 8 Art Unit: 2663 Application/Control Number: 18/533,422 Page 9 Art Unit: 2663 Application/Control Number: 18/533,422 Page 10 Art Unit: 2663 Application/Control Number: 18/533,422 Page 11 Art Unit: 2663 Application/Control Number: 18/533,422 Page 12 Art Unit: 2663 Application/Control Number: 18/533,422 Page 13 Art Unit: 2663 Application/Control Number: 18/533,422 Page 14 Art Unit: 2663 Application/Control Number: 18/533,422 Page 15 Art Unit: 2663 Application/Control Number: 18/533,422 Page 16 Art Unit: 2663 Application/Control Number: 18/533,422 Page 17 Art Unit: 2663 Application/Control Number: 18/533,422 Page 18 Art Unit: 2663 Application/Control Number: 18/533,422 Page 19 Art Unit: 2663 Application/Control Number: 18/533,422 Page 20 Art Unit: 2663 Application/Control Number: 18/533,422 Page 21 Art Unit: 2663
Prosecution Insights