DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 42 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 42 recites the limitation "the multiple panorama images" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/forms/. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim(s) 25-29, 31-33, 37-38, 40-43 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 2-4, 9, 11, 12 of patent US 12175562. Although the claims at issue are not identical, they are not patentably distinct from each other because the pending claim(s) is/are an obvious variation of the patented claims, or entirely covered by the patented claims.
For example, claim 25 of the instant application discloses A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations including at least: obtaining information from analysis of visual data of multiple images acquired in a building, generating a graph neural network with multiple layers to determine global acquisition pose information for the multiple images, initializing the nodes and edges of the first layer of the graph neural network using the obtained information from the analysis of the visual data of the pairs of the multiple images; propagating information from the initialized nodes and edges of the first layer through the multiple layers; generating mapping information for the building based at least in part on the determined global inter-image acquisition pose information for all of the multiple images; and providing, by the one or more computing devices, the generated mapping information for the building for further use. These limitations are all disclosed by the claim 9 of the patent US 12175562. Therefore, claim 25 of the instant application is covered by the claim 9 of the patent US 12175562, and is/are not patently distinct from the mentioned patent claim.
The following table illustrates a comparative mapping between the limitations of claim 25 of the instant application and the mapping claim 9 of patent US 12175562.
Claim 25 of the Instant Application 18/942331
Claim 9 of the Patent 12175562
A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations including at least:
A non-transitory computer-readable medium having stored instructions that cause one or more computing devices to perform automated operations including at least:
obtaining, by the one or more computing devices, information from analysis of visual data of multiple images acquired in a building, the obtained information including at least initial estimated local inter-image acquisition pose information for each of multiple image pairs that indicates position and orientation between two images for that pair in a local coordinate system for that pair;
obtaining, by the one or more computing devices, information from analysis of visual data of pairs of multiple panorama images acquired in a building that include at least a first image pair of first and second panorama images having first visual overlap including first visual data showing first walls of a first room of the building, and that further include at least a second image pair of the second panorama image and a third panorama image that has second visual overlap with the second panorama image including second visual data showing second walls of a second room of the building and that lacks visual overlap with the first panorama image, wherein the obtained information includes at least initial estimates of local inter-image acquisition pose information for each of the first and second image pairs that indicates relative position and orientation between the panorama images for that image pair in a local coordinate system for that image pair;
generating, by the one or more computing devices, a graph neural network with multiple layers to determine global acquisition pose information for the multiple images, wherein a first of the multiple layers of the graph neural network includes multiple nodes each associated with a respective one of the multiple images, and further includes multiple edges between at least some pairs of the multiple nodes to each represent inter-image acquisition pose information between two images associated with two nodes of the pair connected by that edge, the multiple edges including a plurality of edges each corresponding to one of the multiple image pairs;
generating, by the one or more computing devices, a graph neural network with multiple layers to determine global acquisition pose information for the multiple panorama images, wherein a first of the multiple layers of the graph neural network includes multiple nodes each associated with a respective one of the multiple panorama images, and further includes multiple edges between at least some pairs of the multiple nodes to each represent inter-image acquisition pose information between two panorama images associated with two nodes of the pair connected by that edge, the multiple edges including a first edge corresponding to the first image pair and a second edge corresponding to the second image pair;
initializing, by the one or more computing devices, the nodes and edges of the first layer of the graph neural network using the obtained information from the analysis of the visual data of the pairs of the multiple images, including adding encoded data to each of the nodes of the first layer about elements of the building visible in the image associated with that node, and adding information to each of the plurality of edges about the initial estimated local inter-image acquisition pose information for the image pair to which that edge corresponds;
initializing, by the one or more computing devices, the nodes and edges of the first layer of the graph neural network using the obtained information from the analysis of the visual data of the pairs of the multiple panorama images, including adding a representation to each of the nodes of the first layer that encodes data about elements visible in the panorama image associated with that node, and adding information to each of the edges about local inter-image acquisition pose information between the two panorama images associated with the two nodes for that edge, wherein the adding of the information to the edges includes adding information to the first edge about the initial estimates of the local inter-image acquisition pose information for the first image pair, and includes adding information to the second edge about the initial estimates of the local inter-image acquisition pose information for the second image pair;
propagating, by the one or more computing devices and using one or more loss functions, information from the initialized nodes and edges of the first layer through the multiple layers, including successively updating acquisition pose information associated with the multiple edges to produce, in a last of the multiple layers, determined global inter-image acquisition pose information for all of the multiple images in a common coordinate system;
propagating, by the one or more computing devices and using one or more loss functions, information from the initialized nodes and edges of the first layer through the multiple layers to coordinate local coordinate systems of the local inter-image acquisition pose information added to the multiple edges, including successively updating the local inter-image acquisition pose information associated with the multiple edges to produce, in a last of the multiple layers, determined global inter-image acquisition pose information for all of the multiple panorama images in a common coordinate system;
generating, by the one or more computing devices, mapping information for the building based at least in part on the determined global inter-image acquisition pose information for all of the multiple images; and
generating, by the one or more computing devices and using the determined global inter-image acquisition pose information of the multiple panorama images, at least a partial floor plan for the building that includes room shapes of at least the first and second rooms positioned relative to each other; and
providing, by the one or more computing devices, the generated mapping information for the building for further use.
presenting, by the one or more computing devices, the at least partial floor plan for the building, to enable use of the at least partial floor plan for navigation of the building.
The following is a complete listing of the correspondence between the claim of the instant application and the patents:
Claims of the Instant Application 18/942331
25, 37, 41
26, 38, 42
27, 43
28, 40
29
31
32
33
Claims of the Patent 12175562
9
9
12
11
4
2
3
9
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 25, 34, 37, 41, 44 is/are rejected under 35 U.S.C. 103 as being unpatentable over Türkoglu et al. (US 20220189060 A1) in view of KIM et al. (US 20200320738).
Regarding claim 25, Türkoglu discloses A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations (claim 13, “A non-transitory computer-readable storage medium storing instructions that, when executed by a computer processor of a client device, cause the computer processor to perform operations”) including at least:
obtaining, by the one or more computing devices, information from analysis of visual data of multiple images acquired in a building, the obtained information including at least initial estimated local inter-image acquisition pose information for each of multiple image pairs that indicates position and orientation between two images for that pair in a local coordinate system for that pair (“[0020] The real-world landmarks or objects 240 can be works of art, monuments, buildings, businesses, libraries, museums, or other suitable real-world landmarks or objects. [0028] The camera assembly 125 may be configured to capture single images or video as the image data. [0032] The positioning module 140 can be any device or circuitry for monitoring the position of the client device 110. For example, the positioning module 140 can determine actual or relative position by using a satellite navigation positioning system (e.g. a GPS system, a Galileo positioning system, the Global Navigation satellite system (GLONASS), the BeiDou Satellite Navigation and Positioning system), an inertial navigation system, a dead reckoning system, based on IP address, by using triangulation and/or proximity to cellular towers or Wi-Fi hotspots, and/or other suitable techniques for determining position. [0034] The re-localization module 145 uses a trained re-localization model to determine the pose of one or more cameras of the camera assembly 125. The re-localization model is trained to determine the relative pose of a camera from one or more images captured by that camera relative to one or more existing images of the physical environment around the client device 110. [0043] the re-localization training system 170 inputs an image into the re-localization model to generate relative pose predictions between the input image and other images captured in the same physical environment represented in a GNN”. Therefore, the input image and another image in a GNN corresponds to an image pair);
generating, by the one or more computing devices, a graph neural network with multiple layers to determine global acquisition pose information for the multiple images, wherein a first of the multiple layers of the graph neural network includes multiple nodes each associated with a respective one of the multiple images, and further includes multiple edges between at least some pairs of the multiple nodes to each represent inter-image information between two images associated with two nodes of the pair connected by that edge, the multiple edges including a plurality of edges each corresponding to one of the multiple image pairs (fig.4, “[0005] The resulting GNN has image representations on nodes and image-pair representations on edges. [0043] the re-localization training system 170 inputs an image into the re-localization model to generate relative pose predictions between the input image and other images captured in the same physical environment represented in a GNN. [0048] The image retrieval layer 420 retrieves a set number of reference images 425 for building a GNN. [0049] The graph generation layer 440 builds a graph (as part of a GNN) based on the reference images 425. [0063] The re-localization training system 170 builds 530 a GNN from the reference images. The GNN includes a node for each reference image and edges connecting each possible pair of reference images.”);
initializing, by the one or more computing devices, the nodes and edges of the first layer of the graph neural network using the obtained information from the analysis of the visual data of the pairs of the multiple images, including adding encoded data to each of the nodes of the first layer about elements of the building visible in the image associated with that node, and adding information to each of the plurality of edges about the initial estimated local inter-image information for the image pair to which that edge corresponds (“[0049] The graph generation layer 440 initializes nodes of the initialized graph 445 based on the reference images 425 and the input image 405. Each node may be represented as a node feature vector, e.g., as features of the feature map extracted by the feature extractor 410. An edge is located between two nodes and represented by an edge feature vector, and may be initialized by concatenating features of their nodes and projecting them with a learned linear mapping into an edge feature vector. [0061] In the embodiment shown, the method 500 begins with the re-localization training system 170 encoding 510 a training image. [0063] The GNN includes a node for each reference image and edges connecting each possible pair of reference images”. Claims 5&6, “building the graph comprises: initializing the reference images and the image of the scene as nodes of the graph; and initializing edges between the nodes of the graph. initializing the reference images and the image of the scene as nodes comprises encoding the image and the reference images into node feature vectors.”);
propagating, by the one or more computing devices and using one or more loss functions, information from the initialized nodes and edges of the first layer through the multiple layers, including successively updating acquisition pose information associated with the multiple edges to produce, in a last of the multiple layers, determined global inter-image acquisition pose information for all of the multiple images in a common coordinate system (“[0043] the re-localization training system 170 inputs an image into the re-localization model to generate relative pose predictions between the input image and other images captured in the same physical environment represented in a GNN, calculates a loss from the predicted relative pose and the ground truth relative pose, and updates the mode iteratively to minimize a total loss. [0050] Once the initialized graph 445 is initialized, the message passing layer 450 may refine the graph through message passing. To train the pose determination layer 460, a loss function calculates a loss between the predicted pose and the ground truth pose of an image pair in the graph. [0051] The pose determination layer 460 determines a pose 465 of the input image 405 based on the final graph 455. In one or more embodiments, the pose determination layer 460 may output a pose 465 (e.g., in 6 degrees of freedom) based on one or more edges of the final graph 455. In one or more embodiments, the pose determination layer 460 regresses the edge features in the final graph 455 to translation and orientation with a linear projection. [0053] The pose 465 may be a relative pose, i.e., defined as a translation and orientation shift from another image of the graph, or may be an absolute pose as determined based on known poses of the images of the image database 430. [0064] A loss function is used to enable the re-localization system 170 to train the re-localization model by updating one or more parameters of the model based on the loss function.”);
On the other hand, Türkoglu fails to explicitly disclose but KIM discloses inter-image information is inter-image acquisition pose information (“[0014] The estimating of the absolute pose of the remaining image may include generating a graph structure in which a pose of each image is defined as a node and a pose difference between images is defined as an edge with respect to the consecutive images”);
generating, by the one or more computing devices, mapping information for the space based at least in part on the determined global inter-image acquisition pose information for all of the multiple images; and providing, by the one or more computing devices, the generated mapping information for the space for further use (“abstract, updating a map used for visual localization (VL) based on the absolute pose. [0099] The map generator and updater 530 may generate the map for VL and may update the existing map using the absolute poses of the images calculated in operation S620. [0105] According to example embodiments, to seamlessly provide a VL based service, it is possible to maintain a VL map up to date, that is, update the VL map based on crowdsourcing and accordingly, to provide the VL based service robust and stable for a spatial/state change of a target space.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined KIM and Türkoglu, to include all limitations of claim 25. That is, applying the edge definition of KIM to the edge in the graph of Türkoglu, and adding the updated map of KIM to the building of Türkoglu. The motivation/ suggestion would have been Technology for updating a VL map using a crowdsourcing based image may be further efficient and permanent compared to an existing method of performing scanning using data collection equipment and then updating a VL map every time a change occurs in a target space (KIM, [0105]).
Regarding claim(s) 37, 41, they are interpreted and rejected for the same reasons set forth in claim(s) 25.
Regarding claim 34, Türkoglu in view of KIM discloses The non-transitory computer-readable medium of claim 25.
KIM discloses after the providing of the generated mapping information for the building, obtaining information about one or more additional images acquired at the building, using further information from analysis of further visual data of the one or more additional images to update the determined global inter-image acquisition pose information for all of the multiple images in the common coordinate system, and providing the updated determined global inter-image acquisition pose information (“[0099] The map generator and updater 530 may generate the map for VL and may update the existing map using the absolute poses of the images calculated in operation S620. Here, the map update may be performed by additionally inserting, into the existing map, information, for example, an image descriptor, a feature extracted from an image, a descriptor of the feature, and a 3D point of the feature, required for a pose estimation. [0100] That is, the map generator and updater 530 may generate and update the VL map based on the graph structure described with reference to FIG. 9. [0103] The map generator and updater 530 may update a VL map by adding images (failure images) of which absolute pose calculation through VL is a failure, that is, images of which poses are estimated through a graph-based pose optimization as a reference image of a corresponding point.”). The same motivation of claim 25 applies here.
Regarding claim(s) 44, it is interpreted and rejected for the same reasons set forth in claim(s) 34.
Claim(s) 26, 38, 42 is/are rejected under 35 U.S.C. 103 as being unpatentable over Türkoglu et al. (US 20220189060 A1) in view of KIM et al. (US 20200320738), and further in view of Cuthbertson et al. (US 20190019346).
Regarding claim 26, Türkoglu in view of KIM discloses The non-transitory computer-readable medium of claim 25.
On the other hand, Türkoglu in view of KIM fails to explicitly disclose but Cuthbertson discloses wherein the stored contents include software instructions that, when executed, cause the one or more computing devices to perform further automated operations including performing the generating of the mapping information for the building by generating at least a partial floor plan for the building that includes room shapes of at least two rooms of the building positioned relative to each other, and wherein the providing of the generated mapping information for the building includes presenting, by the one or more computing devices, the at least partial floor plan for the building, to enable use of the at least partial floor plan for navigation of the building (fig.34A, “[0053] In some embodiments, the premise is displayed a blueprint (e.g., a layout or a floor plan) that provides an overall view of the premise and allows the first user to navigate the rooms within the premises using the blueprint. [0268] Responsive to selection (3506) of the affordance and in accordance with FIG. 34A, a layout 53-1 of the premises 51 including the plurality of rooms and relationships between the plurality of rooms is displayed (3508). The layout 53-1 is also referred to as a blueprint (or floor plan/plan view) that the user uses to navigate the premises and access the various rooms. As depicted in FIG. 34A, the layout 53-1 shows a relative location (position) of each room in the premises 51, relationships between the plurality of rooms 3416, and includes walls separating the rooms 3416. [0325] In accordance with some implementations, a computing system (e.g., the client device 102) comprises a display 408, one or more processors 402, and memory 407 coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, singularly or collectively comprising instructions for running an application on the computing device that performs/executes any of the above methods and/or operations.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Cuthbertson into the combination of KIM and Türkoglu, to include all limitations of claim 26. That is, adding the floor plan view that the user uses to navigate of Cuthbertson to the display of the building of Türkoglu and KIM. The motivation/ suggestion would have been the user uses to navigate the premises and access the various rooms (Cuthbertson, [0268]).
Regarding claim(s) 38, 42, they are interpreted and rejected for the same reasons set forth in claim(s) 26.
Claim(s) 27, 43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Türkoglu et al. (US 20220189060 A1) in view of KIM et al. (US 20200320738), and further in view of Dawson et al. (US 20200336675).
Regarding claim 27, Türkoglu in view of KIM discloses The non-transitory computer-readable medium of claim 25.
On the other hand, Türkoglu in view of KIM fails to explicitly disclose but Dawson discloses wherein the automated operations further include performing the generating of the mapping information for the building by determining positions within rooms of the building at which each of the multiple images was acquired, and wherein the providing of the generated mapping information for the building further includes displaying the determined positions of the multiple images on determined room shapes of the rooms (fig.1D, “[0030] FIG. 1D further illustrates that inter-panorama links may be determined and used in some embodiments and situations to further link at least some viewing locations that are not visible to each other (e.g., the link in direction 215-BE between viewing locations 210B and 210E). [0035] The target positions at which to acquire and save images from a viewing location may be determined in various manners in various embodiments. In other embodiments, the target positions may be determined relative to one or previous positions at which images are acquired and saved, such as based on an amount of rotation and/or time since a last saved image, an amount of overlap in content with one or more previous saved images, etc.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Dawson into the combination of KIM and Türkoglu, to include all limitations of claim 27. That is, adding the determining and displaying the viewing location of the image captured by the mobile device of Dawson to the system of Türkoglu and KIM. The motivation/ suggestion would have been automatically controlling acquisition of images in a defined area for use in generating mapping information for the area, as well as for subsequently using the generated mapping information in one or more manners, such as by real-time analysis and use of sensor data on a mobile device that acquires the images (Dawson, [0002]).
Regarding claim(s) 43, it is interpreted and rejected for the same reasons set forth in claim(s) 27.
Claim(s) 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Türkoglu et al. (US 20220189060 A1) in view of KIM et al. (US 20200320738), and further in view of Schpok (US 20190051029).
Regarding claim 39, it is covered by claim 25, except that it further discloses the multiple images are each panorama images.
On the other hand, Türkoglu in view of KIM fails to explicitly disclose but Schpok discloses wherein the multiple images are each panorama images (“[0027] By way of example, the disclosed technology can include a computing device that receives image data from a remote image data source (e.g., a server computing device that stores a repository of panoramic images)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Schpok into the combination of KIM and Türkoglu, to include all limitations of claim 39. That is, applying the panoramic images of Schpok to the images of Türkoglu and KIM. The motivation/ suggestion would have been to more effectively process, manipulate, and modify images, thereby enhancing the user experience (Schpok, [0003]).
Allowable Subject Matter
Claim(s) 28-33, 35-36, 40 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 28, it recites, wherein the visual data of the multiple images includes only RGB (red-green-blue) pixel data, and wherein the obtaining of the information from the analysis of the visual data of the multiple images includes analyzing, by the one or more computing devices and using a neural network trained to jointly determine multiple types of information about the building, the multiple image pairs by, for each of the multiple image pairs: determining, as one of the multiple types of information and using partial visual overlap between the two images of the image pair that shows at least some of at least one room, image angular correspondence information for multiple pixel column matches that are each between a first column of pixels of a first of the two images and a respective second column of pixels of a second of the two images, with the first and second columns of pixels of a pixel column match both illustrating a same vertical slice of a wall of the at least one room, determining, as one of the multiple types of information and based on the RGB pixel data for the images of the image pair, structural layout information for the at least one room that includes positions of at least some walls of the at least one room, and that includes positions of one or more borders between one of the walls and at least one of an additional one of the walls or a floor of the at least one room or a ceiling of the at least one room, and that includes positions of at least one of a doorway or non-doorway wall opening of the at least one room; and determining, as one of the multiple types of information and based at least in part on information determined for the image pair that includes the determined multiple pixel column matches and the determined structural layout information, the initial estimated inter-image acquisition pose information for the image pair, including initial determined acquisition locations for the two images of the pair. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 40, it is interpreted and allowed under similar rationale set forth in claim 28.
Regarding claim 29, it recites, wherein the obtained information from the analysis of the visual data includes, for each of the multiple image pairs, information about structural elements of at least one room that are visible in the two images of the image pair and information about respective pixel columns in those two images that show same parts of the at least one room, and wherein the one or more loss functions include a node loss function to minimize errors in the global inter-image acquisition pose information in the common coordinate system and to minimize errors in the inter-image acquisition pose information for the multiple image pairs, and an edge loss function to minimize errors in the information about the structural elements and in the information about the respective pixel columns. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 30, it recites, wherein the obtained information from the analysis of the visual data includes information about walls of at least some rooms of the building, and wherein the one or more loss functions are based at least in part on geometrical constraints on positions of the walls. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 31, it recites, wherein the generating of the graph neural network includes creating a fully connected network in the first layer with edges between all pairs of nodes, and wherein the propagating of the information through the multiple layers includes determining degrees of confidence in the inter-image acquisition pose information associated with the multiple edges for each of the multiple layers, and performing, for at least one of the multiple edges having an associated determined degree of confidence below a determined threshold, at least one of removing the at least one edge from the graph neural network or discounting a weight associated with inter-image acquisition pose information for the at least one edge. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 32, it recites, wherein the propagating of the information through the multiple layers includes using message passing between nodes and layers of the graph neural network, and suspending, for at least one node having inter-image acquisition pose information in one or more attached edges with associated error that is below a determined threshold for a layer before the last layer, suspending message passing for the at least one node in subsequent layers of the graph neural network. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 33, it recites, wherein the automated operations further include obtaining initial estimates for the global inter-image acquisition pose information before the propagating of the information through the multiple layers, and further adding information to edges of the first layer from the initial estimates for the global inter-image acquisition pose information. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 35, it recites, wherein the automated operations further include, after the providing of the generated mapping information for the building, obtaining information about one or more additional images acquired at the building, using further information from analysis of further visual data of the one or more additional images in combination with the determined global inter-image acquisition pose information to determine further acquisition pose information for the one or more additional images in the common coordinate system, and providing the determined further acquisition pose information for the one or more additional images. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 36, it recites, wherein the building includes a plurality of rooms on two stories, wherein the multiple images include at least one image on each of the two stories and two or more images whose visual data include a stairway between the two stories, and wherein the determined global inter-image acquisition pose information for all of the multiple images includes acquisition pose information on both of the two stories using the two or more images to connect the at least one image on each of the two stories. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRACE Q LI whose telephone number is (571)270-0497. The examiner can normally be reached Monday - Friday, 8:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DEVONA FAULK can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRACE Q LI/Primary Examiner, Art Unit 2618 8/7/2026
