DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 102-117 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Phan et al1 (“Phan”).
Regarding claim 102, Phan teaches one or more non-transitory computer-readable media comprising executable instructions, the executable instructions being executable by one or more processors to perform a method, the method comprising (see Phan, paragraph 0089 teaching “a Building Information Viewer system routine 700, such as may be implemented by the BICPVRDP client application in some cases. The routine may be performed by, for example, execution of a BICPVRDP client application 154 of mobile device 175 of FIGS. 1 or 3 and/or of such a client application or other building information viewer system otherwise executing on a mobile device 175 and/or other computing system or device as described elsewhere herein” where such system includes implementation using such a non-transitory CRM with instructions executed by a processor as in paragraph 0063 this includes “Some or all of the components, systems and data structures may also be stored (e.g., as software instructions or structured data) on a non-transitory computer-readable storage mediums, such as a hard disk or flash drive or other non-volatile storage device, volatile or non-volatile memory (e.g., RAM or flash RAM), a network storage device, or a portable media article (e.g., a DVD disk, a CD disk, an optical disk, a flash memory device, etc.) to be read by an appropriate drive or via an appropriate connection”):
receiving an environment 3D representation (see Phan, paragraphs 0012-0014 teaching “generating a 3DSRF model or other visual model representation of a building and optionally additional associated types of building data, the automated operations may further include presenting or otherwise providing generated data for a building in one or more manners, such as to generate and present a new image with a view of the building exterior from a particular view pose (geographical location and orientation) in a displayed GUI (graphical user interface) and to provide user-selectable or otherwise manipulatable controls in the GUI to enable user input to interactively change the view of the building exterior” where this 3DSRF model for the building is an environment 3D representation” as further explain in paragraphs 0091-0092 teaching “to retrieve information about each of the target building(s) that includes a generated building model (e.g., floor plan model and/or a 3D volumetric model)” and “to present an interactive 3D building visual representation of a building using new synthetic images generated by a 3DSRF model of the building” and “to receive an indication of a current pose inside or around building, optionally via virtual movement controlled via user input in a displayed GUI from a prior or default pose to the current pose” and “to use a 3DSRF model for the building to generate a new building image for the current pose (e.g., a rendered rasterized view visualization), and receives the corresponding new building image. The routine then continues from block 732 to block 779 to present the current new building image, and to optionally receive requests for further new images to generate with respect to block 779 as discussed above” such that this “3DSRF model for the building” is an environment 3D representation), the environment 3D representation including Gaussian splats representing an environment (see Phan, paragraphs 0012-0014 teaching to use “3D Guassian splat points” and paragraph 0091 teaching as above “to use a 3DSRF model for the building to generate a new building image for the current pose” where this 3DSRF model is an environment 3D representation including Guassian splats representing the environment for the exterior views of the building which are modeled using 3D Guassian splatting as in paragraph 0034 teaching “system 140 obtains the images and other captured data 155 and uses it to determine one or more 3D building visual model representations 157 for the exterior of the building 198, such as one or more 3DSRF models (e.g., a 3DGS model with a 3D point cloud of 3D Gaussian splat points)”), the environment including a building having an interior and an exterior (note that importantly here the “environment” is recited as including a “building having an interior and an exterior” but the environment 3D representation that is received and includes Guassian splats is not recited as necessarily including building interior or exterior models, but rather the environment being modeled must include such a building having an interior and exterior; furthermore note that a “building having an interior and exterior” encompasses essentially any known type of building as for example a building defined by some frame or boundary or shape forming a structure could be defined as having an interior and exterior as even a completely solid building with no hollowed out interior would still have parts of it that are interior compared to others, however, it is noted that such definition is further defined and rendered functionally meaningful by recitations below which further specify recitations about the building representation in relation to its interior; see Phan, paragraph 0011 teaching “automatically generating visual models representing appearances of buildings based at least in part on captured external imagery of the buildings, and using the generated building visual models to generate and present corresponding new building images with views from particular view poses, and in some cases subsequently using the generated building visual models and associated information in one or more additional manners, such as to further improve navigation of a building and/or its surroundings” and “include acquiring building data of an exterior of a building from a plurality of exterior acquisition locations at multiple heights and view angles” and “further obtaining other additional data from indoor acquisition locations within the building” such that the environment is one including a building having an interior and exterior, which are both modeled, although the environment 3D representation in Phan corresponds to the external model of the building, and see paragraphs 0031 explicitly defining that a building as taught “the term “building” refers herein to any partially or fully enclosed structure, typically but not necessarily encompassing one or more rooms that visually or otherwise divide the interior space of the structure—non-limiting examples of such buildings include houses, apartment buildings or individual apartments therein, condominiums, office buildings, commercial buildings or other wholesale and retail structures (e.g., shopping malls, department stores, warehouses, etc.)”; see figures 2A-2H showing examples of an environment 3D representation), the environment 3D representation having a first location (note that the claim does not define what exactly constitutes a location and does not distinguish the first location from the second location such by requiring them to be distinct or necessarily different, and further note that the manner in which a 3D representation is “having a first location” is not limited such that if the environment 3D representation has a first location that occupies some coordinate or location space in any manner than the requirement is met and does not require the location to be absolute or unchanging, geographic, computed by any specific technique, stored as a discrete value, or independently generated; see Phan, paragraph 0012 teaching the environment 3D representation including the building exterior and interior where for example the 3DGS splat model has splat points occupying positions in a coordinate space of the 3DGS model that will be viewed where the system generates “analyzing the building data to generate one or more visual model representations of the building area(s) that encode visual appearances of them, including in at least some cases to analyze visual data of captured exterior images of a building to generate one or more 3D spatial radiance field building models to represent the building exterior by encoding visual appearances of visible surfaces of the building exterior” and “in the case of generating a 3DGS model, the techniques may include generating a plurality of 3D Gaussian splat points (e.g., a 3D point cloud with thousands or millions of 3D Gaussian splats) each having an associated 3D position on a visible surface around some or all of the building exterior, such as with each such 3D Gaussian splat point corresponding to a 3D ellipsoid blob with a shape defined by an associated mean x,y,z covariance matrix and encoding a view-dependent radiance function in which the color and transparency of the splat may vary based on an observer's angle of view to that splat's 3D location” such that the environment 3D representation has a first location such as the location of various splat points representing the environment and furthermore, a “new exterior building image generated using such a 3DSRF model from a particular 3D view pose (geographical location and orientation) may include some or all of the building exterior that is visible from that view pose” such that for example a “particular 3D view pose (geographical location and orientation” can also be considered the 3D environment model having a first location such a viewing location; furthermore, each 3DGS model 3D splat location in the model is assigned a location as in Phan, paragraphs 0033-0034 teaching “captured data 155 a (images and optionally additional data) is optionally associated with other capture metadata (e.g., GPS data for the acquisition locations from GPS sensors 134 a; pose data for the acquisition locations from IMU, or inertial measurement unit, sensor modules 148; depth data to the building from the acquisition locations, such as from optional depth sensors 136; etc.)” and determining absolute location data for the Guassian splat representation of the building exterior and interior where the system “obtains the images and other captured data 155 and uses it to determine one or more 3D building visual model representations 157 for the exterior of the building 198, such as one or more 3DSRF models (e.g., a 3DGS model with a 3D point cloud of 3D Gaussian splat points)” and “determines initial acquisition pose information for the captured images (e.g., using SLAM and/or SfM techniques), and uses that initial acquisition pose information to initialize an optimization process for initial generated 3D Gaussian splats to further refine the aisle acquisition pose information for the splats (e.g., an optimization process that uses gradient descent and/or a heuristic algorithm to optimize the 3D Gaussian splats' 3D locations and/or included Gaussians)” and “automatically determines particular GPS location data or other absolute location data to associate with some or all pieces of generated building data 165 and optionally with some or all of the generated building visual model representation 157 (e.g., with some or all 3D Gaussian splat points in a 3DGS model)” );
receiving a building representation, the building representation representing at least a portion of the interior of the building (note that the building representation is not recited as necessarily separate or distinct from the environment 3D representation such that for example if the environment 3D representation also contains such a building representation as defined then the environment 3D representation can correspond to the building representation, or the building representation can be considered a distinct representation or some combination with the environment 3D representation, so long as it is representing a portion of the interior of the building in some manner; see Phan, paragraph 0013 teaching “determining or otherwise generating additional types of building data such as a computer model of the building that encodes types of data other than visual appearance data (e.g., a 2D and/or 3D structural floor plan showing a layout of room shapes and other structures and areas of the building” such that this additional data of room shapes and 2D/3D models of the floor plan is building representation representing a portion of the interior such as the rooms which are interior and as in paragraph 0018, teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” such that a user may switch from view of the building exterior to a building representation of the interior to “enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.)” and this representation of the interior is a building representation that is received for viewing where such a building representation can be seen as discussed in paragraph 0052 and figure 2J teaching “an additional example GUI 207 j to show several types of information about an interior of the building. In this example, a primary pane 207 show an image of a living room of the building”), the building representation having a second location (here the “second location” is interpreted in the same manner as the “first location” and similarly as explained above, if the building representation has a second location that occupies some coordinate or location space in any manner than the requirement is met and does not require the location to be absolute or unchanging, distinct or different from the first location, geographic, computed by any specific technique, stored as a discrete value, or independently generated; see Phan, paragraph 0022 for example teaching a “3D model of the building’s interior” generated from “indoor images and/or other data for a building” and “to identify structural elements such as walls and doorways and windows and non-doorway wall openings, to determine the relative position of each image's acquisition location to such identified structural elements (e.g., within a local coordinate system for that image), to determine room shapes based on the identified structural elements and to identify each image's acquisition location within one of the room shapes (e.g., within a local coordinate system for that room), and to position such room shapes relative to each other to form at least a partial floor plan in a common local coordinate system for the floor plan, or to otherwise determine relative positions of acquisition locations of images without such a floor plan based at least in part on visual overlap between the images' visual data” such that this 3D model of the building interior occupies positions in a coordinate space defined in relation to these positions so that room shapes are positioned “relative to each other to form at least a partial floor plan in a common local coordinate system for the floor plan” such that these are building representations having second locations; note further that as explained in paragraph 0023 the interior building representation model can have absolution location data such as GPS data extended to be associated with GPS location as where “system may include automatically mapping target images (e.g., target panorama images, perspective rectilinear photos and other images, etc.) acquired at a building (e.g., in one or more rooms or other defined areas) to other absolute location data acquired at the building separately from the acquisition of the images (e.g., GPS data or other GNSS, or global navigation satellite system, data), and using such mappings to determine associated absolute locations for a visual representation of the building (e.g., a 3D point cloud of 3D Gaussian splat points) and/or a floor plan generated from the target images, such as to enable GPS location data or other absolute location data acquired at one or more data capture locations at the building to be extended to other locations that are determined at least in part from analysis of visual data of the one or more target images (e.g., locations of a room shape of a surrounding room, such as locations of at least walls of that room)” and thus the second location could correspond to any of these interior building representations determined absolute locations such as for example “locations of at least walls of that room” being second locations that have now been extended absolute GPS position data);
receiving one or more first inputs to navigate the environment 3D representation (note that to “navigate” the environment corresponds to any manner of traveling, steering, moving or otherwise making one’s way through some medium where in the context of navigating a 3D representation this can also include any viewing or display of an environment as such viewing and display of an environment corresponds to navigating the environment as this is a manner of moving through or accessing portions of the medium functionally equivalent to navigating; thus, see Phan, paragraph 0014 teaching “After generating a 3DSRF model or other visual model representation of a building and optionally additional associated types of building data, the automated operations may further include presenting or otherwise providing generated data for a building in one or more manners, such as to generate and present a new image with a view of the building exterior from a particular view pose (geographical location and orientation) in a displayed GUI (graphical user interface)” such that this corresponds to one or more inputs to navigate the environment 3D representation as the model itself is input as well as the particular view pose for the view of the image of the environment 3D representation which allows the user to navigate the environment 3D representation by viewing the representation and further “to provide user-selectable or otherwise manipulatable controls in the GUI to enable user input to interactively change the view of the building exterior, such as to perform virtual movements from a prior view pose or other default pose to a new current view pose from which a new image is generated and presented, and such as with corresponding new images (also referred to at times herein as a “visual rasterization rendering” or “rasterized building view rendering”) being generated for a current view pose in a real-time or near-real-time manner (e.g., with a response time within milliseconds, centiseconds, deciseconds, seconds, etc.) with respect to selection of that current pose (e.g., via user input from the GUI)” such that this allows a user to further navigate the environment 3D representation through such changing of the current view);
moving, based on the one or more first inputs, a first virtual camera in the environment 3D representation (note that a “virtual camera” is considered a functionally defined element understood by one having ordinary skill in the art as referring to an element that functions as a virtual camera where an element functioning as a virtual camera is one that is used to determine a viewpoint, perspective, or rendering angle within 3D or other visual data for capture of such visual data; thus see Phan, paragraph 0014 as explained above where a user can navigate the environment by moving a viewpoint for capturing an image of the environment 3D model, where it is taught that “user-selectable or otherwise manipulatable controls in the GUI to enable user input to interactively change the view of the building exterior, such as to perform virtual movements from a prior view pose or other default pose to a new current view pose from which a new image is generated and presented, and such as with corresponding new images (also referred to at times herein as a “visual rasterization rendering” or “rasterized building view rendering”) being generated for a current view pose in a real-time or near-real-time manner (e.g., with a response time within milliseconds, centiseconds, deciseconds, seconds, etc.) with respect to selection of that current pose (e.g., via user input from the GUI)” such that here this rendering of the rasterized image from the “view pose” selected corresponds to a first virtual camera in the 3D environment representation and this input can include moving the first virtual camera from the default or current viewing position to a “new current view pose” such that this moves the first virtual camera in the environment 3D representation to render the image from the spot the camera was moved to; see paragraph 0016 teaching for example “system may further in some cases enable some or all such limitations or restrictions to be overcome (e.g., based on additional supplied user input), including to enable moving the view pose towards or away from the building exterior” and “to change the orientation so that the building exterior is not shown (e.g., to show other outbuildings and/or other parts of a property on which the building is located, to point outwards from the building to show a surrounding neighborhood or otherwise to show surroundings, etc.)” such that for example “moving the view pose towards or away from the building” corresponds to moving the virtual camera );
receiving one or more second inputs to navigate the building representation (note that as above “navigate” is interpreted under the BRI established above; see Phan, paragraph 0018 teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” where this input to “switch” or change views to show the interior building representation corresponds to an input to navigate the building representation as the user can then have a “generated new image of a building interior” and furthermore as the 3DSRF model may be “used for representing and presenting view of building interiors”, this means the disclosed techniques for representing and presenting views of the building interior include the same movement of a virtual camera within the 3D model of the environment to present views of the 3D environment; see also paragraph 0052 and figure 2J showing other ways in which inputs are received to navigate the building representation once the building representation has been selected for viewing/manipulation where a user may select to view the interior building representation and can “show several types of information about an interior of the building” and “user-selectable controls are available to scroll through other images, and two additional smaller panes 208 and 209 are shown with other types of building information that are coordinated with the information display in the main pane 207, such as for pane 209 to show a portion of a 3D computer model of the building that corresponds to the current image displayed in pane 207, and for pane 208 to show part of an interactive virtual tour of the building corresponding to the current image displayed in pane 207” and “3D computer model includes illustrations of the positions of the viewing/capture locations for the current image of pane 207, with the visual indicator 219 b being added to correspond to the current viewing location and direction. In addition, the types of information shown in the different panes may be modified in various manners (e.g., to select the information of panes 208 or 209 to cause it to be enlarged in pane 207, with the information in the selected pane 208 or 209 changed to the photo(s) previously in pane 207 and/or to a different type of information). The interactive virtual tour information shown in pane 208 includes two user-selectable links via which corresponding other images may be displayed upon selection of the respective link (and with that other displayed image similarly having one or more selectable links to one or more images)” such that any input corresponding to the interactive model of the building interior can be a navigation input for the building representation such as by navigating the views and information); and
moving, based on the one or more second inputs, a second virtual camera in the building representation (see Phan, paragraph 0041 teaching “a floor plan (or portion of it) may be linked to or otherwise associated with one or more additional types of information, such as one or more associated and linked images or other associated and linked information, including for a two-dimensional (“2D”) floor plan of a building to be linked to or otherwise associated with a separate 2.5D model floor plan rendering of the building and/or a 3D model floor plan rendering of the building, etc” and “examples of an end-user's interactions with a displayed or otherwise generated 2D floor plan of a building may include one or more of the following: to change between a floor plan view and a view of a particular image at an acquisition location within or near the floor plan; to change between a 2D floor plan view and a 2.5D or 3D model view that optionally includes images texture-mapped to walls of the displayed model; to change the horizontal and/or vertical viewing direction from which a corresponding subset view of (or portal into) a panorama image is displayed, such as to determine a portion of a panorama image in a 3D coordinate system to which a current user viewing direction is directed, and to render a corresponding planar image that illustrates that portion of the panorama image without the curvature or other distortions present in the original panorama image; etc” such that here a user may interact with interior building representation through one or more second inputs for moving a second virtual camera in the building representation as this “change” of the “viewing direction” corresponds to moving a second virtual camera “to determine a portion of a panorama image in a 3D coordinate system to which a current user viewing direction is directed, and to render a corresponding planar image that illustrates that portion of the panorama image”; see further paragraph 0029 teaching “that a user viewing such a panorama image (or other image with sufficient horizontal and/or vertical coverage that only a portion of the image is displayed at any given time) may be permitted to move the viewing direction within the panorama image to different orientations to cause different subset images (or “views”) to be rendered within the panorama image, and that such a panorama image may in some situations be represented in a spherical coordinate system (including, if the panorama image is represented in a spherical coordinate system and a particular view is being rendered, to convert the image being rendered into a planar coordinate system, such as for a perspective image view before it is displayed)” such that a user may change input to move a virtual camera to a different capturing position; additionally and alternatively, Phan teaches such moving of a second virtual camera in the interior building representation as in paragraph 0018 teaching “Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” such that as explained with reference to paragraphs 0013-0014, such use of this type of model includes inputs to move a virtual camera through the model of the environment),
wherein the environment 3D representation and the building representation are aligned based at least in part on the first location of the environment 3D representation and the second location of the building representation (note that the claim merely recites a state, that building representations “are aligned,” but does not recite any actual alignment step or that such representations were in some manner out of alignment previously, and furthermore “based on” such locations means only that such two locations in some way contribute to the aligned state; see Phan, paragraph 0051 and figure 2I teaching a first manner in which the environment 3D representation and building representation are aligned based on the first and second locations where a selectable control allows “showing an exterior building view of a portion of the front of the building from an overhead location, and in which a control 202 i has been selected to cause a portion of a floor plan of the building to be overlaid on the image. In this example, lines and other visual indications are overlaid on the exterior building view to correspond to internal and external walls of portions of the building that are visible” and “it will be appreciated that a 3D mesh or other 3D volumetric model may similarly be overlaid on such an image in other cases” such that here it can be seen that the environment 3D representation which is the basis for the rendered view of the building representation and the respective points of the model being visualized are in some state of alignment with the locations of the walls and other information of the interior representation as can be seen in the spatially aligned images; further note that as in paragraph 0034 as explained above, “system 140 obtains the images and other captured data 155 and uses it to determine one or more 3D building visual model representations 157 for the exterior of the building 198, such as one or more 3DSRF models (e.g., a 3DGS model with a 3D point cloud of 3D Gaussian splat points)” and “system 160 further uses the visual data of at least captured interior images to determine room shapes of surrounding rooms, optionally in combination with some of the additional captured data (e.g., device motion data for the mobile data capture device), and combines the determined room shapes to generate associated building floor plans 165, optionally along with the identification of other building data from the building images/data 155 (e.g., POIs), although in other cases the system 140 may directly control some or all such generation of building floor plans, whether in addition to or of the MIGM system. The BICPVRDP system 140 also in some cases automatically determines particular GPS location data or other absolute location data to associate with some or all pieces of generated building data 165 and optionally with some or all of the generated building visual model representation 157 (e.g., with some or all 3D Gaussian splat points in a 3DGS model)” such that here the GPS location data may be associated with both the environment 3D representation representing the exterior portion of the building and the interior building data 165 of the interior portion of the building such that both may be considered aligned to the GPS coordinate system and to each other based on their model locations being mapped to the same GPS coordinate system; further note paragraph 0023 teaches the GPS location being extended to the interior building data where “BICPVRDP system may include automatically mapping target images (e.g., target panorama images, perspective rectilinear photos and other images, etc.) acquired at a building (e.g., in one or more rooms or other defined areas) to other absolute location data acquired at the building separately from the acquisition of the images (e.g., GPS data or other GNSS, or global navigation satellite system, data), and using such mappings to determine associated absolute locations for a visual representation of the building (e.g., a 3D point cloud of 3D Gaussian splat points) and/or a floor plan generated from the target images, such as to enable GPS location data or other absolute location data acquired at one or more data capture locations at the building to be extended to other locations that are determined at least in part from analysis of visual data of the one or more target images (e.g., locations of a room shape of a surrounding room, such as locations of at least walls of that room)” such that again as both representations are processed to determine such information this aligns them in the GPS coordinate space ).
Regarding claim 103, Phan teaches all that is required as applied to claim 102 above and further teaches wherein the environment 3D representation and the building representation are not displayed concurrently (note that the instant limitation of the representations “not displayed concurrently” has no temporal limitation functionally attached thereto and thus if the environment 3D representation and the building representation are not displayed concurrently at any time then the limitation is met; furthermore with regard to the following limitation, there is reciting of “while displaying the environment 3D representation” but there is no specific functional tie back that that “while displaying” is necessarily a situation “wherein the environment 3D representation and the building representation are not displayed concurrently”; see Phan, paragraph 0014 teaching “presenting or otherwise providing generated data for a building in one or more manners, such as to generate and present a new image with a view of the building exterior from a particular view pose (geographical location and orientation) in a displayed GUI (graphical user interface) and to provide user-selectable or otherwise manipulatable controls in the GUI to enable user input to interactively change the view of the building exterior” and “to otherwise select and display additional types of generated building data (whether in addition to a displayed building exterior image, such as overlaid on it or alongside it, or instead of the displayed building exterior image)” where here showing the “additional types of generated building data” (which as explained in the rejection above corresponds to a building representation model) is “instead of the displayed building exterior image” such that the representations are not shown concurrently; see also paragraph 0018 teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” where this input to “switch” or change views to show the interior building representation is another instance of environment 3D representation and building representation not being displayed concurrently; see also 2B-2G also showing examples of the environment 3D representation displayed not concurrently with the building representation and see figures 2J-2L showing a building representation displayed not concurrently with an environment 3D representation), and the method further comprising: while displaying the environment 3D representation, receiving one or more third inputs to select the building representation; and displaying the building representation (as explained above, the non-concurrent display of the environment 3D representation and building representation is not required to be during this “while displaying” and “receiving” step as it is not functionally required as explained above ;see Phan, paragraph 0018 as explained above teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” where this input to “switch” or change views to show the interior building representation is a third input to select the building representation as it selects to switch to the interior building representation where that is then displayed to the user; see also paragraphs 0091-0094 and figures 7A-7B teaching “to retrieve information about each of the target building(s) that includes a generated building model (e.g., floor plan model and/or a 3D volumetric model)” and “to present an interactive 3D building visual representation of a building using new synthetic images generated by a 3DSRF model of the building” and “to receive an indication of a current pose inside or around building, optionally via virtual movement controlled via user input in a displayed GUI from a prior or default pose to the current pose” such that here a reception of “an indication of a current pose inside…from a prior or default pose to the current pose” is one or more third inputs to select the building representation and to display the building representation where selection corresponding to interactively viewing and displaying the interior building representation corresponds to where “it is instead determined in block 732 that the instructions or other information received in block 705 do not indicate to present an interactive 3D building visual representation of an exterior of a building, the routine continues to block 738 to retrieve other information for the target building for display (e.g., a floor plan; other generated mapping information for the building, such as a group of inter-linked images for use as part of a virtual tour; generated building description information; etc.), and optionally indications of associated linked information for the building interior and/or a surrounding location external to the building, and/or information about one or more generated explanations or other descriptions of the target building, and selects an initial view of the retrieved information (e.g., a view of the floor plan, a particular room shape, a particular image, some or all of the generated building description information, etc.). After blocks 734 or 738, the routine in block 740 then displays or otherwise presents the current view of the retrieved information from block 738 or the retrieved/generated information from block 734 in a GUI, and waits in block 745 for a user selection (or optionally a timeout). After a user selection in block 745, if it is determined in block 750 that the user selection corresponds to adjusting the current view for the current target building (e.g., to change one or more aspects of the current view), the routine continues to block 755 to update the current view in accordance with the user selection (optionally interacting with the BICPVRDP system to obtain a modified view based on a user interaction with the previously presented view, or instead using previously received 3D visual representation(s) to generate the modified view), and then returns to block 740 to update the displayed or otherwise presented information accordingly” and “user selection and corresponding updating of the current view may include, for example, displaying or otherwise presenting a piece of associated linked information that the user selects (e.g., overlaying a selected type of information on a current view, such a particular image associated with a displayed visual indication of a determined acquisition location, POI information of one or more types, etc.; a particular other image linked to a current image and selected from the current image using a user-selectable control overlaid on the current image to represent that other image; etc.), and/or changing how the current view is displayed (e.g., zooming in or out; rotating information if appropriate; selecting a new portion of the floor plan to be displayed or otherwise presented, such as with some or all of the new portion not being previously visible, or instead with the new portion being a subset of the previously visible information; etc.)” such that these inputs lead to the displaying of the building representation).
Regarding claim 104, Phan teaches all that is required as applied to claim 103 above and further teaches wherein receiving the one or more third inputs to select the building representation includes receiving one or more requests to move the first virtual camera towards the building representation (see Phan, paragraph 0018 teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.)” such that a third input such as a user manipulable control allows to select a “transition…through” the environment representation towards the building representation where the building representation can then be displayed as an interior building view from the virtual camera now displaying the interior view where “use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors”; see also paragraph 0015 teaching display of the environment 3D model and that virtual camera movements may be “movements along the X and Y axes being permitted, including movement towards and/or away from the building, and with some or all of the building exterior being maintained in a center of the view” such that here as the movement is toward the building which is represented as the building and thus the camera movement is toward the building representation within the environment 3D model).
Regarding claim 105, Phan teaches all that is required as applied to claim 102 above and further teaches wherein the environment 3D representation and the building representation are not displayed concurrently (note that the instant limitation of the representations “not displayed concurrently” has no temporal limitation functionally attached thereto and thus if the environment 3D representation and the building representation are not displayed concurrently at any time then the limitation is met; furthermore with regard to the following limitation, there is reciting of “while displaying the environment 3D representation” but there is no specific functional tie back that that “while displaying” is necessarily a situation “wherein the environment 3D representation and the building representation are not displayed concurrently”; see Phan, paragraph 0014 teaching “presenting or otherwise providing generated data for a building in one or more manners, such as to generate and present a new image with a view of the building exterior from a particular view pose (geographical location and orientation) in a displayed GUI (graphical user interface) and to provide user-selectable or otherwise manipulatable controls in the GUI to enable user input to interactively change the view of the building exterior” and “to otherwise select and display additional types of generated building data (whether in addition to a displayed building exterior image, such as overlaid on it or alongside it, or instead of the displayed building exterior image)” where here showing the “additional types of generated building data” (which as explained in the rejection above corresponds to a building representation model) is “instead of the displayed building exterior image” such that the representations are not shown concurrently; see also paragraph 0018 teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” where this input to “switch” or change views to show the interior building representation is another instance of environment 3D representation and building representation not being displayed concurrently; see also 2B-2G also showing examples of the environment 3D representation displayed not concurrently with the building representation and see figures 2J-2L showing a building representation displayed not concurrently with an environment 3D representation), and the method further comprising: while displaying the building representation, receiving one or more third inputs to select the environment 3D representation; and displaying the environment 3D representation (see Phan, paragraph 0038 teaching “receive and respond to interactions by one or more users with the presented information (e.g., with displayed user-manipulatable controls, such as part of the generated visual data enhancements)” where these include “to change between a rasterized exterior building view rendering and/or a floor plan view and/or a view of a particular image at an acquisition location at the building” such that here this this discloses one or more third inputs that can be received while displaying the building representation such as “floor plan view” to “change” between that and “a rasterized exterior building view” which is a selection to select the environment 3D representation for display; when it is a change between the building representation to the environment 3D representation; see also paragraph 0018 teaching that the building representation may also be modeled by a 3DSRF model where “use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” and as in paragraph 0092 it is taught “to present an interactive 3D building visual representation of a building using new synthetic images generated by a 3DSRF model of the building, and if so the routine continues to block 734 to receive an indication of a current pose inside or around building, optionally via virtual movement controlled via user input in a displayed GUI from a prior or default pose to the current pose” such that here the pose input by the user for the current pose can select the environment 3D representation “inside or around the building” and “sends a request to BICPVRDP system to use a 3DSRF model for the building to generate a new building image for the current pose (e.g., a rendered rasterized view visualization), and receives the corresponding new building image” such that this would then display the environment 3D representation from the new pose of the virtual camera ).
Regarding claim 106, Phan teaches all that is required as applied to claim 105 above and further teaches wherein receiving the one or more third inputs to select the environment 3D representation includes receiving one or more requests to move the second virtual camera toward the environment 3D representation (see Phan, paragraph 0018 as explained above teaching to model the interior and exterior of the building using a 3D representation where views of either can be switched between and “use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” shows that one can select either representation by inputting a current view pose that selects to view either representation as in paragraph 0092 above teaching “to present an interactive 3D building visual representation of a building using new synthetic images generated by a 3DSRF model of the building, and if so the routine continues to block 734 to receive an indication of a current pose inside or around building, optionally via virtual movement controlled via user input in a displayed GUI from a prior or default pose to the current pose” such that here the pose input by the user for the current pose can select the environment 3D representation “inside or around the building” and “sends a request to BICPVRDP system to use a 3DSRF model for the building to generate a new building image for the current pose (e.g., a rendered rasterized view visualization), and receives the corresponding new building image” such that this would then display the environment 3D representation from the new pose of the virtual camera meaning that this was a request to move the second virtual camera toward the virtual environment representation location so that it can be located in the environment 3D representation in order to then provide the current pose of the virtual camera for display of the view from the moved camera; note in this case the first and second virtual cameras are defined by their current functions and are not required to be for example a clone of some specific instance of a virtual camera necessarily).
Regarding claim 107, Phan teaches all that is required as applied to claim 102 above and further teaches transitioning from displaying the environment 3D representation from a first perspective of the first virtual camera to displaying the building representation from a second perspective of the second virtual camera (see Phan, paragraph 0018, as explained above teaching “user-manipulatable controls may further enable a user to switch from such a generated new image of a building exterior to one or more other types of building data (e.g., particular images; videos; a floor plan or other computer model; an interactive tour of inter-linked images; a generated new image of a building interior, including based on virtual movements that transition from an exterior ground-level pose through a building external doorway or otherwise through an external surface of the building, or based on other virtual movements that transition from an exterior aerial pose (e.g., above a ground-level view, such as above 7 feet or 10 feet or other defined height, etc.) through an external surface of the building; etc.). Such use of 3DSRF model visual representations (e.g., 3D Gaussian splat point cloud visual representations) and/or other types of building visual representations may in some cases be used for representing and presenting views of building interiors, such as using one or more corresponding 3DSRF models generated for a building exterior (of a single 3DSRF model that encodes visual appears for both a building's interior and exterior), and whether in addition to or instead of building exteriors” where this input to “switch” or change views to show the interior building representation is transition from the first camera to displaying the building representation from a second perspective of the second virtual camera where this is simply a second perspective compared to the first camera and corresponds to the virtual camera that is providing the second perspective; see also paragraphs 0091-0094 and figures 7A-7B teaching “to retrieve information about each of the target building(s) that includes a generated building model (e.g., floor plan model and/or a 3D volumetric model)” and “to present an interactive 3D building visual representation of a building using new synthetic images generated by a 3DSRF model of the building” and “to receive an indication of a current pose inside or around building, optionally via virtual movement controlled via user input in a displayed GUI from a prior or default pose to the current pose” such that here a reception of “an indication of a current pose inside…from a prior or default pose to the current pose” is one or more third inputs to select to transition to the building representation and to display the building representation where selection corresponding to interactively viewing and displaying the interior building representation corresponds to where “it is instead determined in block 732 that the instructions or other information received in block 705 do not indicate to present an interactive 3D building visual representation of an exterior of a building, the routine continues to block 738 to retrieve other information for the target building for display (e.g., a floor plan; other generated mapping information for the building, such as a group of inter-linked images for use as part of a virtual tour; generated building description information; etc.), and optionally indications of associated linked information for the building interior and/or a surrounding location external to the building, and/or information about one or more generated explanations or other descriptions of the target building, and selects an initial view of the retrieved information (e.g., a view of the floor plan, a particular room shape, a particular image, some or all of the generated building description information, etc.). After blocks 734 or 738, the routine in block 740 then displays or otherwise presents the current view of the retrieved information from block 738 or the retrieved/generated information from block 734 in a GUI, and waits in block 745 for a user selection (or optionally a timeout). After a user selection in block 745, if it is determined in block 750 that the user selection corresponds to adjusting the current view for the current target building (e.g., to change one or more aspects of the current view), the routine continues to block 755 to update the current view in accordance with the user selection (optionally interacting with the BICPVRDP system to obtain a modified view based on a user interaction with the previously presented view, or instead using previously received 3D visual representation(s) to generate the modified view), and then returns to block 740 to update the displayed or otherwise presented information accordingly” and “user selection and corresponding updating of the current view may include, for example, displaying or otherwise presenting a piece of associated linked information that the user selects (e.g., overlaying a selected type of information on a current view, such a particular image associated with a displayed visual indication of a determined acquisition location, POI information of one or more types, etc.; a particular other image linked to a current image and selected from the current image using a user-selectable control overlaid on the current image to represent that other image; etc.), and/or changing how the current view is displayed (e.g., zooming in or out; rotating information if appropriate; selecting a new portion of the floor plan to be displayed or otherwise presented, such as with some or all of the new portion not being previously visible, or instead with the new portion being a subset of the previously visible information; etc.)” such that these inputs lead to the displaying of the building representation from a second perspective of a second virtual camera as for example figure 2J shows a second perspective of a second virtual camera of the building representation) or from displaying the building representation from the second perspective of the second virtual camera to displaying the environment 3D representation from the first perspective of the first virtual camera.
Regarding claim 114, Phan teaches all that is required as applied to claim 102 above and further teaches wherein multiple third locations are associated with the environment 3D representation, the first location is determined based at least in part on the multiple third locations (note that the locations of the environment and building representations are not recited as specifically interacting with each other given that for example the alignment of the parent claim does not require any alignment step, but rather requires a state, however such state must still be in accordance with the states recited in the current claim; see Phan, paragraph 0012 teaching as explained above that the first location may be the location of some 3D splat point in a generated 3DGS model as in “generating a plurality of 3D Gaussian splat points (e.g., a 3D point cloud with thousands or millions of 3D Gaussian splats) each having an associated 3D position on a visible surface around some or all of the building exterior, such as with each such 3D Gaussian splat point corresponding to a 3D ellipsoid blob with a shape defined by an associated mean x,y,z covariance matrix and encoding a view-dependent radiance function in which the color and transparency of the splat may vary based on an observer's angle of view to that splat's 3D location” and as in paragraphs 0033-0034 these splat locations which include the first location explained above, are determined based on multiple third locations associated with the 3D environment representation such as the locations of the images from which the 3DGS environment representation model is generated and thus based on where “captures building exterior images and/or other data, the captured data 155 a (images and optionally additional data) is optionally associated with other capture metadata (e.g., GPS data for the acquisition locations from GPS sensors 134 a; pose data for the acquisition locations” and then using the data from those multiple third locations, “BICPVRDP system 140 obtains the images and other captured data 155 and uses it to determine one or more 3D building visual model representations 157 for the exterior of the building 198, such as one or more 3DSRF models (e.g., a 3DGS model with a 3D point cloud of 3D Gaussian splat points)” where “system 140 determines initial acquisition pose information for the captured images (e.g., using SLAM and/or SfM techniques), and uses that initial acquisition pose information to initialize an optimization process for initial generated 3D Gaussian splats to further refine the aisle acquisition pose information for the splats (e.g., an optimization process that uses gradient descent and/or a heuristic algorithm to optimize the 3D Gaussian splats' 3D locations and/or included Gaussians)” such that here then the splat points in the model and their locations are based on the multiple capture locations of the environment; furthermore, the determined absolute location; note finally paragraph 0042 teaching that the location of each of the exterior building model and interior building model are based on multiple third and fourth locations where “the system 140 receives building images/data 155 of an exterior of a building from one or more flying drone imaging devices 179 and/or ground-based camera devices 184 (and optionally additional interior images/data from an IDCA system 150), such as in response to a BICPVRDP Imagery Automated Capture Plan Determiner component 141 generating and providing an optional imagery capture plan 156 to the drone(s) 179 and/or the camera device(s) 184 and/or to an associated operator user device 185. As part of preprocessing activities, a BICPVRDP 3D Building Visual Model Representation Generator component 143 analyzes the building images and optionally other data and generates one or more resulting 3D building visual representations 157, such as a 3D point cloud of 3D Gaussian splats. An optional MIGM system 160 may further analyze the building images/data 155 and/or the 3D building visual model representations 157 in order to generate other building data 165, such as a building floor plan, POIs, etc” such that here the locations of the captured images form the basis for the location of the model points in the environment 3D representation including the building exterior and the locations of the indoor captured images form the basis for the location of the model points in the interior building representation), multiple fourth locations are associated with the building representation, and the second location is determined based at least in part on the multiple fourth locations (see Phan, paragraph 0022 teaching “system 140 determines initial acquisition pose information for the captured images (e.g., using SLAM and/or SfM techniques), and uses that initial acquisition pose information to initialize an optimization process for initial generated 3D Gaussian splats to further refine the aisle acquisition pose information for the splats (e.g., an optimization process that uses gradient descent and/or a heuristic algorithm to optimize the 3D Gaussian splats' 3D locations and/or included Gaussians)” such that the locations of the “captured images” are the basis for generating the location of each point in the model of the interior and to locate and “position such room shapes relative to each other to form at least a partial floor plan in a common local coordinate system for the floor plan” such that in order to generate the location for any point being modeled in the building interior representation this depends on the multiple images supplying surrounding information to inform where the information contributes to the model generation as further explained in paragraph 0029 teaching the images acquired as explained above “some or all of the target images acquired for a building may be panorama images that are each acquired at one of multiple acquisition locations in or around the building, such as to generate a panorama image at each such acquisition location” such that these are multiple fourth locations and further “acquisition metadata regarding the capture of such panorama images may be obtained and used in various manners, such as data acquired from IMU sensors or other sensors of a mobile device as it is carried by a user or otherwise moved between acquisition locations—non-exclusive examples of such acquisition metadata may include one or more of acquisition time; acquisition location, such as GPS coordinates or other indication of location; acquisition direction and/or orientation; relative or absolute order of acquisition for multiple images acquired for a building or that are otherwise associated; etc” and as in paragraph 0034 it is explained how such data is used where “(MIGM) system 160 further uses the visual data of at least captured interior images to determine room shapes of surrounding rooms, optionally in combination with some of the additional captured data (e.g., device motion data for the mobile data capture device), and combines the determined room shapes to generate associated building floor plans 165” and “system 140 also in some cases automatically determines particular GPS location data or other absolute location data to associate with some or all pieces of generated building data 165 and optionally with some or all of the generated building visual model representation 157 (e.g., with some or all 3D Gaussian splat points in a 3DGS model)” such that again the multiple capture locations determine room shapes and the locations of these shapes in the interior model making the second location based on multiple 4th locations; note finally paragraph 0042 teaching that the location of each of the exterior building model and interior building model are based on multiple third and fourth locations where “the system 140 receives building images/data 155 of an exterior of a building from one or more flying drone imaging devices 179 and/or ground-based camera devices 184 (and optionally additional interior images/data from an IDCA system 150), such as in response to a BICPVRDP Imagery Automated Capture Plan Determiner component 141 generating and providing an optional imagery capture plan 156 to the drone(s) 179 and/or the camera device(s) 184 and/or to an associated operator user device 185. As part of preprocessing activities, a BICPVRDP 3D Building Visual Model Representation Generator component 143 analyzes the building images and optionally other data and generates one or more resulting 3D building visual representations 157, such as a 3D point cloud of 3D Gaussian splats. An optional MIGM system 160 may further analyze the building images/data 155 and/or the 3D building visual model representations 157 in order to generate other building data 165, such as a building floor plan, POIs, etc” such that here the locations of the captured images form the basis for the location of the model points in the environment 3D representation including the building exterior and the locations of the indoor captured images form the basis for the location of the model points in the interior building representation).
Regarding claim 115, Phan teaches all that is required as applied to claim 114 and further teaches wherein the multiple third locations include multiple GPS locations and the first location is determined at least in part by applying a fitting algorithm to the multiple GPS locations (note that “applying a fitting algorithm” is extremely broad as recited and the broadest reasonable interpretation of such fitting is any process that operates on data to produce a result made acceptable or suitable for a given purpose (as to fit, means to make something acceptable for some purpose for example) and does not require any particular algorithm or manner in which the fitting is to be performed, thus the limitation is met by any determination that produces an acceptable location value from multiple GPS locations where the multiple GPS locations need only be operated on and contribute to the determination in any manner; thus see Phan, paragraph 0033 as explained above teaching an instance of multiple capture locations associated with multiple GPS locations and “the captured data 155 a (images and optionally additional data) is optionally associated with other capture metadata (e.g., GPS data for the acquisition locations from GPS sensors 134 a; pose data for the acquisition locations from IMU” and “stored in memory/storage 152 a of the drone 179 in this example before being transmitted to the BICPVRDP system 140 (optionally via the drone operator device 185). Similarly, if the camera device 184 is used to capture at least some of the images and optionally other data for the exterior of the building, the resulting captured data 155 b may similarly be optionally associated with some or all of the same types of capture metadata and stored on the camera device before being transmitted to the BICPVRDP system 140” and in paragraphs 0073-0074 teaching “more image acquisition poses at which to acquire additional images from the camera device at the current time and/or more data capture poses at which to acquire other data from the mobile device at the current time, the routine continues to block 522 to optionally initiate the acquisition of linking information (e.g., acceleration data, visual data, etc.)” and “acquired linking information may include additional sensor data (e.g., from one or more IMU, or inertial measurement units, on the mobile device or otherwise carried by the user; from one or more LiDAR or other depth-sensing sensors; from one or more GPS sensors; etc.)” and “the images and other captured data and any associated generated or obtained information is stored for later use, and optionally provided to one or more recipients” where as in paragraph 0034 it explained that initial location data for a “splats’ 3D locations” in the 3DGS model is determined from “the captured images” and then this data is assigned a first GPS location by applying a fitting algorithm to the multiple GPS locations where “BICPVRDP system 140 also in some cases automatically determines particular GPS location data or other absolute location data to associate with some or all pieces of generated building data 165 and optionally with some or all of the generated building visual model representation 157 (e.g., with some or all 3D Gaussian splat points in a 3DGS model)” such that here a “particular GPS location” is automatically determined to be assigned to such point based on the GPS locations that are stored for use, such that the first location is based on multiple GPS locations supplying GPS data, from which the particular GPS location that is assigned is fit from).
Regarding claims 108-113, and 116-117 the instant claims recite a method that corresponds to the functions carried out by the apparatus as in claims 102-107, and 114-115 respectively. As the apparatus claims 102-107, and 114-115 function to perform the method as in claims 102-107, and 114-115 the limitations correspond. Therefore the limitations of claims 108-113 and 116-117 correspond to the limitations of claims 102-107, and 114-115 respectively; thus they are rejected on the same grounds as claims 102-107, and 114-115 respectively.
Response to Arguments
Applicant's arguments filed 7/6/2026 have been fully considered but they are not persuasive. Applicant argues on pages 10-12 of “REMARKS” filed 7/6/2026. First, regarding Applicant’s arguments against claims 102 and 108, note that Applicant does not actually argue that Phan fails to teach or suggest any specific, recited claim language, rather Applicant merely alleges that Phan teaches certain techniques, and then in general alleges that Phan does not teach various techniques, none of which techniques are specific tied to the specific claim language or to any BRI of the claim language.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “an alignment of the environment 3D representation with the building representation” and “takes the exterior Guassian splat representation as one object and the interior building representation…as another object and aligns them with each other based on their respective locations” and “two representations exist as separate, independently located objects that must be brought into spatial registration”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). As is explained in the rejections above, the claims do not require “an alignment” necessarily, nor do they require the method “aligns them with each other based on their respective locations” nor that the “two representations exist as separate, independently located objects that must be brought into spatial registration.” Rather, the claims require the representations “are aligned” which describes a state, not a function or specific act. As explained above, Phan teaches that such positions that can be attributed as claimed to both the environment 3D representation and building representation are aligned. Furthermore, while not required by the claim, Phan also teaches aligning representations in a more active association step as explained above. Thus Applicant’s arguments are not persuasive and the claims stand rejected as fully explained above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Hovden et al (US PGPUB No. 2019/0251352) – note for example paragraphs 0064-0070 teaching filtering related to GPS locations when capturing images at multiple locations with GPS data in order to assign the best GPS data to locations teaching “the filtered/remaining set of GPS measurements with high SNR values (e.g., greater than 25.0 dB) after removal of the GPS measurements having low SNR values” and “even if the noisy GPS measurements are removed, they still may be a plurality of GPS measurements that could be associated with a single scan location. For example, the number of GPS measurements captured over the course of a scan can exceed the number of capture locations, especially in implementations in which the GPS module is configured to capture several GPS measurements during rotation of the camera at a fixed location and/or between capture locations” and “filtering out the “bad” raw GPS coordinates and grouping or clustering the remaining “good” (e.g., high SNR) GPS coordinates to respective capture locations based on time of capture, each capture location can be associated with a cluster or group (of one or more) raw, “good” GPS coordinates”.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT E SONNERS whose telephone number is (571)270-7504. The examiner can normally be reached Mon-Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT E SONNERS/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
1 US PGPUB No. 20260004523, note all disclosures in the PGPUB mapped above have full support in the provisional application filed 6/26/2024
Prosecution Insights