DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-18 and 23-42 are presented for examination based on the amended claims in the application filed on April 9, 2026. Claims 19-22 have been cancelled by the applicant.
Claims 1-18 and 23-42 are rejected under 35 U.S.C. § 103 as being unpatentable over US 2020/0005428 A1 Sedeffow et al. [herein “Sedeffow”] in view of US 2019/0026958 A1 et al. [herein “Gausebeck”].
This action is made Non-Final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 9, 2026 has been entered.
Response to Amendment
The amendment filed April 9, 2026 has been entered. Claims 1-18 and 23-42 remain pending in the application. Applicant’s amendments to the Claims have overcome each and every objection and 112(b) rejections previously set forth in the Final Office Action mailed January 9, 2026.
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-18 and 23-42 are rejected under 35 U.S.C. § 103 as being unpatentable over US 2020/0005428 A1 Sedeffow et al. [herein “Sedeffow”] in view of US 2019/0026958 A1 et al. [herein “Gausebeck”].
As per claim 1, Sedeffow teaches “A computer-implemented method comprising obtaining, by one or more computing devices, a plurality of panorama images that are captured at a plurality of acquisition locations in multiple rooms of a house, wherein each of the panorama images is captured in one of the multiple rooms and includes 360 degrees of horizontal visual coverage around a vertical axis that provides RGB (red-green-blue) pixel data in an equirectangular format for at least some of walls and a floor and a ceiling of that one room”. (Para. 0071, “At step 402, the user uploads images to their computing device” [obtaining, by one or more computing devices, a plurality of images]. Para. 0062, “the invention described herein provides a method of linking a set of panoramic photographs” [a plurality of panorama images]. Para. 0063, “Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [panorama images that are captured at a plurality of acquisition locations in multiple rooms of a house, wherein each of the panorama images is captured in one of the multiple rooms]. Para. 0062, “A panoramic photograph is a rectangular photograph that represents a 360° view of a space” [includes 360 degrees visual coverage]. Para. 0075, “An example of a spherical photograph of the study 205, taken by the spherical camera 501, is shown in FIG. 6. It is an equirectangular projection, which is the typical output of a spherical camera. The ceiling and the floor are stretched, but the walls are mainly in proportion” [in an equirectangular format for at least some of walls and a floor and a ceiling of that one room]. Fig 6 also shows that the images are captured in 360 degrees about the vertical axis to provide a horizontal visual coverage of the room. Para. 0132, “The pixel information imported is the RGB (red, green, blue) and alpha (opacity) values for the selected pixel” [that provides RGB (red-green-blue) pixel data]. The examiner has interpreted that uploading a set of panoramic photographs to a computing device where the images are taken of each room in a 360° view of a space, provide RGB pixel values, and is an equirectangular projection showing the ceiling, floor, and wall as obtaining, by one or more computing devices, a plurality of panorama images that are captured at a plurality of acquisition locations in multiple rooms of a house, wherein each of the panorama images is captured in one of the multiple rooms and includes 360 degrees of horizontal visual coverage around a vertical axis that provides RGB (red-green-blue) pixel data in an equirectangular format for at least some of walls and a floor and a ceiling of that one room.)
Sedeffow also teaches “analyzing, by the one or more computing devices and [using a first neural network trained to jointly determine layout information for rooms visible in images and determine image pose information for those images within those layouts, and] for each of the multiple rooms using only the RGB pixel data of one of the panorama images captured in that room, the RGB pixel data of that one panorama image to generate information about that room that includes a generated structural layout of that room indicating positions of at least some identified structural wall elements of that room including the at least some walls of that room, and that includes a determined position within that structural layout of the acquisition location for that one panorama image”. (Para. 0170, “The enhancements are carried out by viewer instructions 1311 on the memory 1102 of laptop 108” [analyzing, by the one or more computing devices]. Para. 0175, “However, the image contains no data or coordinates relating to whether particular areas of the image relate to a floor, a wall, a ceiling, etc. Each pixel is merely a point on a sphere and no further information is available. Further, each pixel is considered to be the same distance away from the centre of the notional sphere, and therefore no information is available relating to distances within the physical space shown in image 2908” [using only the RGB pixel data]. Para. 0176, “The invention as described herein provides a method of converting the spherical coordinates of a cursor point to two-dimensional Cartesian coordinates on a specific plane within the sphere. Thus, for example, if the floor is a plane of the sphere, then spherical coordinates can be converted to two-dimensional Cartesian coordinates on the floor, thus providing a floor plan of the room. This will be described further with reference to FIGS. 38 to 48.” Para. 0241, “FIG. 39 shows interface 2901, in which image 2908 is shown in the editing area” [the RGB pixel data of that one panorama image]. Para. 0245, “The user selects a corner of the room by moving the cursor to a corner that is visible in image 2908” [using only the RGB pixel data of one of the panorama images captured in that room]. Para. 0256, “The user continues to select corners as in the examples shown in FIGS. 39 to 41 until all corners have been selected. The first corner should be selected as the final corner, thus closing the cycle graph” [to generate information about that room that includes a generated structural layout of that room]. Para. 0239, “Each room belongs to a floor, and each vertex belongs to a room. Within each list of vertices for a room, an ordering field 3805 gives the position of each vertex, thereby creating a cycle graph for each room” [for each of the multiple rooms]. Para. 0249, “After the user has selected two corners, a wall is considered to be fully defined” [identified structural wall elements of that room including the at least some walls of that room]. Para. 0183, “a plurality of intersections between walls are identified and their Cartesian coordinates calculated” [indicating positions of at least some identified structural wall elements of that room including the at least some walls]. Para. 0182, “These two-dimensional Cartesian coordinates represent an actual location on the floor in the physical space imaged, for a point in image 2908 having the specified spherical coordinates” [includes a determined position within that structural layout of the acquisition location for that one panorama image]. The examiner has interpreted that carrying out instructions on the memory of a laptop using only the visible pixel points in the image that involve selecting corners of the room to form a cycle graph of a room and each room, defining a wall in the room, and calculating the intersections of walls in addition to the actual location of the floor that the physical space was imaged as analyzing, by the one or more computing devices and for each of the multiple rooms using only the RGB pixel data of one of the panorama images captured in that room, the RGB pixel data of that one panorama image to generate information about that room that includes a generated structural layout of that room indicating positions of at least some identified structural wall elements of that room including the at least some walls of that room, and that includes a determined position within that structural layout of the acquisition location for that one panorama image.)
Sedeffow also teaches “generating, by the one or more computing devices and for each of the panorama images using only information determined from the RGB pixel data of that panorama image, views that are rendered in two dimensions in a perspective or orthographic format and that include a floor view of the at least some floor of the one room in which that panorama image is captured and that include a ceiling view of the at least some ceiling of that one room, wherein each of the rendered views includes some of the RGB pixel data of that panorama image that is positioned in that rendered view based at least in part on the generated structural layout for that one room and by using raycasting for pixels of that rendered view [to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view], and further includes at least one additional type of information overlaid on the some RGB pixel data included in that rendered view”. (Para. 0170, “The enhancements are carried out by viewer instructions 1311 on the memory 1102 of laptop 108” [by the one or more computing devices]. Para. 0175, “However, the image contains no data or coordinates relating to whether particular areas of the image relate to a floor, a wall, a ceiling, etc. Each pixel is merely a point on a sphere and no further information is available. Further, each pixel is considered to be the same distance away from the centre of the notional sphere, and therefore no information is available relating to distances within the physical space shown in image 2908” [using only the RGB pixel data]. Para. 0110, “The conversion of an equirectangular image 601 to a circular projection 1701 is shown in FIG. 17” [generating views that are rendered]. FIG. 14 and FIG. 17 show 2D perspective views of the captured panorama images and further show additional information such as door and windows in this rendered view. Para. 0221, “the viewer is looking downwards, and the Cartesian coordinates should be calculated with respect to a plane below the centre of the sphere, which in this case is the floor” [that include a floor view of the at least some floor of the one room in which that panorama image is captured]. Para. 0222, “the viewer is looking upwards and the location should be calculated with respect to a plane above the centre of the sphere, in this case the ceiling” [include a ceiling view of the at least some ceiling of that one room]. Para. 0121, “FIGS. 19 and 20 show the formulae used in the steps carried out in FIG. 18. FIG. 19 shows how the distance from the centre of a containing square 1901 to a selected point (c, r) is used to determine whether the point is filled in, with pixel information from the original equirectangular image, or is left blank”. [e.g., using raycasting for pixels of that rendered view]. Further see Fig. 19 that shows processing the pixels on the images and the distance to such, e.g., using raycasting for pixels of that rendered view. Para. 0132, “The pixel information imported is the RGB (red, green, blue) and alpha (opacity) values for the selected pixel”. Para. 0124, “Point (c.sub.2, r.sub.2) is within the annulus and therefore pixel information from the source image will be imported” [wherein each of the rendered views includes some of the RGB pixel data of that panorama image that is positioned in that rendered view]. Para. 0234, “Each vertex in the cycle graph comprises a set of spherical coordinates and corresponding Cartesian coordinates. If two planes are being considered, such as the floor and the ceiling” [based at least in part on the generated structural layout for that one room]. Para. 0283, “The floor plan created by rendering the cycle graph is shown at 4601. It will be seen that the positions of the hotspots have been transformed into doorways in the floor plan” [further includes at least one additional type of information overlaid on the some RGB pixel data included in that rendered view]. The examiner has interpreted that carrying out instructions on the memory of a laptop using only the visible pixel points in the image to convert the panorama image from an equirectangular image to a circular projection in a 2D format from the imported RGB pixel information showing doors and windows while looking downwards to the floor or upwards to the ceiling and from the calculated coordinates that are calculated from the distance to the selected point from the center as generating, by the one or more computing devices and for each of the panorama images using only information determined from the RGB pixel data that panorama image, views that are rendered in two dimensions in a perspective or orthographic format and that include a floor view of the at least some floor of the one room in which that panorama image is captured and that include a ceiling view of the at least some ceiling of that one room, wherein each of the rendered views includes some of the RGB pixel data of that panorama image that is positioned in that rendered view based at least in part on the generated structural layout for that one room and by using raycasting for pixels of that rendered view, and further includes at least one additional type of information overlaid on the some RGB pixel data included in that rendered view.)
Sedeffow also teaches “generating, by the one or more computing devices, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms [around the positions from the global alignment information for the plurality of acquisition locations,] and including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms”. (Para. 0061, “A ground floor of a house 201 for which a virtual map and a floor plan is to be created is shown in FIG. 2” [generating a floor plan for the house]. Para. 0235, “this cycle graph is used to render a floor plan of the room. At this step, if cycle graphs exist for other rooms that have connection points to this room, then the cycle graphs will be rendered together as a plan of a building” [generating a floor plan including fitting the generated structural layout for each of the multiple rooms]. Para. 0283, “The floor plan created by rendering the cycle graph is shown at 4601. It will be seen that the positions of the hotspots have been transformed into doorways in the floor plan” [including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms]. Fig. 46 shows both a doorway opening into a corridor and a non-doorway opening into a kitchen. Para. 0170, “The enhancements are carried out by viewer instructions 1311 on the memory 1102 of laptop 108. These instructions comprise instructions identical to instructions 2603, but include further processes. These include camera pose compensation process 2801, floor plan creation process 2802” [by the one or more computing devices]. The examiner has interpreted that carrying out instructions on the memory of a laptop such as a floor plan creation process where a floor plan for a house is rendered using cycle graphs of each room together and including doorways and non-doorway wall openings between the multiple rooms as generating, by the one or more computing devices, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms and including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms.)
Sedeffow teaches “providing, by the one or more computing devices, the generated floor plan.” (Para. 0295, “after the floor plan has been rendered at step 4708, it may be displayed to a user for editing” [providing the generated floor plan]. Para. 0170, “The enhancements are carried out by viewer instructions 1311 on the memory 1102 of laptop 108. These instructions comprise instructions identical to instructions 2603, but include further processes. These include camera pose compensation process 2801, floor plan creation process 2802” [by the one or more computing devices]. The examiner has interpreted that carrying out instructions on the memory of a laptop such as displaying the floor plan to a user for editing as providing, by the one or more computing devices, the generated floor plan.)
Sedeffow does not specifically teach “using a first neural network trained to jointly determine layout information for rooms visible in images and determine image pose information for those images within those layouts”, “to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view”, “determining, by the one or more computing devices and for each combination of two of the panorama images, one or more potential local alignments between the visual coverages of those two panorama images, including matching one or more structural wall elements identified in each of those two panorama images and using the matching one or more structural wall elements for at least one of the one or more potential alignments”, “validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images”, “generating, by the one or more computing devices, global alignment information that includes positions for the plurality of acquisition locations in a common coordinate system, including further validating and retaining some of the validated potential local alignments for some of the combinations of two panorama images based at least in part on determined alignment scores associated with that some validated potential local alignments, and including discarding other determined potential local alignments that are not further validated, and including combining the further validated and retained some validated potential local alignments”, “generating, by the one or more computing devices, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms around the positions from the global alignment information for the plurality of acquisition locations, and including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms”.
However, in the same field of endeavor namely developing of floor plans for buildings, Gausebeck teaches “using a first neural network trained to jointly determine layout information for rooms visible in images and determine image pose information for those images within those layouts”. (Para. 0038, “deriving the 3D information using a neural network model of the one or more neural network models configured to infer the 3D information based on the 2D image and the depth information.” The examiner has interpreted that using a using a neural network model to infer the 3D information based on the 2D image and the depth information as using a first neural network trained to jointly determine layout information for rooms visible in images and determine image pose information for those images within those layouts.)
Gausebeck teaches “to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view”. (Para. 0070, “a 3D model of an interior environment of building can comprise mesh data (e.g., a triangle mesh, a quad mesh, a parametric mesh, etc.), one or more texture-mapped meshes” [texture-mapping]. Para. 0072, “model generation component 118 can use depth information for respective pixels, superpixels, features, etc., derived for the 2D image to generate a 3D point cloud, 3D mesh, or the like corresponding to the respective pixels in 3D. The 3D model generation component 118 can further register visual data of the respective pixels, superpixels, features, etc. (e.g., color, texture, luminosity, etc.) with their corresponding geometric points in 3D” [to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view]. The examiner has interpreted that generating a 3D model based on the image data and its associated derived depth data where the color of the respective pixels are registered on the corresponding geometric points in 2D as to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view.)
Gausebeck teaches “determining, by the one or more computing devices and for each combination of two of the panorama images, one or more potential local alignments between the visual coverages of those two panorama images, including matching one or more structural wall elements identified in each of those two panorama images and using the matching one or more structural wall elements for at least one of the one or more potential alignments” (Para. 0102, “the one or more panorama models 514 can be configured to receive pano-image data 502 that is in the form of an equirectangular projection or has otherwise been projected onto a 2D plane” [panorama images in equirectangular format]. Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [determining, for each combination of two of the images, one or more potential local alignments between the visual coverages of those two images]. “The alignment data can also include for example, information mapping respective pixels, superpixels, objects, features, etc., represented in the image data” [including matching elements identified in each of those two images and using the matching one or more elements for at least one of the one or more potential alignments]. Para. 0079, “the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.)” [structural wall elements]. Para. 0046, “the 2D images can comprise panoramic color images” [panorama images]. Para. 0031, “Various elements described in connection with the disclosed techniques can be embodied in computer implemented system or device and/or a different form such as a computer-implemented method, a computer program product, or another form, (and vice versa)” [by the one or more computing devices]. The examiner has interpreted that performing an alignment process through the use of a computer device by aligning 2D panorama images in equirectangular format to one another from the image data by the mapping of objects and features in the images where the features are windows and doors as determining, by the one or more computing devices and for each combination of two of the panorama images, one or more potential local alignments between the visual coverages of those two panorama images, including matching one or more structural wall elements identified in each of those two panorama images and using the matching one or more structural wall elements for at least one of the one or more potential alignments.)
Gausebeck teaches “validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images”. (Para. 0118, “the 3D-from-2D convolutional neural network accounts for weighted values applied to respective pixels based on their projected angular area during training” [using a second neural network that is a convolutional neural network trained to determine local alignments between visual data]. Para. 0070, “the 3D model generation component 118 can employ the derived 3D data 116 for respective images received by the computing device 104 to generate reconstructed 3D models of objects or environments included in the images” [included in rendered views of two images]. Para. 0075, “the alignment process can involve determining position information (e.g., relative to a 3D coordinate space) and visual feature information for respective points in received 2D images relative to one another a common 3D coordinate space. In this regard, the 2D images, derived 3D data respectively associated with the 2D images, visual feature data mapped to the derived 3D data geometry” [based at least in part on comparing information for the rendered views from those two panorama images]. Para. 0075, “The model generation component 118 can further evaluate the potential alignments for their quality, and once an alignment of sufficiently high relative or absolute quality is achieved, the 2D images may be aligned together” [validating and for each of the combinations of two panorama images one of the determined one or more potential local alignments for that combination of two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images]. Para 0102, “the one or more panorama models 514 can be configured to receive pano-image data 502 that is in the form of an equirectangular projection or has otherwise been projected onto a 2D plane” [equirectangular format]. Para. 0076, “the 3D model generation component 118 can also employ sets of aligned 2D image data and/or associated 3D data to generate various representations of a 3D model of the environment or object from different perspectives or views” [perspective views, rendered views of equirectangular format]. Para. 0191, “the capture device 1901 can also provide the auxiliary data to the user device 1902 to facilitate the alignment process in association with generation of a 3D model based on the image data and its associated derived depth data” [by the one or more computing devices]. The examiner has interpreted that using an user device, a 3D-from-2D convolutional neural network that accounts for weighted values applied to respective pixels based on their projected angular area during training, and a model generation component to generate reconstructed 3D models of objects or environments included in the images, align visual feature information between the images, mapped the aligned features, and evaluate the potential alignments for their quality if the alignment of sufficiently high relative or absolute quality from equirectangular projection in a perspective view is achieved as validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images.)
Gausebeck teaches “generating, by the one or more computing devices, global alignment information that includes positions for the plurality of acquisition locations in a common coordinate system, including further validating and retaining some of the validated potential local alignments for some of the combinations of two panorama images based at least in part on determined alignment scores associated with that some validated potential local alignments, and including discarding other determined potential local alignments that are not further validated, and including combining the further validated and retained some validated potential local alignments”. (Para. 0075, “The alignment process can involve iteratively aligning different point clouds from neighboring and overlapping images captured from different positions and orientations relative to an object or environment to generate a global alignment between the respective point clouds using correspondences in derived position information for the respective points” [generating global alignment information]. Para. 0129, “auxiliary data can be used by the 3D model generation component 118 to facilitate aligning images, (and with their associated derived 3D data 116), captured at different capture positions and/or orientations relative to one another in a three-dimensional coordinate space” [that includes positions for the plurality of acquisition locations in a common coordinate system]. Para. 0075, “The model generation component 118 can further evaluate the potential alignments for their quality” [including further validating], “and once an alignment of sufficiently high relative or absolute quality is achieved” [retaining some of the validated potential local alignments for some of the combinations of two panorama images based at least in part on determined alignment scores associated with that some validated potential local alignments, and including discarding other determined potential local alignments that are not further validated], the 2D images may be aligned together” [including combining the further validated and retained some validated potential local alignments]. Para. 0191, “the capture device 1901 can also provide the auxiliary data to the user device 1902 to facilitate the alignment process in association with generation of a 3D model based on the image data and its associated derived depth data” [by the one or more computing devices]. The examiner has interpreted that generating a global alignment through an user device between the respective point clouds using correspondences in derived position information for the respective points for captured at different capture positions and relative to one another in a three-dimensional coordinate space, further evaluating the potential alignments for their quality, and only aligning the images that have a high quality as generating, by the one or more computing devices, global alignment information that includes positions for the plurality of acquisition locations in a common coordinate system, including further validating and retaining some of the validated potential local alignments for some of the combinations of two panorama images based at least in part on determined alignment scores associated with that some validated potential local alignments, and including discarding other determined potential local alignments that are not further validated, and including combining the further validated and retained some validated potential local alignments.)
Gausebeck teaches “generating, by the one or more computing devices, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms around the positions from the global alignment information for the plurality of acquisition locations, and including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms”. (Para. 0081, “the 3D models and the various representations of the 3D models capable of being generated by the 3D model generation component 118 (e.g., different views of the 3D model, a floorplan model in 2D or 3D, etc.), and/or associated aligned 2D and 3D data” [generating a floor plan]. Gausebeck Para. 0082, “3D data can be derived for the respective images and used to align them to generate a 3D model of the environment” [generating a floor plan around the positions from the global alignment information for the plurality of acquisition locations]. The examiner has interpreted that generating a floorplan model with associated aligned data as generating, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms around the positions from the global alignment information for the plurality of acquisition locations.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “using a first neural network trained to jointly determine layout information for rooms visible in images and determine image pose information for those images within those layouts”, “to texture map RGB pixel values from the RGB pixel data of that panorama image to other pixels of that rendered view”, “determining, by the one or more computing devices and for each combination of two of the panorama images, one or more potential local alignments between the visual coverages of those two panorama images, including matching one or more structural wall elements identified in each of those two panorama images and using the matching one or more structural wall elements for at least one of the one or more potential alignments”, “validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images”, “generating, by the one or more computing devices, global alignment information that includes positions for the plurality of acquisition locations in a common coordinate system, including further validating and retaining some of the validated potential local alignments for some of the combinations of two panorama images based at least in part on determined alignment scores associated with that some validated potential local alignments, and including discarding other determined potential local alignments that is are further validated, and including combining the further validated and retained some validated potential local alignments”, “generating, by the one or more computing devices, a floor plan for the house, including fitting the generated structural layout for each of the multiple rooms around the positions from the global alignment information for the plurality of acquisition locations, and including aligning the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of determining alignments between images and creating a floor plan based on the alignments for the advantageous purpose of providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129), allowing of the real-time alignment of captured imagery (Gausebeck Para. 0004), and generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
As per claim 2, Sedeffow teaches “analyzing, by the one or more computing devices and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to determine depth information for that panorama image, including estimating monocular depth from the acquisition location for that panorama image to surrounding structural elements of the room in which that panorama image was captured”. (Para. 0170, “The enhancements are carried out by viewer instructions 1311 on the memory 1102 of laptop 108” [analyzing, by the one or more computing devices]. Para. 0175, “However, the image contains no data or coordinates relating to whether particular areas of the image relate to a floor, a wall, a ceiling, etc. Each pixel is merely a point on a sphere and no further information is available. Further, each pixel is considered to be the same distance away from the centre of the notional sphere, and therefore no information is available relating to distances within the physical space shown in image 2908” [using only the RGB pixel data]. Para. 0176, “The invention as described herein provides a method of converting the spherical coordinates of a cursor point to two-dimensional Cartesian coordinates on a specific plane within the sphere. Thus, for example, if the floor is a plane of the sphere, then spherical coordinates can be converted to two-dimensional Cartesian coordinates on the floor, thus providing a floor plan of the room. This will be described further with reference to FIGS. 38 to 48.” Para. 0241, “FIG. 39 shows interface 2901, in which image 2908 is shown in the editing area” [the RGB pixel data of that one panorama image]. Para. 0245, “The user selects a corner of the room by moving the cursor to a corner that is visible in image 2908” [using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image]. Para. 0182, “These two-dimensional Cartesian coordinates represent an actual location on the floor in the physical space imaged, for a point in image 2908 having the specified spherical coordinates” [determine depth information for that panorama image, including estimating monocular depth from the acquisition location for that panorama image]. Para. 0192, “a location in a Cartesian coordinate system can be calculated on the plane at the point where it intersects with the line, thereby identifying the location of the point on the surface” [to surrounding structural elements of the room in which that panorama image was captured]. The examiner has interpreted that carrying out instructions on the memory of a laptop using only the visible pixel points in the image to identify points on the surfaces to determine the location of the points on the surface and respective distance where the space was imaged as analyzing, by the one or more computing devices and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to determine depth information for that panorama image, including estimating monocular depth from the acquisition location for that panorama image to surrounding structural elements of the room in which that panorama image was captured.)
Sedeffow does not specifically teach “performing, by the one or more computing devices and for each rendered view of each of the panorama images, the rendering of that rendered view using the determined depth information for texture mapping the some RGB pixel data included in that rendered view to corresponding pixel positions within that rendered view.”
However, Gausebeck teaches “performing, by the one or more computing devices and for each rendered view of each of the panorama images, the rendering of that rendered view using the determined depth information for texture mapping the some RGB pixel data included in that rendered view to corresponding pixel positions within that rendered view.” (Para. 0191, “the capture device 1901 can also provide the auxiliary data to the user device 1902 to facilitate the alignment process in association with generation of a 3D model based on the image data and its associated derived depth data” [by the one or more computing devices]. Para. 0070, “a 3D model of an interior environment of building can comprise mesh data (e.g., a triangle mesh, a quad mesh, a parametric mesh, etc.), one or more texture-mapped meshes” [texture-mapping]. Para. 0072, “model generation component 118 can use depth information for respective pixels, superpixels, features, etc., derived for the 2D image to generate a 3D point cloud, 3D mesh, or the like corresponding to the respective pixels in 3D. The 3D model generation component 118 can further register visual data of the respective pixels, superpixels, features, etc. (e.g., color, texture, luminosity, etc.) with their corresponding geometric points in 3D” [performing, for each rendered view of each of the panorama images, the rendering of that rendered view using the determined depth information for texture mapping the some RGB pixel data included in that rendered view to corresponding pixel positions within that rendered view]. Para. 0057, “A representation of a 3D model can include 2D image data” [2-D rendering]. The examiner has interpreted that generating a 3D model through a user device based on the image data and its associated derived depth data where the color of the respective pixels are registered on the corresponding geometric points in 2D as performing, by the one or more computing devices and for each rendered view of each of the panorama images, the rendering of that rendered view using the determined depth information for texture mapping the some RGB pixel data included in that rendered view to corresponding pixel positions within that rendered view.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “performing, by the one or more computing devices and for each rendered view of each of the panorama images, the rendering of that rendered view using the determined depth information for texture mapping the some RGB pixel data included in that rendered view to corresponding pixel positions within that rendered view” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of including the pixel data on the rendered view for the advantageous purpose of showing depth as with respect to each pixel in the image (Gausebeck Para 0071-0073) and providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
As per claim 3, Sedeffow teaches “wherein the at least one additional type of information overlaid on the some RGB pixel data included in the rendered views for each panorama image includes the location of the at least one doorway or non-doorway wall opening determined for that panorama image and includes the additional location of the at least one window determined for that panorama image.” (FIG. 14 and FIG. 17 show 2D perspective views of the captured panorama images and further show additional information such as the locations of door and windows in this rendered view. Para. 0224, “Once the distance has been calculated, then at step 3605 the coordinates on the x-axis and the y-axis can be calculated using equations 3110 and 3111” [the determined position of the acquisition location for that panorama image]. The examiner has interpreted that showing the locations of doors and windows from the calculated coordinates as wherein the at least one additional type of information overlaid on the some RGB pixel data included in the rendered views for each panorama image includes the location of the at least one doorway or non-doorway wall opening determined for that panorama image and includes the additional location of the at least one window determined for that panorama image.)
Sedeffow does not specifically teach “analyzing, by the one or more computing devices and using a third neural network trained to segment rooms visible in images into structural wall elements, and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to generate further information about the room in which that panorama image was captured that includes the identified structural wall elements of that room, wherein the identified structural wall elements include a determined location of at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one additional determined location of at least one window for that room” and “wherein the aligning of the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms includes using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms”.
However, Gausebeck teaches “analyzing, by the one or more computing devices and using a third neural network trained to segment rooms visible in images into structural wall elements, and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to generate further information about the room in which that panorama image was captured that includes the identified structural wall elements of that room, wherein the identified structural wall elements include a determined location of at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one additional determined location of at least one window for that room”. (Para. 0191, “the capture device 1901 can also provide the auxiliary data to the user device 1902 to facilitate the alignment process in association with generation of a 3D model based on the image data and its associated derived depth data” [by the one or more computing devices]. Para. 0160, “The semantic labeling component 928 can further assign labels to the recognized objects identifying the object. In some implementations, the semantic labeling component 928 can also perform semantic segmentation and further identify and defined boundaries of recognized objects in the 2D images” [segment rooms visible in images into structural wall elements, and for each of the panorama images, to generate further information about the room in which that panorama image was captured that includes the identified structural wall elements of that room]. FIG. 8 and FIG. 9 show that the semantic labeling component 928 is a component of the 3D-from-2D processing module 804 which uses neural networks [i.e., using a third neural network trained]. Para. 0067, “Non-parameter algorithms learn depth from a single RGB image” [using only the RGB pixel data of that panorama image]. Para. 0079, “the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.)” [wherein the identified structural wall elements include at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one window for that room]. Para. 0079, “floorplan model can comprise a series of lines in 3D space which represent intersections of walls and/or floors, outlines of doorways and/or windows, edges of steps, outlines of other objects of interest” [a determined location of at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one additional determined location of at least one window for that room]. The examiner has interpreted that the generation of a 3D model by 3D-from-2D processing module on an user device to perform semantic segmentation, defined boundaries of recognized objects in the 2D images from learned depth of single RGB image, identify features of the structure being doors and windows, and providing these features in floor plans as analyzing, by the one or more computing devices and using a third neural network trained to segment rooms visible in images into structural wall elements, and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to generate further information about the room in which that panorama image was captured that includes the identified structural wall elements of that room, wherein the identified structural wall elements include a determined location of at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one additional determined location of at least one window for that room.)
Gausebeck also teaches “wherein the aligning of the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms includes using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms” (Para. 0079, “the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.)” [identified doorways and non-doorway wall openings]. Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [wherein the aligning of the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms includes using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms]. Para. 0079, “floorplan model can comprise a series of lines in 3D space which represent intersections of walls and/or floors, outlines of doorways and/or windows, edges of steps, outlines of other objects of interest” [using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms]. The examiner has interpreted that the generation of a 3D model on an user device to align images in a common 3D coordinate space from identified features of the structure being doors and windows and providing these features in floor plans as wherein the aligning of the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms includes using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “analyzing, by the one or more computing devices and using a third neural network trained to segment rooms visible in images into structural wall elements, and for each of the panorama images using only the RGB pixel data of that panorama image, the RGB pixel data of that panorama image to generate further information about the room in which that panorama image was captured that includes the identified structural wall elements of that room, wherein the identified structural wall elements include a determined location of at least one doorway or non-doorway wall opening for that room on the generated structural layout of that room and that further includes at least one additional determined location of at least one window for that room” and “wherein the aligning of the fitted generated structural layouts based on identified doorways and non-doorway wall openings between the multiple rooms includes using the determined location of the at least one doorway or non-doorway wall opening for each of the multiple rooms” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of segmenting the rooms based on the identified elements and aligning images based on the elements for the advantageous purpose of generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062) and providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
As per claim 4, Sedeffow does not specifically teach “wherein the combining of the validated some local alignment information further includes generating one or more groups each having at least three acquisition locations that are all inter-connected via the validated some local alignment information, performing rotation averaging to estimate directions between the at least three acquisition locations in the common coordinate system, and performing one or more checks on the estimated directions to confirm that the estimated directions between the at least three acquisition locations are consistent.”
However, Gausebeck teaches “wherein the combining of the validated some local alignment information further includes generating one or more groups each having at least three acquisition locations that are all inter-connected via the validated some local alignment information, performing rotation averaging to estimate directions between the at least three acquisition locations in the common coordinate system, and performing one or more checks on the estimated directions to confirm that the estimated directions between the at least three acquisition locations are consistent.” (Para. 0108, “In this regard” [i.e., with respect to the rotation of images], “the stitching component 508 can be configured to align or “stitch together” respective 2D images providing different perspectives of a same environment” [generating one or more groups having acquisition locations that are all inter-connected via the validated some local alignment information]. Para. 0108, “the stitching component 508 can also employ known or derived (e.g., using techniques described herein) information regarding the capture positions and orientations of the respective 2D images to align and order the respective 2D images relative to one another, and then merge or combine the respective images” [wherein the combining of the validated some local alignment information further]. Para. 0108, “the stitching component 508 can be configured to align or “stitch together” respective 2D images providing different perspectives of a same environment” [same acquisition location] Para. 0108, “By combining two or more 2D images” [at least three acquisition locations]. Para. 0160, “the pre-processing component 926 can also use position and/or orientation information about the relative positions and/or orientations from which the input images were captured to rotate the input images prior to input to an augmented neural network model so that the direction of motion between them is horizontal” [performing rotation averaging, and performing one or more checks on the estimated directions to confirm that the estimated directions between the at least three acquisition locations are consistent]. Para. 0138, “The orientation estimation component 912 can be configured to determine or estimate the capture orientation or pitch of a 2D image and/or a relative orientation/pitch of the 2D to a common 3D coordinate space” [to estimate directions between the at least three acquisition locations in the common coordinate system]. The examiner has interpreted that aligning and combining more than two respective images based on providing different perspectives of a same environment that are rotated to provide a horizontal orientation between all the images and determine this orientation and pitch of the images with respect to a common coordinate space as wherein the combining of the validated some local alignment information further includes generating one or more groups each having at least three acquisition locations that are all inter-connected via the validated some local alignment information, performing rotation averaging to estimate directions between the at least three acquisition locations in the common coordinate system, and performing one or more checks on the estimated directions to confirm that the estimated directions between the at least three acquisition locations are consistent.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein the combining of the validated some local alignment information further includes generating one or more groups each having at least three acquisition locations that are all inter-connected via the validated some local alignment information, performing rotation averaging to estimate directions between the at least three acquisition locations in the common coordinate system, and performing one or more checks on the estimated directions to confirm that the estimated directions between the at least three acquisition locations are consistent” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of aligning and combined images that are in the same orientation for the advantageous purpose of providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0108 & 0110). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 5, it is a method claim, having similar limitations of claim 1. Thus, claim 5 is also rejected under the similar rationale as cited in the rejection of claim 1.
Re Claim 6, it is a method claim, having similar limitations of claim 1. Thus, claim 6 is also rejected under the similar rationale as cited in the rejection of claim 1.
Sedeffow does not specifically teach “wherein the plurality of image pairs includes at least a first image pair having two panorama images captured in different first and second rooms but having overlapping visual coverage through at least one of a doorway or a non-doorway wall opening of at least one of first and second rooms, and a second image pair having two panorama images captured in different third and fourth rooms but lacking any overlapping visual coverage, and a third image pair having two panorama images captured in different fifth and sixth rooms but lacking any overlapping visual coverage”.
Furthermore, regarding claim 6, Gausebeck teaches “wherein the plurality of image pairs includes at least a first image pair having two panorama images captured in different first and second rooms but having overlapping visual coverage through at least one of a doorway or a non-doorway wall opening of at least one of first and second rooms, and a second image pair having two panorama images captured in different third and fourth rooms but lacking any overlapping visual coverage, and a third image pair having two panorama images captured in different fifth and sixth rooms but lacking any overlapping visual coverage”. (Para. 0141, “related 2D images can include two or more images respectively captured by two or more cameras with partially overlapping fields-of-view or different perspective of an environment” [having overlapping visual coverage]. Para. 0141, “the related 2D images can include images that form a stereo-image pair” [a first image pair having two panorama images having overlapping visual coverage]. Para. 0037, “the first 2D image and the one or more second 2D images were captured in association with movement of a capture device to different positions relative to an environment” [different rooms]. Para. 0073, “The alignment data can also include for example, information mapping respective pixels, superpixels, objects, features, etc., represented in the image data” [matching elements through overlapping coverage]. Para. 0079, “the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.)” [doorway or a non-doorway wall opening of at least one of first and second rooms]. Para. 0141, “The related 2D images can also include images captured by two or more different cameras that are not arranged as a stereo pair” [e.g., a second image pair having two panorama images lacking any overlapping visual coverage, a third image pair having two panorama images lacking any overlapping visual coverage]. The examiner has interpreted that related 2D images can include two or more images with partially overlapping fields-of-view or different perspective associated with different positions in an environment where features being doors and windows are used to align the images to create a stereo pair and similarly create a pair for to images that are not a stereo pair as wherein the plurality of image pairs includes at least a first image pair having two panorama images captured in different first and second rooms but having overlapping visual coverage through at least one of a doorway or a non-doorway wall opening of at least one of first and second rooms, and a second image pair having two panorama images captured in different third and fourth rooms but lacking any overlapping visual coverage, and a third image pair having two panorama images captured in different fifth and sixth rooms but lacking any overlapping visual coverage.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein the plurality of image pairs includes at least a first image pair having two panorama images captured in different first and second rooms but having overlapping visual coverage through at least one of a doorway or a non-doorway wall opening of at least one of first and second rooms, and a second image pair having two panorama images captured in different third and fourth rooms but lacking any overlapping visual coverage, and a third image pair having two panorama images captured in different fifth and sixth rooms but lacking any overlapping visual coverage” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of creating pairs for images with and without overlapping coverage for the advantageous purpose of generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062) and providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 7, it is a method claim, having similar limitations of claim 2. Thus, claim 7 is also rejected under the similar rationale as cited in the rejection of claim 2.
Re Claim 8, it is a method claim, having similar limitations of claim 3. Thus, claim 8 is also rejected under the similar rationale as cited in the rejection of claim 3.
Re Claim 9, it is a method claim, having similar limitations of claim 4. Thus, claim 9 is also rejected under the similar rationale as cited in the rejection of claim 4.
Re Claim 10, it is an articles of manufacture claim, having similar limitations of claim 1. Thus, claim 10 is also rejected under the similar rationale as cited in the rejection of claim 1.
Furthermore, regarding claim 10, Sedeffow teaches “A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations” and “two or more of the panorama images”: (Para. 0168, “In order to carry out these enhancements, it is not necessary, except in certain situations that will be detailed, for the user to have created a virtual map as previously described” [e.g., automatically create a virtual map]. Para. 0169, “Enhancements to the images in spherical format are carried out, in this embodiment, after the user has created a virtual map at step 406. However, they could be carried out at any time after a user has uploaded an image at step 402” [analyzing, after the obtaining images]. Para. 0170, “some or all of the enhancement processes could be run on virtual map server 107, particularly if they are to be applied automatically without user input” [automatically perform enhancement processes (i.e., to perform automated operations)]. Para. 0008, “there is provided a non-transitory computer-readable medium with computer executable instruction”. Para. 0062-0063, “a method of linking a set of panoramic photographs. A panoramic photograph is a rectangular photograph that represents a 360° view of a space. In the embodiment described herein, omnidirectional or spherical photographs are used, which also show the floor and ceiling. However, in other embodiments, basic panoramic photographs could be used. Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [two or more of the panorama images]. The examiner has interpreted that running the enhancement processes automatically on a server with a non-transitory computer-readable medium and linking set of panoramic photographs where two photographs of a room are taken as a non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations and wherein two or more of the panorama images.)
Re Claim 11, it is an articles of manufacture claim, having similar limitations of claim 1. Thus, claim 11 is also rejected under the similar rationale as cited in the rejection of claim 1.
Furthermore, regarding claim 11, Sedeffow teaches “wherein the two or more panorama images include all of the plurality of panorama images”. Para. 0062-0063, “a method of linking a set of panoramic photographs. A panoramic photograph is a rectangular photograph that represents a 360° view of a space. In the embodiment described herein, omnidirectional or spherical photographs are used, which also show the floor and ceiling. However, in other embodiments, basic panoramic photographs could be used. Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [wherein the two or more panorama images include all of the plurality of panorama images]. The examiner has interpreted that linking set of panoramic photographs where two photographs of a room are taken as wherein the two or more panorama images include all of the plurality of panorama images.)
However, Sedeffow does not teach “wherein the providing of the generated floor plan includes transmitting, by the one or more computing devices and to one or more client devices over one or more networks, the generated floor plan to cause display of the generated floor plan on the one or more client devices”.
Furthermore, regarding claim 11, Gausebeck teaches “wherein the providing of the generated floor plan includes transmitting, by the one or more computing devices and to one or more client devices over one or more networks, the generated floor plan to cause display of the generated floor plan on the one or more client devices”. (Para. 0172, “Systems 100, 500, 800, and 1300 discussed above respectively depict an architecture wherein 2D image data, and optionally auxiliary data associated with the 2D image data, is received and processed by a universal computing device (e.g., computing device 104) to generate derived depth data for the 2D images, generate 3D reconstructed models and/or facilitate navigation of the 3D reconstructed models” [wherein the providing of the generated floor plan]. “For example, universal computing device can be or correspond to a server device, a client device, a virtual machine, a cloud computing device, etc. Systems 100, 500, 800, and 1300 further include a user device 130 configured to receive and display the reconstructed models” [transmitting the generated floor plan to cause display of the generated floor plan on the one or more client devices]. Para. 0194, “the user device 2002 and the server device 2003 can be configured to operate in a server-client relationship, wherein the server device 2003 provides services and information to the user device 2002, including various 3D modeling services provided by the 3D model generation component 118 and navigation services provided by the navigation component 126 that facilitate navigating 3D models as displayed at the user device 2002. The respective devices can communicate with one another via one or more wireless communication networks” [transmitting, by the one or more computing devices and to one or more client devices over one or more networks]. The examiner has interpreted that having an user device that receives and displays reconstruction models from a serve device that is communicated over a wireless communication network as wherein the providing of the generated floor plan includes transmitting, by the one or more computing devices and to one or more client devices over one or more networks, the generated floor plan to cause display of the generated floor plan on the one or more client devices.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein the providing of the generated floor plan includes transmitting, by the one or more computing devices and to one or more client devices over one or more networks, the generated floor plan to cause display of the generated floor plan on the one or more client devices” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of sending the model to another device for the advantageous purpose of allowing access and user input on model from remote locations (Gausebeck Para. 0087). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 12, it is an articles of manufacture claim, having similar limitations of claim 1 and claim 6. Thus, claim 12 is also rejected under the similar rationale as cited in the rejection of claim 1 and claim 6.
Re Claim 13, it is an articles of manufacture claim, having similar limitations of claim 1 and claim 6. Thus, claim 13 is also rejected under the similar rationale as cited in the rejection of claim 1 and claim 6.
Re Claim 14, it is an articles of manufacture claim, having similar limitations of claim 1 and claim 6. Thus, claim 14 is also rejected under the similar rationale as cited in the rejection of claim 1 and claim 6.
Re Claim 15, it is an articles of manufacture claim, having similar limitations of claim 2. Thus, claim 15 is also rejected under the similar rationale as cited in the rejection of claim 2.
Re Claim 16, it is an articles of manufacture claim, having similar limitations of claim 3. Thus, claim 16 is also rejected under the similar rationale as cited in the rejection of claim 3.
As per claim 17, Sedeffow teaches “wherein the information included in at least one of the one or more rendered views for each of the panorama images includes overlaid information to indicate one or more locations of at least one type of object or surface of the one or more rooms that are included in the visual coverage of that panorama image and that are identified based only on analysis of the RGB color pixel data of that panorama image.” (Para. 0175, “However, the image contains no data or coordinates relating to whether particular areas of the image relate to a floor, a wall, a ceiling, etc. Each pixel is merely a point on a sphere and no further information is available. Further, each pixel is considered to be the same distance away from the centre of the notional sphere, and therefore no information is available relating to distances within the physical space shown in image 2908” [identified based only on analysis of the color pixel data of that panorama image]. Para. 0110, “The conversion of an equirectangular image 601 to a circular projection 1701 is shown in FIG. 17” [generating views that are rendered]. FIG. 14 and FIG. 17 show 2D perspective views of the captured panorama images and further show additional information such as surfaces of the room in this rendered view. Para. 0192, “a location in a Cartesian coordinate system can be calculated on the plane at the point where it intersects with the line, thereby identifying the location of the point on the surface” [to indicate one or more locations of at least one type of surface of the one or more rooms that are included in the visual coverage of that panorama image]. The examiner has interpreted that converting the panorama image from an equirectangular image to a circular projection in a 2D format from the each of the imported RGB pixel information showing surfaces of the room using only the visible pixel points in the image and identifying the location of the point on the surface as wherein the information included in at least one of the one or more rendered views for each of the panorama images includes overlaid information to indicate one or more locations of at least one type of object or surface of the one or more rooms that are included in the visual coverage of that panorama image and that are identified based only on analysis of the color pixel data of that panorama image.)
Re Claim 18, it is an articles of manufacture claim, having similar limitations of claim 2 and claim 3. Thus, claim 18 is also rejected under the similar rationale as cited in the rejection of claim 2 and claim 3.
Re Claim 23, it is an articles of manufacture claim, having similar limitations of claim 3. Thus, claim 23 is also rejected under the similar rationale as cited in the rejection of claim 3.
Re Claim 24, it is an articles of manufacture claim, having similar limitations of claim 4. Thus, claim 23 is also rejected under the similar rationale as cited in the rejection of claim 4.
As per claim 25, Sedeffow does not specifically teach “wherein the structural layout generated for each of the two or more panorama images from the analyzing of the RGB color pixel data of that panorama image is a generated three-dimensional shape of the at least one room visible in that panorama image, and wherein the determining of the local alignment information between the acquisition locations of the two panorama images for each of the one or more image pairs includes comparing the generated three-dimensional shapes for those two panorama images.”
However, Gausebeck teaches “wherein the structural layout generated for each of the two or more panorama images from the analyzing of the RGB color pixel data of that panorama image is a generated three-dimensional shape of the at least one room visible in that panorama image, and wherein the determining of the local alignment information between the acquisition locations of the two panorama images for each of the one or more image pairs includes comparing the generated three-dimensional shapes for those two panorama images.” (Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [generated three-dimensional shape of the at least one room visible in that panorama image and comparing the generated three-dimensional shapes for those two panorama images]. Para. 0079, “floorplan model can comprise a series of lines in 3D space which represent intersections of walls and/or floors, outlines of doorways and/or windows, edges of steps, outlines of other objects of interest” [acquisition locations of the two panorama images for each of the one or more image pairs]. The examiner has interpreted that the generation of a 3D model to align images in a common 3D coordinate space from identified features of the structure being doors and windows and providing these features in respective locations in floor plans as wherein the structural layout generated for each of the two or more panorama images from the analyzing of the RGB color pixel data of that panorama image is a generated three-dimensional shape of the at least one room visible in that panorama image, and wherein the determining of the local alignment information between the acquisition locations of the two panorama images for each of the one or more image pairs includes comparing the generated three-dimensional shapes for those two panorama images.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein the structural layout generated for each of the two or more panorama images from the analyzing of the RGB color pixel data of that panorama image is a generated three-dimensional shape of the at least one room visible in that panorama image, and wherein the determining of the local alignment information between the acquisition locations of the two panorama images for each of the one or more image pairs includes comparing the generated three-dimensional shapes for those two panorama images” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of generating layouts and making alignments of 3D object for the advantageous purpose of generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062), providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129), and viewing a 3D panoramic environment (Gausebeck Para. 0003). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 26, it is an articles of manufacture claim, having similar limitations of claim 25. Thus, claim 26 is also rejected under the similar rationale as cited in the rejection of claim 25.
Furthermore, regarding claim 26, Sedeffow does not specifically teach “wherein the fitting of the generated structural layouts for the at least one or more rooms includes combining the generated three-dimensional shapes from all of the panorama images based at least in part on at least one of doorways or non-doorway openings identified in the multiple rooms from analysis of the RGB color pixel data of the panorama images.”
However, Gausebeck teaches “wherein the fitting of the generated structural layouts for the at least one or more rooms includes combining the generated three-dimensional shapes from all of the panorama images based at least in part on at least one of doorways or non-doorway openings identified in the multiple rooms from analysis of the RGB color pixel data of the panorama images.” (Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [combining the generated three-dimensional shapes from all of the panorama images]. Para. 0079, “floorplan model can comprise a series of lines in 3D space which represent intersections of walls and/or floors, outlines of doorways and/or windows, edges of steps, outlines of other objects of interest” [based at least in part on at least one of doorways or non-doorway openings identified in the multiple rooms]. Para. 0067, “Non-parameter algorithms learn depth from a single RGB image” [from analysis of the RGB color pixel data of the panorama images]. The examiner has interpreted that the generation of a 3D model to align images in a common 3D coordinate space from identified features of the structure from learned depth of single RGB image and providing these features in respective locations in floor plans as wherein the fitting of the generated structural layouts for the at least one or more rooms includes combining the generated three-dimensional shapes from all of the panorama images based at least in part on at least one of doorways or non-doorway openings identified in the multiple rooms from analysis of the RGB color pixel data of the panorama images.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein the fitting of the generated structural layouts for the at least one or more rooms includes combining the generated three-dimensional shapes from all of the panorama images based at least in part on at least one of doorways or non-doorway openings identified in the multiple rooms from analysis of the RGB color pixel data of the panorama images” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of combing the rooms based on the identified elements and aligning images based on the elements of the room for the advantageous purpose of generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062), providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129), and viewing a 3D panoramic environment (Gausebeck Para. 0003). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
As per claim 27, Sedeffow teaches “wherein the two or more panorama images include all of the plurality of panorama images”. Para. 0062-0063, “a method of linking a set of panoramic photographs. A panoramic photograph is a rectangular photograph that represents a 360° view of a space. In the embodiment described herein, omnidirectional or spherical photographs are used, which also show the floor and ceiling. However, in other embodiments, basic panoramic photographs could be used. Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [wherein the two or more panorama images include all of the plurality of panorama images]. The examiner has interpreted that linking set of panoramic photographs where two photographs of a room are taken as wherein the two or more panorama images include all of the plurality of panorama images.)
Sedeffow does not specifically teach “wherein at least some of the panorama images each includes visual coverage for two or more rooms and the multiple types of information generated for that panorama image are for all of the two or more rooms, and wherein generating of the multiple types of information for each of the panorama includes analyzing a combination of RGB (red-green-blue) color pixel data of that panorama image and additional depth data acquired from the acquisition location of that panorama image using one or more depth-sensing devices.”
However, Gausebeck teaches “wherein at least some of the panorama images each includes visual coverage for two or more rooms and the multiple types of information generated for that panorama image are for all of the two or more rooms, and wherein generating of the multiple types of information for each of the panorama includes analyzing a combination of RGB (red-green-blue) color pixel data of that panorama image and additional depth data acquired from the acquisition location of that panorama image using one or more depth-sensing devices.” (Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [wherein at least some of the panorama images each includes visual coverage for two or more rooms and the multiple types of information generated for that panorama image are for all of the two or more rooms]. Para. 0046, “the device can also comprise a 3D sensor configured to capture depth data for a portion of the 2D images,” [additional depth data acquired from the acquisition location of that panorama image using one or more depth-sensing devices]. Para. 0064, “the 2D image data can include standard red, green, blue (RGB) image” [RGB (red-green-blue) color pixel data]. Para. 0086, “the 3D model can be viewed from various perspectives” [two or more rooms]. Para. 0038, “deriving the 3D information using a neural network model of the one or more neural network models configured to infer the 3D information based on the 2D image and the depth information” [wherein generating of the multiple types of information for each of the panorama includes analyzing a combination of RGB (red-green-blue) color pixel data of that panorama image and additional depth data]. The examiner has interpreted that the generation of a 3D model to align images in a common 3D coordinate space from respective features from various perspectives using RGB image data and depth information data captured from a 3D sensor as wherein at least some of the panorama images each includes visual coverage for two or more rooms and the multiple types of information generated for that panorama image are for all of the two or more rooms, and wherein generating of the multiple types of information for each of the panorama includes analyzing a combination of RGB (red-green-blue) color pixel data of that panorama image and additional depth data acquired from the acquisition location of that panorama image using one or more depth-sensing devices.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add “wherein at least some of the panorama images each includes visual coverage for two or more rooms and the multiple types of information generated for that panorama image are for all of the two or more rooms, and wherein generating of the multiple types of information for each of the panorama includes analyzing a combination of RGB (red-green-blue) color pixel data of that panorama image and additional depth data acquired from the acquisition location of that panorama image using one or more depth-sensing devices” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of provide multiple types of information through multiple data sources for the advantageous purpose of generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062) and providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 28, it is an articles of manufacture claim, having similar limitations of claim 2. Thus, claim 28 is also rejected under the similar rationale as cited in the rejection of claim 2.
Furthermore, regard claim 28, Sedeffow teaches “wherein the two or more panorama images include all of the plurality of panorama images” and “generate depth data from the acquisition location of that panorama image to the at least some walls of the at least one room visible in that panorama image”. (Para. 0062-0063, “a method of linking a set of panoramic photographs. A panoramic photograph is a rectangular photograph that represents a 360° view of a space. In the embodiment described herein, omnidirectional or spherical photographs are used, which also show the floor and ceiling. However, in other embodiments, basic panoramic photographs could be used. Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [wherein the two or more panorama images include all of the plurality of panorama images]. Para. 0182, “These two-dimensional Cartesian coordinates represent an actual location on the floor in the physical space imaged, for a point in image 2908 having the specified spherical coordinates” [generate depth data from the acquisition location of that panorama image]. Para. 0192, “a location in a Cartesian coordinate system can be calculated on the plane at the point where it intersects with the line, thereby identifying the location of the point on the surface” [to the at least some walls of the at least one room visible in that panorama image]. The examiner has interpreted that identified points on the surfaces to determine the location of the points on the surface and respective distance where the space was imaged and linking set of panoramic photographs where two photographs of a room are taken as wherein the two or more panorama images include all of the plurality of panorama images and generate depth data from the acquisition location of that panorama image to the at least some walls of the at least one room visible in that panorama image.)
Re Claim 29, it is an articles of manufacture claim, having similar limitations of claim 3. Thus, claim 28 is also rejected under the similar rationale as cited in the rejection of claim 3.
Furthermore, regarding claim 29, Sedeffow teaches “wherein the two or more panorama images include all of the plurality of panorama images”. Para. 0062-0063, “a method of linking a set of panoramic photographs. A panoramic photograph is a rectangular photograph that represents a 360° view of a space. In the embodiment described herein, omnidirectional or spherical photographs are used, which also show the floor and ceiling. However, in other embodiments, basic panoramic photographs could be used. Photographs are taken of each room to be included in the tour. In some situations, two photographs of a room may be appropriate” [wherein the two or more panorama images include all of the plurality of panorama images]. The examiner has interpreted that linking set of panoramic photographs where two photographs of a room are taken as wherein the two or more panorama images include all of the plurality of panorama images.)
Re Claim 30, it is an articles of manufacture claim, having the same limitations of claim 23. Thus, claim 30 is also rejected under the same rationale as cited in the rejection of claim 23.
Re Claim 31, it is a system, having similar limitations of claim 1 and 10. Thus, claim 31 is also rejected under the similar rationale as cited in the rejection of claim 1 and 10.
Furthermore, regarding claim 31, Sedeffow teaches “ A system comprising one or more hardware processors of one or more computing systems; and one or more memories with stored instructions that, when executed by at least one of the one or more hardware processors, cause the one or more computing systems to perform automated operations”. (Para. 0077, “server 701 includes a processor which in this example is CPU 704” [A system comprising one or more hardware processors of one or more computing systems], “processor memory provided by RAM 705” [and one or more memories]”. Para. 0075, “at step 804 instructions are installed from a portable storage medium” [memories with stored instructions]. Para. 0090, “At step 1206 the instructions are received from the server and the processor loads them in the browser environment” [executed by at least one of the one or more hardware processors]. Para. 0170, “some or all of the enhancement processes could be run on virtual map server 107, particularly if they are to be applied automatically without user input” [automatically perform enhancement processes (i.e., to perform automated operations)].)
Re Claim 32, it is a system, having similar limitations of claim 1 and claim 11. Thus, claim 32 is also rejected under the similar rationale as cited in the rejection of claim 1 and claim 11.
Re Claim 33, it is a system, having similar limitations of claim 1 and claim 11. Thus, claim 33 is also rejected under the similar rationale as cited in the rejection of claim 1 and claim 11.
Re Claim 34, it is a system, having similar limitations of claim 4 and claim 11. Thus, claim 34 is also rejected under the similar rationale as cited in the rejection of claim 4 and claim 11.
As per claim 35, Sedeffow teaches “wherein generating of the one or more rendered views for each of the at least some panorama images includes rendering one or more wall views in two dimensions to each represent a distinct wall of the at least one room visible in that panorama image”. (FIG. 17s show 2D perspective views of the captured panorama images and further show additional information walls for the respective rooms that are linked through features in the walls. The examiner has interpreted that generating figures which have walls of the rooms that are generated from the panorama images in two dimension that are linked through features in the walls as wherein generating of the one or more rendered views for each of the at least some panorama images includes rendering one or more wall views in two dimensions to each represent a distinct wall of the at least one room visible in that panorama image.)
Sedeffow does not specifically teach “wherein the comparing of the information for the one or more rendered views for each of those at least two panorama images in each of the image groups includes comparing at least one wall view for one of the at least two panorama images to at least one other wall view for each other of the at least two panorama images.”
However, Gausebeck teaches “wherein the comparing of the information for the one or more rendered views for each of those at least two panorama images in each of the image groups includes comparing at least one wall view for one of the at least two panorama images to at least one other wall view for each other of the at least two panorama images.” (Para. 0070, “the 3D model generation component 118 can employ the derived 3D data 116 for respective images received by the computing device 104 to generate reconstructed 3D models of objects or environments included in the images” [included in rendered views of two images]. Para. 0073, “the 3D model generation component 118 can perform an alignment process that involves aligning the 2D images and/or features in the 2D images to one another and a common 3D coordinate space, based at least in part, the derived 3D data 116 for the respective images, to generate an alignment between the image data and/or the respective features in the image data” [the comparing of the information for the one or more rendered views for each of those at least two panorama images in each of the image groups]. Para. 0079, “the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.)” [comparing at least one wall view for one of the at least two panorama images to at least one other wall view for each other of the at least two panorama images]. The examiner has interpreted that performing an alignment aligning 2D panorama images to one another from the image data by the mapping of objects and features in the images where the features wall lengths as wherein the comparing of the information for the one or more rendered views for each of those at least two panorama images in each of the image groups includes comparing at least one wall view for one of the at least two panorama images to at least one other wall view for each other of the at least two panorama images.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add teaches “wherein the comparing of the information for the one or more rendered views for each of those at least two panorama images in each of the image groups includes comparing at least one wall view for one of the at least two panorama images to at least one other wall view for each other of the at least two panorama images” as conceptually seen from the teaching of Gausebeck, into that of Sedeffow because this modification of determining comparing the walls in the images of one another for the advantageous purpose of providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129) and generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings.
Re Claim 36, it is an articles of manufacture claim, having similar limitations of claim 27. Thus, claim 36 is also rejected under the similar rationale as cited in the rejection of claim 27.
Re Claim 37, it is an articles of manufacture claim, having similar limitations of claim 2 and 27. Thus, claim 36 is also rejected under the similar rationale as cited in the rejection of claim 2 and 27.
Re Claim 38, it is an articles of manufacture claim, having similar limitations of claim 3. Thus, claim 38 is also rejected under the similar rationale as cited in the rejection of claim 3.
Re Claim 39, it is a method claim, having similar limitations of claim 4 and 11. Thus, claim 39 is also rejected under the similar rationale as cited in the rejection of claim 4 and 11.
Re Claim 40, it is a method claim, having similar limitations of claims 4, 11, and 32. Thus, claim 40 is also rejected under the similar rationale as cited in the rejection of claim 4, 11, and 32.
Re Claim 41, it is a method claim, having similar limitations of claims 4, 11, and 33. Thus, claim 41 is also rejected under the similar rationale as cited in the rejection of claim 4, 11, and 33.
Re Claim 42, it is a method claim, having similar limitations of claims 4, 11, and 34. Thus, claim 42 is also rejected under the similar rationale as cited in the rejection of claim 4, 11, and 34.
Response to Arguments
Applicant's arguments filed on April 9, 2026 have been fully considered but they are not persuasive.
Applicant argues that the combination of references does not teach each and every limitation in claim 1 because cited references fail to teach (A) “generating, by the one or more computing devices and for each of the panorama images using only information determined from the RGB pixel data of that panorama image, views that are rendered in two dimensions in a perspective or orthographic format and that include a floor view of the at least some floor of the one room in which that panorama image is captured and that include a ceiling view of the at least some ceiling of that one room, wherein each of the rendered views includes some of the RGB pixel data of that panorama image that is positioned in that rendered view based at least in part on the generated structural layout for that one room and using raycasting for pixels of that rendered view, and further includes at least one additional type of information overlaid on the some RGB pixel data included in that rendered view” (See Applicant’s response, Pg. 30-31).
MPEP § 2145(IV) recites “one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references.” Applicant’s reply fails to address the combined teaching of the applied references and instead only argues that each reference individually does not teach all of the claim limitations. One cannot show nonobviousness by attacking reference individually where the rejections are based on combinations of references.
Sedeffow discloses “generating, by the one or more computing devices and for each of the panorama images using only information determined from the RGB pixel data of that panorama image, views that are rendered in two dimensions in a perspective or orthographic format and that include a floor view of the at least some floor of the one room in which that panorama image is captured and that include a ceiling view of the at least some ceiling of that one room, wherein each of the rendered views includes some of the RGB pixel data of that panorama image that is positioned in that rendered view based at least in part on the generated structural layout for that one room and using raycasting for pixels of that rendered view, and further includes at least one additional type of information overlaid on the some RGB pixel data included in that rendered view” as carrying out instructions on the memory of a laptop using only the visible pixel points in the image to convert the panorama image from an equirectangular image to a circular projection in a 2D format from the imported RGB pixel information showing doors and windows while looking downwards to the floor or upwards to the ceiling and from the calculated coordinates that are calculated from the distance to the selected point from the center (further see Fig. 17 which includes all the features of this limitation as provided below). While the applicant argues that Sedeffow does not render a view from a panorama image in an equirectangular format, as provided above in the rejection and as seen Sedeffow Para. 0110, “The conversion of an equirectangular image 601 to a circular projection 1701 is shown in FIG. 17”, Sedeffow teaches a rendered view from a equirectangular image. Further, while the applicant argues that Sedeffow does not render a view showing a floor or ceiling in Para. 0110, this is merely one embodiment. The embodiment as shown in Fig. 17 is what the examiner has relied upon for the rejection.
[AltContent: textbox (Ceiling)][AltContent: arrow][AltContent: textbox (Floor)][AltContent: arrow]
PNG
media_image1.png
509
438
media_image1.png
Greyscale
Figure 1: Fig. 17 from Sedeffow
Therefore, all of the limitations of the amended claims 1 are disclosed in either Sedeffow and Gausebeck, and the combination of these references renders the claimed invention obvious. Therefore, applicant’s arguments are not persuasive and the rejection of claim 1 as obvious over Sedeffow in view of Gausebeck is maintained.
Applicant argues that the combination of references does not teach each and every limitation in the amend claim 1 because cited references fail to teach (B)/(C) “validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images” (See Applicant’s response, Pg. 31-34).
MPEP § 2145(IV) recites “one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references.” Applicant’s reply fails to address the combined teaching of the applied references and instead only argues that each reference individually does not teach all of the claim limitations. One cannot show nonobviousness by attacking reference individually where the rejections are based on combinations of references.
Gausebeck discloses “validating, by the one or more computing devices and using a second neural network that is a convolutional neural network trained to determine local alignments between visual data included in rendered views of two images, and for each of the combinations of two panorama images, one of the determined one or more potential local alignments for that combination of two panorama images based at least in part on comparing information for the rendered views from those two panorama images, including generating an alignment score associated with accuracy of the validated one potential local alignment for that combination of the two panorama images” as using an user device, a 3D-from-2D convolutional neural network that accounts for weighted values applied to respective pixels based on their projected angular area during training, and a model generation component to generate reconstructed 3D models of objects or environments included in the images, align visual feature information between the images, mapped the aligned features, and evaluate the potential alignments for their quality if the alignment of sufficiently high relative or absolute quality from equirectangular projection in a perspective view is achieved. While the applicant argues that Gausebeck does not discloses this validating step after the conversion from perspective view in equirectangular format to a perspective view, as provided in the rejection above in Para. 0102, “the one or more panorama models 514 can be configured to receive pano-image data 502 that is in the form of an equirectangular projection or has otherwise been projected onto a 2D plane” and Para. 0076, “the 3D model generation component 118 can also employ sets of aligned 2D image data and/or associated 3D data to generate various representations of a 3D model of the environment or object from different perspectives or views”. Since equirectangular pano-images are received and perspective views are generated in order to find the alignments, thus the claimed limitation is taught. Further, combing the teaching of Gausebeck into Sedeffow, the claimed limitation is taught also taught as Sedeffow teaches the rendering of equirectangular panorama images in perspective view.
Therefore, all of the limitations of the amended claims 1 are disclosed in either Sedeffow and Gausebeck, and the combination of these references renders the claimed invention obvious. Therefore, applicant’s arguments are not persuasive and the rejection of claim 1 as obvious over Sedeffow in view of Gausebeck is maintained.
Applicant argues that there is no motivation to modify the cited references to teach the claimed features (See Applicant’s response, Pg. 35).
The applicant rightly asserts MPEP § 2145(IV) recites “there must be some articulated reasoning with some rational underpinning to support the legal conclusion of obviousness”. MPEP § 2145(IV) also recites “Obviousness can be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so.”
The examiner provide the motivation to combine Gausebeck into Sedeffow as also seen above. The motivations include providing additional accuracy and immersion in generating a floor plan (Gausebeck Para. 0110 & 0129), allowing of the real-time alignment of captured imagery (Gausebeck Para. 0004), and generating accurate and efficient representations of the model of the room (Gausebeck Para. 0004 & 0062). Further motivation to combine be that Sedeffow and Gausebeck are analogous art to the current claim are directed to developing of floor plans for buildings. New functionality to the combination of these references is not relied upon for the teaching of the claims, only by combination of these refences of using their provided teachings such as the analysis and alignment information generated by Gausebeck into the generated structural layout of the perspective view of Sedeffow, the claimed invention is taught.
Therefore, there is motivation to combine Gausebeck into that of Sedeffow, and the examiner has established a prima facie case of obviousness. Therefore, applicant’s arguments are not persuasive and the rejection of claim 1 as obvious over Sedeffow in view of Gausebeck is maintained.
Applicant’s arguments, see Pg. 21-27, filed April 9, 2026, with respect to the rejection(s) of claims 1-18 and 23-42 under 35 U.S.C. § 101 have been fully considered and are persuasive with regards to the amended independent claims that integrate the claimed invention into a practical application. Therefore, the rejection has been withdrawn.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2023/0032888 A1 Li, Yuguang et al. teaches a method to determine the acquisition locations of panorama images, such as within a building interior based on automatically determined shapes of rooms of the building for the building's floor plan.
Examiner’s Note: The examiner has cited particular columns and line numbers in the reference that applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. In the case of amending the claimed invention, the applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for the proper interpretation and also to verify and ascertain the metes and bound of the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Simeon P Drapeau whose telephone number is (571)-272-1173. The examiner can normally be reached Monday - Friday, 8 a.m. - 5 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached on (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIMEON P DRAPEAU/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights