DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 7/12/2023 was in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Status of Claims
This action is in reply to the application filed on 4/13/2023, wherein:
Claims 1-20 are currently pending and have been examined.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-13, and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0247416 to Ruda et al. (hereinafter referred to as Ruda), in view of US 10,007,992 to Harvey et al. (hereinafter referred to as Harvey).
In regards to claim 1, Ruda discloses a method of contactless structural analysis (image capture system 100 including subject building 102 and unmanned aerial vehicle (UAV) 104 to assess damage to the roof, façade and siding of the building, paras. 0023-0026, fig. 1) on at least one visualization platform associated with a computing device (assessing damage to building 102 consists of capturing images of building 102 using a camera mounted onto UAV 104 and device 200 comprising network interfaces 210, at least one processor 220, and memory 240 storing software programs and data structures, , paras. 0028-0036, figs. 1 and 2), the method comprising the steps of: receiving, via at least one user interface associated with the at least one visual platform (user interface 900 for the system displays assessment data 314 for review by a human user, para. 0087, fig. 9), a data query from at least one user regarding at least one civil structure (UAV 104 is provided the GPS coordinates of the building and the UAV automatically captures the required images, para. 0028, figs. 1A and 1B); transmitting, via at least one processor of the computing device (device 200 with processor 220, fig. 2) communicatively coupled to at least one sensor (UAV 104 with camera, figs. 1 and 2) in mechanical communication with the at least one civil structure (Image analysis process 248 receives captured images 312 of the subject building downloaded from UAV 104 and transferred via a network to device 200 for analysis by image analysis process 248, para. 0045), at least one sensorial input to the at least one visualization platform (captured images 312 transferred to device 200 for analysis by image analysis process 248, para. 0045), the at least one sensorial input being configured (image analyzer 248 executes geometry analyzer 310 to create a three dimensional (3D) model of a roof based on the input aerial images 312 using stereophotogrammetry, para. 0062) to be displayed on the at least one visualization platform (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); generating, via the at least one visualization platform, at least one spatial model of the at least one civil structure based on the at least one received sensorial input (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); and automatically displaying the at least one generated spatial model on the at least one visualization platform associated with the computing device (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068), but fails to disclose displaying by: based on a determination that at least one tactile-input is received from the at least one user, via the at least one user-interface, generating at least one spatial alteration to the at least one generated spatial model; and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches displaying (aggregate aerial images of infrastructure to create a 2D or 3D adjustable images displayed on a client device, col. 4, lines 47-64) by: based on a determination that at least one tactile-input is received from the at least one user (client device may display a 3D aerial image of a property with damage severity level indicators overlaying each property component, col. 20, lines 50-67), via the at least one user-interface (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58), generating at least one spatial alteration to the at least one generated spatial model (damage severity level indicators may include numbers which are placed over each property component, labels, symbols, different shading techniques, etc., col. 20, lines 50-67); and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability for the user to modify the model using the interface as taught by the method of Harvey. The motivation for doing so would have been to display infrastructure images with damage indicators overlaying the image on the client device (Harvey, col. 12, lines 36-44).
In regards to claim 2, modified Ruda discloses the method of claim 1, and further discloses wherein the at least one sensor is selected from a group consisting of an accelerometer, a strain gauge, a potentiometer, a camera (camera may be a digital single-lens reflex camera, any form of charge coupled device (CCD)-based camera, infrared (IR) camera, etc., para. 0027), a UAV (camera mounted onto UAV 104, para. 0028), a LiDAR scanner, an NDT tool, an ultrasound system, an infrared camera (camera may be a digital single-lens reflex camera, any form of charge coupled device (CCD)-based camera, infrared (IR) camera, etc., para. 0027), Ground Penetrating Radar (GPR), and a combination of thereof.
In regards to claim 4, modified Ruda discloses the method of claim 3, further comprising the step of, recording, via the at least one processor of the computing device, the at least one sensorial input to a memory of the computing device (Image analysis process 248 receives captured images 312 of the subject building downloaded from UAV 104 and transferred via a network to device 200 for analysis by image analysis process 248, para. 0045).
In regards to claim 5, modified Ruda discloses the method of claim 4, but fails to disclose further comprising the step of, after recording the at least one sensorial input, assigning, via the at least one processor, a unique profile to the at least one recorded sensorial input, the at least one generated spatial model, or both associated with the at least one civil structure.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after recording the at least one sensorial input (After an aerial image 45 of an item of infrastructure is captured, the aerial image may be stored in the previous image data 94, so that it may be compared with a newly captured image of the same infrastructure item at a later date, col. 10, lines 44-48), assigning, via the at least one processor, a unique profile to the at least one recorded sensorial input (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21), associated with the at least one civil structure (server 14 may include infrastructure data (e.g., a list of items of infrastructure such as "Highway 80," "the Golden Gate Bridge," "the 'L' Station," etc.), location data (e.g., locations of the items of infrastructure, locations of portions of the items of infrastructure, etc.), previous image data ( e.g., aerial images of items of infrastructure taken at an earlier date), and financial data ( e.g., infrastructure cost estimates of property and materials similar to those that were damaged or destroyed, labor costs for repairing/replacing the infrastructure, etc.) from an infrastructure database 66, a location database 68, a previous image database 94, and a financial database 96, respectively, col. 9, lines 45-58). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability for the user to assign a profile and record image data as taught by the method of Harvey. The motivation for doing so would have been to determine a condition of an item of infrastructure as compared to its previous condition (Harvey, col. 7, lines 20-31).
In regards to claim 6, modified Ruda discloses the method of claim 5, but fails to disclose further comprising the step of, after assigning the at least one unique profile, recording at least one sensorial input from at least one alternative civil structure to the memory of the computing device.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after assigning the at least one unique profile (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21), recording at least one sensorial input (After an aerial image 45 of an item of infrastructure is captured, the aerial image may be stored in the previous image data 94, so that it may be compared with a newly captured image of the same infrastructure item at a later date, col. 10, lines 44-48) from at least one alternative civil structure to the memory of the computing device (a system may perform claims assessments for insured properties in a neighborhood by automatically surveying the entire neighborhood at once by capturing aerial images of the properties with each captured aerial image associated with a location using a GPS location, col. 15, lines 31-67). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to record images from other properties as taught by the method of Harvey. The motivation for doing so would have been to perform an automatic inspection of a neighborhood affected by a catastrophe (Harvey, col. 15, lines 20-67).
In regards to claim 7, modified Ruda discloses the method of claim 6, but fails to disclose further comprising the step of, after recording the at least one sensorial input of the at least one alternative civil structure, assigning, via the at least one processor, an alternative unique profile to the at least one recorded sensorial input, the at least one generated spatial model, or both associated with the at least one alternative civil structure.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after recording the at least one sensorial input of the at least one alternative civil structure, assigning, via the at least one processor, an alternative unique profile (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21) to the at least one generated spatial model (a system may perform claims assessments for insured properties in a neighborhood by automatically surveying the entire neighborhood at once by capturing aerial images of the properties with each captured aerial image associated with a location using a GPS location, col. 15, lines 31-67), associated with the at least one alternative civil structure (group of aerial images are combined to generate a 3D image of the property utilizing the GPS coordinates received with each aerial image, col. 17, lines 51-65). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to assign profiles to captured images of various infrastructure as taught by the method of Harvey. The motivation for doing so would have been to perform an automatic inspection of a neighborhood affected by a catastrophe (Harvey, col. 15, lines 20-67).
In regards to claim 8, modified Ruda discloses the method of claim 7, but fails to disclose further comprising the step of, receiving, via the at least one user-interface, a spatial model query regarding the at least one unique profile, the at least one alternative unique profile, or both from the at least one user, the at least one alternative user, or both, wherein upon receiving the spatial model query, the at least one processor is configured to automatically display the at least one spatial model, the at least one alternative spatial model, or both on the at least one visualization platform associated with the computing device.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches receiving, via the at least one user-interface, a spatial model query regarding the at least one unique profile from the at least one user (client device 12 may request satellite images between specified GPS coordinates, and the image capturing module 90 may transmit satellite images within the specified coordinates, col. 7, lines 1-19), wherein upon receiving the spatial model query, the at least one processor is configured to automatically display the at least one spatial model (image user interface 70 may display aerial images of an infrastructure item and may also display damage severity levels for the infrastructure item, col. 8, lines 17-26), on the at least one visualization platform associated with the computing device (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to request infrastructure data for a property using gps coordinates as taught by the method of Harvey. The motivation for doing so would have been to utilize gps coordinates as location boundaries for the identified item of infrastructure for assessing damage (Harvey, col. 14, lines 17-30).
In regards to claim 9, modified Ruda discloses the method of claim 1, further comprising the step of, after generating the at least one spatial model, overlaying, via the at least one processor, the at least one sensorial input onto the at least one generated spatial model (a label may be applied to a given portion of the image ( e.g., a colored box, etc.) that indicates whether damage is present and the type of damage, para. 0079).
In regards to claim 10, modified Ruda discloses the method of claim 9, but fails to disclose further comprising the step of, selecting, via the at least one user-interface, at least one portion of the at least one generated model, wherein upon receiving the selection, the at least one processor is configured to automatically display the at least one overlayed sensorial input associated with the selected portion of the at least one generated model on the at least one visualization platform associated with the computing device.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches selecting, via the at least one user-interface, at least one portion of the at least one generated model (client device 12 may request satellite images between specified GPS coordinates, and the image capturing module 90 may transmit satellite images within the specified coordinates, col. 7, lines 1-19), wherein upon receiving the selection (image user interface 70 may display aerial images of an infrastructure item and may also display damage severity levels for the infrastructure item, col. 8, lines 17-26), the at least one processor is configured to automatically display the at least one overlayed sensorial input associated with the selected portion of the at least one generated model (at block 606 aerial images may be received and aggregated to form a 3D display of the infrastructure item, col. 14, lines 31-44) on the at least one visualization platform associated with the computing device (at block 608 the infrastructure evaluation module 72 may determine the condition of the infrastructure item based on the aerial images and determine a damage severity level score to assign to the infrastructure item that is overlayed on the aerial images of the infrastructure on a display of the client device, col. 14, line 45 – col. 15, line 19). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to overlay input in the infrastructure model as taught by the method of Harvey. The motivation for doing so would have been to place damage severity level indicators over the aerial images of the infrastructure item and determine a cost estimate for repair (Harvey, col. 15, lines 1-19).
In regards to claim 11, modified Ruda discloses the method of claim 9, but fails to disclose further comprising the step of, after overlaying the at least one sensorial input onto the at least one generated spatial model, displaying the at least one generated spatial model within a background scene comprising the at least one civil structure’s real environment.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after overlaying the at least one sensorial input onto the at least one generated spatial model (at block 608 the infrastructure evaluation module 72 may determine the condition of the infrastructure item based on the aerial images and determine a damage severity level score to assign to the infrastructure item that is overlayed on the aerial images of the infrastructure on a display of the client device, col. 14, line 45 – col. 15, line 19) displaying the at least one generated spatial model within a background scene comprising the at least one civil structure’s real environment (display 500 may include a road 510 as well as terrain surrounding the road, such as grass, mountains, the sky, clouds, etc., col. 13, lines 22-50). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to overlay input in the infrastructure model as taught by the method of Harvey. The motivation for doing so would have been to place damage severity level indicators over the aerial images of the infrastructure item and determine a cost estimate for repair (Harvey, col. 15, lines 1-19).
In regards to claim 12, Ruda discloses a structural analysis optimization system (image capture system 100 including subject building 102 and unmanned aerial vehicle (UAV) 104 to assess damage to the roof, façade and siding of the building, paras. 0023-0026, fig. 1) for automatically displaying a spatial model of at least one civil structure on at least one visualization platform associated with a computing device (assessing damage to building 102 consists of capturing images of building 102 using a camera mounted onto UAV 104 and device 200 comprising network interfaces 210, at least one processor 220, and memory 240 storing software programs and data structures, , paras. 0028-0036, figs. 1 and 2), the structure analysis optimization system comprising: the computing device comprising at least one processor (device 200 with processor 220, fig. 2); and a non-transitory computer-readable medium operably coupled to the at least one processor (non-transitory, computer readable medium storing program instructions that cause a device to execute a process, para. 0022), the computer-readable medium having computer-readable instructions stored thereon that, when executed by the at least one processor, cause the structural analysis optimization system to (non-transitory, computer readable medium storing program instructions that cause a device to execute a process, para. 0022) automatically display at least one spatial model of the at least one civil structure on the at least one visualization platform associated with the computing device by executing instructions (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068) comprising: receiving, via at least one user interface associated with the at least one visual platform (user interface 900 for the system displays assessment data 314 for review by a human user, para. 0087, fig. 9), a data query from at least one user regarding at least one civil structure (UAV 104 is provided the GPS coordinates of the building and the UAV automatically captures the required images, para. 0028, figs. 1A and 1B); transmitting, via at least one processor of the computing device (device 200 with processor 220, fig. 2) communicatively coupled to at least one sensor (UAV 104 with camera, figs. 1 and 2) in mechanical communication with the at least one civil structure (Image analysis process 248 receives captured images 312 of the subject building downloaded from UAV 104 and transferred via a network to device 200 for analysis by image analysis process 248, para. 0045), at least one sensorial input to the at least one visualization platform (captured images 312 transferred to device 200 for analysis by image analysis process 248, para. 0045), the at least one sensorial input being configured (image analyzer 248 executes geometry analyzer 310 to create a three dimensional (3D) model of a roof based on the input aerial images 312 using stereophotogrammetry, para. 0062) to be displayed on the at least one visualization platform (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); generating, via the at least one visualization platform, the at least one spatial model of the at least one civil structure based on the at least one received sensorial input (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); and automatically displaying the at least one generated spatial model on the at least one visualization platform associated with the computing device (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068), but fails to disclose displaying by: based on a determination that at least one tactile-input is received from the at least one user, via the at least one user-interface, generating at least one spatial alteration to the at least one generated spatial model; and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches displaying (aggregate aerial images of infrastructure to create a 2D or 3D adjustable images displayed on a client device, col. 4, lines 47-64) by: based on a determination that at least one tactile-input is received from the at least one user (client device may display a 3D aerial image of a property with damage severity level indicators overlaying each property component, col. 20, lines 50-67), via the at least one user-interface (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58), generating at least one spatial alteration to the at least one generated spatial model (damage severity level indicators may include numbers which are placed over each property component, labels, symbols, different shading techniques, etc., col. 20, lines 50-67); and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability for the user to modify the model using the interface as taught by the method of Harvey. The motivation for doing so would have been to display infrastructure images with damage indicators overlaying the image on the client device (Harvey, col. 12, lines 36-44).
In regards to claim 13, modified Ruda discloses the structural analysis optimization system of claim 12, and further discloses wherein the at least one sensor is selected from a group consisting of an accelerometer, a strain gauge, a potentiometer, a camera (camera may be a digital single-lens reflex camera, any form of charge coupled device (CCD)-based camera, infrared (IR) camera, etc., para. 0027), a UAV (camera mounted onto UAV 104, para. 0028), a LiDAR scanner, an NDT tool, an ultrasound system, an infrared camera (camera may be a digital single-lens reflex camera, any form of charge coupled device (CCD)-based camera, infrared (IR) camera, etc., para. 0027), Ground Penetrating Radar (GPR), and a combination of thereof.
In regards to claim 15, modified Ruda discloses the structural analysis optimization system of claim 14, wherein the executed instructions further comprise recording, via the at least one processor of the computing device, the at least one sensorial input to a memory of the computing device (Image analysis process 248 receives captured images 312 of the subject building downloaded from UAV 104 and transferred via a network to device 200 for analysis by image analysis process 248, para. 0045).
In regards to claim 16, modified Ruda discloses the structural analysis optimization system of claim 15, but fails to disclose wherein the executed instructions further comprise, after recording the at least one sensorial input, assigning, via the at least one processor, a unique profile to the at least one recorded sensorial input, the at least one generated spatial model, or both associated with the at least one civil structure.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after recording the at least one sensorial input (After an aerial image 45 of an item of infrastructure is captured, the aerial image may be stored in the previous image data 94, so that it may be compared with a newly captured image of the same infrastructure item at a later date, col. 10, lines 44-48), assigning, via the at least one processor, a unique profile to the at least one recorded sensorial input (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21), associated with the at least one civil structure (server 14 may include infrastructure data (e.g., a list of items of infrastructure such as "Highway 80," "the Golden Gate Bridge," "the 'L' Station," etc.), location data (e.g., locations of the items of infrastructure, locations of portions of the items of infrastructure, etc.), previous image data ( e.g., aerial images of items of infrastructure taken at an earlier date), and financial data ( e.g., infrastructure cost estimates of property and materials similar to those that were damaged or destroyed, labor costs for repairing/replacing the infrastructure, etc.) from an infrastructure database 66, a location database 68, a previous image database 94, and a financial database 96, respectively, col. 9, lines 45-58). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability for the user to assign a profile and record image data as taught by the method of Harvey. The motivation for doing so would have been to determine a condition of an item of infrastructure as compared to its previous condition (Harvey, col. 7, lines 20-31).
In regards to claim 17, modified Ruda discloses the structural analysis optimization system of claim 16, but fails to disclose wherein the executed instructions further comprise, after assigning the at least one unique profile, recording at least one sensorial input from at least one alternative civil structure to the memory of the computing device.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after assigning the at least one unique profile (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21), recording at least one sensorial input (After an aerial image 45 of an item of infrastructure is captured, the aerial image may be stored in the previous image data 94, so that it may be compared with a newly captured image of the same infrastructure item at a later date, col. 10, lines 44-48) from at least one alternative civil structure to the memory of the computing device (a system may perform claims assessments for insured properties in a neighborhood by automatically surveying the entire neighborhood at once by capturing aerial images of the properties with each captured aerial image associated with a location using a GPS location, col. 15, lines 31-67). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to record images from other properties as taught by the method of Harvey. The motivation for doing so would have been to perform an automatic inspection of a neighborhood affected by a catastrophe (Harvey, col. 15, lines 20-67).
In regards to claim 18, modified Ruda discloses the structural analysis optimization system of claim 17, but fails to disclose wherein the executed instructions further comprise, after recording the at least one sensorial input of the at least one alternative civil structure, assigning, via the at least one processor, an alternative unique profile to the at least one recorded sensorial input, the at least one generated spatial model, or both associated with the at least one alternative civil structure.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches after recording the at least one sensorial input of the at least one alternative civil structure, assigning, via the at least one processor, an alternative unique profile (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21) to the at least one generated spatial model (a system may perform claims assessments for insured properties in a neighborhood by automatically surveying the entire neighborhood at once by capturing aerial images of the properties with each captured aerial image associated with a location using a GPS location, col. 15, lines 31-67), associated with the at least one alternative civil structure (group of aerial images are combined to generate a 3D image of the property utilizing the GPS coordinates received with each aerial image, col. 17, lines 51-65). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to assign profiles to captured images of various infrastructure as taught by the method of Harvey. The motivation for doing so would have been to perform an automatic inspection of a neighborhood affected by a catastrophe (Harvey, col. 15, lines 20-67).
In regards to claim 19, modified Ruda discloses the structural analysis optimization system of claim 18, but fails to disclose wherein the executed instructions further comprise, receiving, via the at least one user-interface, a spatial model query regarding the at least one unique profile, the at least one alternative unique profile, or both from the at least one user, the at least one alternative user, or both, wherein upon receiving the spatial model query, the at least one processor is configured to automatically display the at least one spatial model, the at least one alternative spatial model, or both on the at least one visualization platform associated with the computing device.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches receiving, via the at least one user-interface, a spatial model query regarding the at least one unique profile from the at least one user (client device 12 may request satellite images between specified GPS coordinates, and the image capturing module 90 may transmit satellite images within the specified coordinates, col. 7, lines 1-19), wherein upon receiving the spatial model query, the at least one processor is configured to automatically display the at least one spatial model (image user interface 70 may display aerial images of an infrastructure item and may also display damage severity levels for the infrastructure item, col. 8, lines 17-26), on the at least one visualization platform associated with the computing device (Each captured aerial image may be associated with a GPS location and the GPS location may be used to aggregate the aerial images to form a 3D image, col. 2, lines 6-21). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability to request infrastructure data for a property using gps coordinates as taught by the method of Harvey. The motivation for doing so would have been to utilize gps coordinates as location boundaries for the identified item of infrastructure for assessing damage (Harvey, col. 14, lines 17-30).
In regards to claim 20, modified Ruda discloses a method of contactless structural analysis (image capture system 100 including subject building 102 and unmanned aerial vehicle (UAV) 104 to assess damage to the roof, façade and siding of the building, paras. 0023-0026, fig. 1) on at least one visualization platform associated with a computing device (assessing damage to building 102 consists of capturing images of building 102 using a camera mounted onto UAV 104 and device 200 comprising network interfaces 210, at least one processor 220, and memory 240 storing software programs and data structures, , paras. 0028-0036, figs. 1 and 2), the method comprising the steps of: receiving, via at least one user interface associated with the at least one visual platform (user interface 900 for the system displays assessment data 314 for review by a human user, para. 0087, fig. 9), a data query from at least one user regarding at least one civil structure (UAV 104 is provided the GPS coordinates of the building and the UAV automatically captures the required images, para. 0028, figs. 1A and 1B); transmitting, via at least one processor of the computing device (device 200 with processor 220, fig. 2) communicatively coupled to at least one sensor (UAV 104 with camera, figs. 1 and 2) in mechanical communication with the at least one civil structure (Image analysis process 248 receives captured images 312 of the subject building downloaded from UAV 104 and transferred via a network to device 200 for analysis by image analysis process 248, para. 0045), at least one sensorial input to the at least one visualization platform (captured images 312 transferred to device 200 for analysis by image analysis process 248, para. 0045), the at least one sensorial input being configured (image analyzer 248 executes geometry analyzer 310 to create a three dimensional (3D) model of a roof based on the input aerial images 312 using stereophotogrammetry, para. 0062) to be displayed on the at least one visualization platform (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); generating, via the at least one visualization platform, at least one spatial model of the at least one civil structure based on the at least one received sensorial input (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); overlaying, via the at least one processor, the at least one sensorial input onto the at least one generated spatial model (a label may be applied to a given portion of the image ( e.g., a colored box, etc.) that indicates whether damage is present and the type of damage, para. 0079); automatically displaying the at least one generated spatial model on the at least one visualization platform associated with the computing device (end result of the geometry analyzer 310 is a complete 3D geometry of the subject roof that is output with assessment data 314 to a user interface such as a local or remote display and indicates the extent and type of damage to the roof, para. 0067 and 0068); but fails to disclose creating, via the at least one visualization platform, a background scene comprising the at least one civil structure’s real environment based on the at least one overlayed sensorial input; and automatically displaying the at least one generated spatial model within the background scene by: based on a determination that at least one tactile-input is received from the at least one user, via the at least one user-interface, generating at least one spatial alteration to the at least one generated spatial model; and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model.
Harvey, in the related field of systems for assessing damage to infrastructure, teaches creating, via the at least one visualization platform, a background scene comprising the at least one civil structure’s real environment based on the at least one overlayed sensorial input (display 500 may include a road 510 as well as terrain surrounding the road, such as grass, mountains, the sky, clouds, etc., col. 13, lines 22-50); and automatically displaying the at least one generated spatial model (aggregate aerial images of infrastructure to create a 2D or 3D adjustable images displayed on a client device, col. 4, lines 47-64) within the background scene (display 500 may include a road 510 as well as terrain surrounding the road, such as grass, mountains, the sky, clouds, etc., col. 13, lines 22-50) by: based on a determination that at least one tactile-input is received from the at least one user (client device may display a 3D aerial image of a property with damage severity level indicators overlaying each property component, col. 20, lines 50-67), via the at least one user-interface (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58), generating at least one spatial alteration to the at least one generated spatial model (damage severity level indicators may include numbers which are placed over each property component, labels, symbols, different shading techniques, etc., col. 20, lines 50-67); and based on a determination that at least one tactile-input is not received from the at least one user, via the at least one user-interface, maintaining the at least one generated spatial model (3D aerial image may be adjustable (e.g. a user may be able to “zoom in” or “zoom out”, col. 9, lines 45-58). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the method of Ruda with the ability for the user to modify the model using the interface as taught by the method of Harvey. The motivation for doing so would have been to display infrastructure images with damage indicators overlaying the image on the client device (Harvey, col. 12, lines 36-44).
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Ruda, in view of Harvey, and further in view of US 11,446,815 to Shah et al. (hereinafter referred to as Shaw).
In regards to claim 3, modified Ruda discloses the method of claim 1, but fails to disclose wherein the at least one visualization platform associated with the computing device comprises a multiplayer network, thereby allowing the at least one user and at least one alternative user to engage with the at least one generated spatial model simultaneously.
Shah, in the related field of the simulation and behavior in a virtual environment, teaches wherein the at least one visualization platform associated with the computing device comprises a multiplayer network (primary client device 110 and each user-controlled client device 115 (collectively and individually referred to as “client device” or “client devices” hereinafter) are one or more computing devices capable of receiving user input as well as transmitting and/or receiving data via a network 102, page 7, paras. 1-3), thereby allowing the at least one user and at least one alternative user to engage with the at least one generated spatial model simultaneously (system environment 100 of a robot simulation server 130, according to one embodiment. The system environment 100 includes a user 105 associated with a primary client device 110, users 120 associated with a set of user controlled client devices 115, a set of machine-controlled client devices 125, and a robot simulation server 130, all connected via the network 102, page 7, paras. 1-3) to engage with the at least one generated spatial model simultaneously (user 105 or the users 120 can monitor the autonomous control of a robot within a virtual environment by the robot simulation server 130 and can manually assume control of the monitored robot using the primary client device 110 or a user-controlled client device 115, page 7, paras. 1-3). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the platform of Ruda with the ability to use multiple terminals as taught by the system of Shah. The motivation for doing so would have been to enable multiple users to interact with the simulation server using user controlled client devices to provide inputs during simulation sessions (Shah, page 7, paras. 1-3).
In regards to claim 14, modified Ruda discloses the structural analysis optimization system of claim 12, but fails to disclose wherein the at least one visualization platform associated with the computing device comprises a multiplayer network, thereby allowing the at least one user and at least one alternative user to engage with the at least one generated spatial model simultaneously.
Shah, in the related field of the simulation and behavior in a virtual environment, teaches wherein the at least one visualization platform associated with the computing device comprises a multiplayer network (primary client device 110 and each user-controlled client device 115 (collectively and individually referred to as “client device” or “client devices” hereinafter) are one or more computing devices capable of receiving user input as well as transmitting and/or receiving data via a network 102, page 7, paras. 1-3), thereby allowing the at least one user and at least one alternative user to engage with the at least one generated spatial model simultaneously (system environment 100 of a robot simulation server 130, according to one embodiment. The system environment 100 includes a user 105 associated with a primary client device 110, users 120 associated with a set of user controlled client devices 115, a set of machine-controlled client devices 125, and a robot simulation server 130, all connected via the network 102, page 7, paras. 1-3) to engage with the at least one generated spatial model simultaneously (user 105 or the users 120 can monitor the autonomous control of a robot within a virtual environment by the robot simulation server 130 and can manually assume control of the monitored robot using the primary client device 110 or a user-controlled client device 115, page 7, paras. 1-3). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to provide the platform of Ruda with the ability to use multiple terminals as taught by the system of Shah. The motivation for doing so would have been to enable multiple users to interact with the simulation server using user controlled client devices to provide inputs during simulation sessions (Shah, page 7, paras. 1-3).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rakshit et al. (US 2016/0284127) teaches individualized content in an augmented reality system.
Loveland et al. (WO 2018/156506) teaches systems and method for subsurface damage assessments using UAVs.
Kottenstette et al. (US 2017/0076438) teaches receiving targe images, determining target class, and generating a heat map using a neural network to build a 3D model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul Schwarzenberg whose telephone number is (313) 446-6611. The examiner can normally be reached on Monday-Thursday (7:30-6:30).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christine Behncke, can be reached on (571) 272-8103. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL S SCHWARZENBERG/Primary Examiner, Art Unit 3695 6/1/2026
Prosecution Insights