DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 5 February 2026 have been fully considered but they are not persuasive.
Applicant requests that it be clear in the record as to what Castillo is missing that Hopper is providing. Examiner notes that the rejection describes Hopper teaching predicting, using a second model, pitch and height of each perimeter edge of each roof top component based on the predicted outline, wherein the pitch and height are outputs of the second model. As noted by applicant, Hopper does not teach that the model is a machine learning model. Castillo teaches predicting a pitch and height of roof top components based on the predicted outline by a second machine learning model, as stated in the rejection (emphasis added).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 9, 11-14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis (U.S. Publication 2018/0089833) in view of Hopper (U.S. Publication 2019/0051043) and Castillo (U.S. Publication 2023/0215087).
As to claim 1, Lewis discloses a method comprising:
obtaining, using one or more processors (p. 5-6, section 0046; the method runs as instructions executed by a computer, necessitating a processor), an aerial image of a building based on an input address (p. 2, section 0023; an address is input and associated aerial images are obtained);
obtaining, using the one or more processors (p. 5-6, section 0046; the method runs as instructions executed by a computer, necessitating a processor), three-dimensional (3D) data containing the building based on the input address (p. 2, section 0023; an address is input, associated images are obtained, and 3D lines are derived to approximate the building contour);
pre-processing the aerial image and 3D data (fig. 15; p. 2, sections 0023-0024; p. 5-6, section 0046; aerial imagery is fed to a pre-processing module; before 3D lines data is actually used to determine the building contour, the lines are filtered, which reads on a pre-processing step);
reconstructing, using the one or more processors, a 3D building model of a roof top of the building from the pre-processed image and 3D data, the reconstructing including predicting, using instance segmentation (fig. 6a; p. 2, section 0025; p. 2-3, section 0028; building segmentation is used to create a mask delimiting different roof faces and a 3D model for the building including the roof components is generated/reconstructed):
predicting, using a first machine learning model, an outline for each roof top component (fig. 6a; fig. 13; fig. 14; p. 2-3, section 0028; p. 5, sections 0042-0043; lines that outline each roof face/component are derived using a machine learning model; a skeleton algorithm is then used to derive missing lines that segment the components)
Lewis does not disclose, but Hopper discloses predicting, using a second model, pitch and height of each perimeter edge of each roof top component based on the predicted outline, wherein the pitch and height are outputs of the second model (p. 2, sections 0021-0022; p. 3-4, sections 0033-0034; a computer vision model can be used to determine edges of an outline; the outline construction module/model is used to output pitch and height of each roof top perimeter edge). The motivation for this is to support roof structures where perimeter edges are at different heights and/or roof structures that contain pitch changes (p. 1, section 0003). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis to predict, using a second model, pitch and height of each perimeter edge of each roof top component based on the predicted outline, wherein the pitch and height are outputs of the second model in order to support roof structures where perimeter edges are at different heights and/or roof structures that contain pitch changes as taught by Hopper.
Lewis does not disclose, but Castillo does disclose predicting, using a second machine learning model, a pitch and height of roof top components based on the predicted outline (p. 2-3, section 0045; p. 3, section 0052; p. 6, section 0079; p. 6, sections 0083-0085; p. 8, sections 0104-0105; a model uses bounding boxes representing the components, reading on outlines, to determine surface normals, which are used to calculate pitch and rise/elevation/height); and rendering, using the one or more processors, the 3D building model based on the predicted outline, at least one pitch and height of each roof top component (p. 11, section 0143; the calculations are used to display a 3D model after surface normals are determined). The motivation for this is to be more accurate and time efficient than manual techniques (p. 2, sections 0041-0043). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis and Hopper to predict, using a second machine learning model, a pitch and height of each roof component based on the predicted outline and rendering the 3D building model based on the predicted outline, at least one pitch and height of each roof top component in order to be more accurate and time efficient than manual techniques as taught by Castillo.
As to claim 2, Lewis discloses, wherein predicting, using the first machine learning model, the outline for each roof top component, further comprises: predicting, for each roof top component in a sequence of roof top components, a location of each perimeter edge of the roof top component (p. 2, section 0028; each edge for each component is predicted using edge detection and a probability distribution); and predicting, for each roof top component, a location of each fold in the roof top component (p. 5, section 0042; lines such as hip lines that would read on folds are predicted). Castillo also discloses prediction of such locations (p. 3-4, section 0055).
As to claim 3, Lewis discloses wherein the locations are predicted by a neural network, which outputs a probability distribution over potential locations (p. 2-3, section 0028; a probability distribution is created, representing whether the particular pixel corresponds to a location of an edge on the roof component).
As to claim 9, Castillo discloses first and second machine learning models as parts of a single neural network (p. 2-3, section 0045; p. 3, section 0052; p. 8, sections 0104-0105; an example single neural network is made up of two networks, one implementing a model for segmentation and another implementing a model for outputting surface normal vectors; the normal vectors are used to calculate pitch and rise/elevation/height). Motivation for the combination is given in the rejection to claim 1.
As to claim 11, Castillo discloses a method further comprising: predicting, using instance segmentation, a mask for each roof component of the building (p. 5, section 0070-0074; masks are predicted based on segmentation output); predicting, using a first machine learning model with the mask as input, an outline for each roof component (p. 5, section 0075-p. 6, section 0079; the model applies a linear detection to the mask to determine a linear feature representing bounds/outlines of components); and predicting, using a second machine learning model with the mask and outline as input, a pitch and height of each roof component (p. 2-3, section 0045; p. 3, section 0052; p. 6, sections 0083-0085; p. 8, sections 0104-0105; another model uses the masked bounded components to determine surface normals, which are used to calculate pitch and rise/elevation/height). Motivation for the combination is given in the rejection to claim 1.
As to claim 12, see the rejection to claim 1. Further, Lewis discloses a system comprising: one or more processors; memory coupled to the one or more processors and storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations (p. 5, section 0046; a computer would contain processors and a memory medium for executing the described code). Castillo also describes these components (p. 10, section 0133-p. 11, section 0138).
As to claim 13, see the rejection to claim 2.
As to claim 14, see the rejection to claim 3.
As to claim 20, see the rejection to claim 9.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis in view of Hopper and Castillo and further in view of Zadeh (U.S. Publication 2022/0121884).
As to claim 4, Lewis does not disclose, but Zadeh does disclose wherein the probability distribution is used to guide a search process that estimates how good each prediction will be (p. 82, sections 1811-1814; p. 85, section 1856; a search process resulting in labels for particular regions is performed; a probability distribution is used to evaluate the search’s quality/reliability and weighting to guide the process is performed based on the reliability). The motivation for this is to determine how well labels and associated data fit into the energy of the system. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, and Castillo to use the probability distribution to guide a search process that estimates how good each prediction will be in order to determine how well labels and associated data fit into the energy of the system as taught by Zadeh.
As to claim 15, see the rejection to claim 4.
Claims 5, 6, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis in view of Hopper, Castillo and Zadeh and further in view of Johnson (U.S. Publication 2021/0233314).
As to claim 5, Lewis and Castillo disclose a roof segmentation, as discussed in the rejection to claim 1. Lewis does not disclose, but Johnson discloses where the search process explores a specified number of forward steps and compares a representation that results from each possible next node or fold to outputs of an instance segmentation network (p. 6, section 0040; a specified number of forward steps/iterations through the nodes of the neural network is performed; until the number of steps is reached, a comparison is performed from each next output node of the network to the ground truth which is determined as an instance segmentation output; if the difference is below a threshold, the process can terminate). The motivation for this is to continually improve a network through iterations. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, Castillo, and Zadeh to have a search process explore a specified number of forward steps and compare a representation that results from each possible next node or fold to outputs of an instance segmentation network in order to continually improve a network through iterations as taught by Johnson.
As to claim 6, Lewis and Castillo discloses where the structure is a rooftop, as discussed in the rejection to claim 1. Lewis does not disclose, but Johnson does disclose wherein the outputs of the instance segmentation network are treated as a close approximation to the actual two-dimensional (2D) structure (p. 6, section 0040; a pre-generated instance segmentation mask output is used as a training image for comparison to a prediction; the instance segmentation mask is considered the equivalent of ground truth, making it a close approximation of the actual 2D structure in the image). Motivation for the combination is given in the rejection to claim 5.
As to claim 16, see the rejection to claim 5.
As to claim 17, see the rejection to claim 6.
Claims 7, 8, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis in view of Hopper, Castillo and Zadeh and further in view of Shen (U.S. Patent 11,270,034).
As to claim 7, Lewis does not disclose, but Shen discloses wherein results of the search are used to update the probability distribution for predicting the location of the next node or fold (col. 21, line 22-col. 22, line 58; a Monte Carlo tree search is performed to determine a next node to travel; when nodes are traveled based on the result of search or a final node is reached, the parameters, which correspond to the probability distribution that various nodes will be selected, are updated). The motivation for this is to refine an optimal layout. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, Castillo, and Zadeh to use results of the search to update the probability distribution for predicting the location of the next node or fold in order to refine an optimal layout as taught by Shen.
As to claim 8, Lewis does not disclose, but Shen discloses wherein the search is a Monte Carlo Tree Search (col. 21, line 22-col. 22, line 58). Motivation for the combination is given in the rejection to claim 7.
As to claim 18, see the rejection to claim 7.
As to claim 19, see the rejection to claim 8.
Claims 10 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis in view of Hopper and Castillo and further in view of Hoiem (U.S. Publication 2021/0183080), Pershing (U.S. Publication 2010/0114537), and Murdoch (U.S. Publication 2014/0132635).
Lewis discloses generating a building mask from the image (p. 2, section 0028-p. 3, section 0030; a mask based on the building edges in the image is generated); using the 3D data with the building mask to calculate an orientation of each roof face of the building (p. 5, sections 0041-0043; slopes and directions for lines on each roof face, which would define the orientation of the roof face, are calculated) and using the building mask to obtain an extent of the building (p. 2, section 0028-p. 3, section 0031; the mask is used to determine the boundaries of a central building).
Lewis does not disclose, but Hoiem does disclose wherein pre-processing the aerial image (fig. 2; p. 5, section 0047; drone footage is used) and 3D data (fig. 2; p. 5, sections 0046-0047; 3D data is derived using SLAM, SfM, or a combination), further comprises: generating a 3D mesh from the 3D data (p. 7, sections 0064-0065); generating a digital surface model (DSM) of the building using the 3D mesh (p. 7, sections 0064-0067; a surface model is generated using the mesh); and aligning the image and DSM (p. 7, section 0066-p. 8, section 0075; the image and model are aligned). The motivation for this is to help visualize progression of construction or as-built conditions of a site (p. 1, section 0005). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, and Castillo to generate a 3D mesh from the 3D data, generate a digital surface model (DSM) of the building using the 3D mesh, and align the image and DSM in order to help visualize progression of construction or as-built conditions of a site as taught by Hoiem.
Lewis does not disclose, but Pershing does disclose snapping the orientation of each roof face to a grid (p. 2, section 0024; p. 12, section 0132; images of a roof are aligned to a set of reference points in a grid; specifically planar roof sections, which would read on faces, are translated to 3D grid reference points, which would parameterize the angle/orientation of the section). The motivation for this is to measure a roof for solar panels (p. 1, section 0009). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, Castillo, and Hoiem to snap the orientation of each roof face to a grid in order to measure a roof for solar panels as taught by Pershing.
Lewis does not disclose, but Murdoch does disclose cropping the image so that the building is centered in the image and axis-aligned to the grid (p. 2-3, section 0029; p. 4, claim 3; an image of the roof of the building is enlarged and centered, which would result in cropping out some of the background; the lines on the roof image are also aligned to be parallel to the horizontal and vertical axes of the display and to an overlaid grid pattern). The motivation for this is to help a user quickly estimate roof surface area (p. 1, section 0006). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Lewis, Hopper, Castillo, Hoiem, and Pershing to crop the image so that the building is centered in the image and axis-aligned to the grid in order to help a user quickly estimate roof surface area as taught by Murdoch.
As to claim 21, see the rejection to claim 10.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/Primary Examiner, Art Unit 2617