DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7-12, 14, 15, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al., “Semi-automatic geometric digital twinning for existing buildings based on images and CAD drawings” (hereinafter Lu) in view of Gallo et al. (US 2021/0150080) (hereinafter Gallo).
Regarding claims 1 and 11, Lu teaches a method and non-transitory computer-readable medium having instructions to perform the method, the method for extracting information about a structure, comprising:
receiving, at a computer-generated interface, a first file and a second file, wherein the first file is of a different format than the second file (pg. 3, fig 3, Module 1 receiving and processing CAD files, Module 2 receiving and processing collected images);
extracting, from the first file and the second file, information relevant to the structure (pg. 3, fig. 3, Module 1 processing of the CAD files extracts information relevant to the structure and Module 2 processing of the collected images and extracts information relevant to the structure; pg. 15, fig. 20, the extracted information from modules 1 and 2) ; and
generating, by correlating information extracted from the first file with information extracted from the second file, additional information about the structure not found in the first file or second file (pg. 3, fig. 3, Module 3 integrating information and creating IFC files; pgs. 13-15, section 4.3 The analysis of module 3; pg. 15, “the information integration and IFC creation module integrates information from Module 1 and 2 and create as-is IFC BIM. Fig. 22 shows the sample of the created IFC file and the corresponding generated IFC BIM of a regular storey, which is opened in IfcViewer. When the information from Module 1 and 2 are integrated, the IFC BIM would be automatically created for the target building storey using this approach.”).
Lu does not explicitly teach that the extracting is done using at least one artificial neural network (ANN). However, Yeh teaches extracting information relevant to the structure from files using at least one artificial neural network (ph. [0004], “generating synthetic data and extracting BIM elements from floor plan drawings using machine learning.”; ph. [0015], “the system not only recognizes floor plan drawing (e.g., electrical) symbols but also extracts geometric and semantic information of the symbols such as symbol labels, orientation and size of the elements for later auto-placement in a BIM model”; ph. [0041], “The room layout can be deduced by/from existing CAD drawings, exhaustively generated (of all possible results) [Per Galle], generated by generative design or created by machine learning algorithms such as GAN (generative adversarial network) [Zheng] or with shape grammars and reinforcement learning [Ruiz-Montiel]. In other words, at step 202, a room layout/floorplan for one or more rooms of a floorplan drawing is obtained (e.g., via one or more different methodologies).”; ph. [0058], “The output 102 can be a floor plan in vector format or an image. Further, the parameters/symbols can be found manually or randomly and may be determined using a machine learning algorithm such as GAN that decides if the output design matches the customer data”; fig. 8, object detection with machine learning model 818, object classification with ML model 834, and Object Orientation with ML Model 836). One of ordinary skill in the before the effective filing date would have been motivated to modify Lu in the manner taught by Yeh in order to help “businesses extract crucial information from their documents using artificial intelligence techniques.”(Yeh, ph. [0010]).
Regarding claim 2, the Lu/Yeh combination teaches the method of claim 1. Lu further teaches at least one of the first file and second file are in a known format that guides extraction of relevant information (pg. 1, “the majority of existing buildings have only 2D drawings and text documents in hard copy formats and/or in electronic CAD formats”; pg. 6, “The general process of extracting structural components from CAD drawings is presented in Fig. 4. The main functions of Module 1 are designed as follows: 1). CAD drawings pre-processing step; establishing the grids and blocks through recognising special symbols (i.e., column network symbols) in a floor plan. Then, 2) Text information filtration step; filtering the text information from the scrambled backgrounds. 3) Text information extraction step; extracting text data using the Optical Character Recognition (OCR) algorithm from two directions and saving them in Excel formats. Step 2 and 3 aim at selecting meaningful text items from the CAD documents based on predefined grids and blocks (step 1).”).
Regarding claims 3 and 15, the Lu/Yeh combination teaches the method of claim 2 and medium of claim 11. Lu further teaches the relevant information is selected to be extracted from the at least one of the first file and second file on the basis of file context (pg. 6, “Then, 2) Text information filtration step; filtering the text information from the scrambled backgrounds. 3) Text information extraction step; extracting text data using the Optical Character Recognition (OCR) algorithm from two directions and saving them in Excel formats. Step 2 and 3 aim at selecting meaningful text items from the CAD documents based on predefined grids and blocks (step 1).”).
Regarding claim 5, the Lu/Yeh combination teaches the method of claim 1. Yeh further teaches the at least one ANN is trained to extract a specific type of construction-related data (fig. 8, train object detection ML Model 804 and Train Symbol Classification and Orientation ML Model 830).
Regarding claims 7 and 18, the Lu/Yeh combination teaches the method of claim 1 and medium of claim 11. Lu further teaches at least one of the first file and second file are blueprints or architectural drawings of the structure (pg. 1, “the majority of existing buildings have only 2D drawings and text documents in hard copy formats and/or in electronic CAD formats”), and the method further comprises determining a scale of the structure (pg. 8, “the keywords of beams in Fig. 7 use ‘BM’. There is no keyword for the column, but sizes (i.e., length and width) of columns can be identified and extracted based on their locations and frequencies in CAD drawings.”).
Regarding claims 8 and 19, the Lu/Yeh combination teaches the method of claim 7 and medium of claim 18. Lu further teaches the method further comprises extrapolating dimensions for the structure that are not indicated in at least one of the first file and second file from the relevant information (pg. 8, “the keywords of beams in Fig. 7 use ‘BM’. There is no keyword for the column, but sizes (i.e., length and width) of columns can be identified and extracted based on their locations and frequencies in CAD drawings.”).
Regarding claims 9 and 20, the Lu/Yeh combination teaches the method of claim 8 and medium of claim 20. Lu further teaches the method further comprises converting relevant information extracted from the first file and second file to the determined scale of the structure (pg. 8, “the keywords of beams in Fig. 7 use ‘BM’. There is no keyword for the column, but sizes (i.e., length and width) of columns can be identified and extracted based on their locations and frequencies in CAD drawings.”).
Regarding claims 10 and 12, the Lu/Yeh combination teaches the method of claim 1 and medium of claim 11. Lu further teaches extracting, using machine vision, information relevant to the structure (pgs. 3-4, section “2.1. From images/video… The image-based point cloud construction approach generates semi-dense or dense scenes using thousands of overlapping images via matching features. Then, semantically-rich as-is 3D models/as-is BIM could be further created based on these resulting point cloud models. Multi-view stereo (MVS) and Structure from Motion (SfM) are the two most widely used and successful methods of constructing 3D point models for target scenes… Image processing approach would mainly use image processing algorithms to detect geometric primitives (e.g., points and patches, edges, and lines) and key features (e.g., colour) from collected images”).
Regarding claim 14, the Lu/Yeh combination teaches the medium of claim 11. Yeh further teaches at least one of the first file and second file are in a format upon which the at least one ANN has been trained (fig. 8, training 804 and 830).
Claim(s) 6 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over the Lu/Yeh combination as applied to claims 1 and 11 above, and further in view of Witney, JR et al. (US 2021/0397757) (hereinafter Witney).
Regarding claim 6, the Lu/Yeh combination teaches the method of claim 1. The combination does not explicitly teach generating additional information comprises using the extracted information to retrieve further information from a remote database. However, Witney teaches generating additional information comprises using the extracted information to retrieve further information from a remote database (ph. [0096], “the generative design computing platform 110 may calculate and/or otherwise determine an estimated cost of purchasing the one or more specified pieces of furniture in the space model (e.g., based on unit-level pricing data and/or other details, which may, e.g., be retrieved by the generative design computing platform 110 from another system or database, such as a Harbor database)”). One of ordinary skill in the art before the effective filing date would be motivated to modify the Lu/Yeh combination in the manner taught by Witney in order to estimate the building construction and replacement costs for example insurance purposes.
Regarding claim 17, the Lu/Yeh combination teaches the medium of claim 11. The combination does not explicitly teach further causing the apparatus to use the extracted information to retrieve information from a remote database, and to use the retrieved information to generate the additional information. However, Witney teaches further causing the apparatus to use the extracted information to retrieve information from a remote database (ph. [0096], “the generative design computing platform 110 may calculate and/or otherwise determine an estimated cost of purchasing the one or more specified pieces of furniture in the space model (e.g., based on unit-level pricing data and/or other details, which may, e.g., be retrieved by the generative design computing platform 110 from another system or database, such as a Harbor database)”)., and to use the retrieved information to generate the additional information (ph. [0096], “In some instances, graphical user interface 600 may include final pricing information for the space model (e.g., based on included blocks, settings, furniture, and/or other information), which may be based on pricing information pulled from an internal and/or external data source”). One of ordinary skill in the art before the effective filing date would be motivated to modify the Lu/Yeh combination in the manner taught by Witney in order to estimate the building construction and replacement costs for example insurance purposes.
Allowable Subject Matter
Claims 4 and 13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Roy et al. (US 2023/0298498) teaches machine learning for parameterizing building information from building images.
Sajanikar (US 2023/0168665) teaches generating PFS diagrams from engineering data.
Liebig et al. (US 2023/0053615) teaches generation of a building information model.
Balasubramanian (US 2023/0020885) teaches automatic conversion of 2D schematics to 3D models.
Hsieh (US 2022/0188625) teaches generating layout plan using neural network.
Austern et al. (US 2021/0073433) teaches automatic extraction of data from 2D floor plans for retention in BIMs.
Levy et al. (US 2021/0073430) teaches optimizing equipment selection in floorplans using modeling and simulation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN W WATHEN whose telephone number is (571)270-5570. The examiner can normally be reached M-F 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at 571-272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
BRIAN W. WATHEN
Primary Examiner
Art Unit 2151
/BRIAN W WATHEN/ Primary Examiner, Art Unit 2151
Prosecution Insights