DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Austern (US 20210073442 A1).
Regarding Claim 1, Austern teaches:
one or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising: (¶118 processor … suitable for executing instructions … memory)
receiving project information including project specifications and drawings for the building; (¶122 the floor plan may be represented as a hand-drawn or scanned image … A floor plan may also include other information associated with the plurality of rooms and the equipment and data contained therein; ¶395 the semantic enrichment process may include accessing a floor plan. As illustrated in FIG. 10E, accessing may include a user 1083 uploading a floor plan file. This file may be CAD file, PDF file, BIM file or image file.)
generating a digital model of the building based on the project information, wherein generating the digital model comprises: (¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079. As described herein, a semantic enrichment model may be a computational model, artificial intelligence model, or other model configured to receive floor plan data as input and output semantic designations for spaces within the floor plan.; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan. BIM may include an analysis of a BIM model (e.g., a three dimensional representation of a building with associated labels, BIM objects and parametric data) using a combination of geometric analysis, semantic analysis, and machine learning methods)
extracting, from the project information, one or more schedules, schedule notes, floor plans, or identified tags; and (¶122 the floor plan may be represented as a hand-drawn or scanned image … A floor plan may also include other information associated with the plurality of rooms; ¶395 the semantic enrichment process may include accessing a floor plan.; ¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
associating each of the one or more schedules, schedule notes, floor plans, or identified tags with a corresponding location in the project information; (¶213 the input may include a sample, and the inferred output may include a classification of the sample (such as an inferred label, an inferred tag, and so forth); ¶250 outputting the solution may include an equipment placement location ... The output solution may be graphical in that it uses symbols to indicate electrical equipment locations and text may also be provided to illuminate the details of the electrical equipment and/or its installation. The output may be provided in a BIM model or CAD drawing, and may include semantic data related to electrical, HVAC and other equipment (e.g. block name, family type, label, or tag); ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
presenting, on a user interface, a portion of the project information in a base layer; and (See Figs. 24D-24G, and discussion thereof in the reference, for example ¶710 FIG. 24D is an illustration depicting an exemplary machine learning door analysis. Machine learning door analysis may be used to identify doors within a floor plan … Natural drawings 2453 may be original floor plans without any specific demarcation of doors ... Once machine learning detection or classification model 2459 is trained, it may receive an input natural drawing 2457. The input natural drawing 2457 may be a drawing for which a user wishes to have doors detected. After receiving the input natural drawing 2457, machine learning detection or classification model 2459 may detect and demarcate doors of the floor plan by one or more of the various methods described herein. In some embodiments, as shown in floor plan illustration 2461, machine learning door analysis may include highlighting the detected doors by, for example, placing a black square over the doors or otherwise demarcating the doors on the floor plan.; See also Fig. 26 and discussion of its user interface in ¶731)
presenting, on the user interface, a model layer overlaying the base layer, the model layer comprising one or more model objects representing each of the one or more schedules, schedule notes, floor plans, or identified tags in the digital model, wherein the model objects are rendered in the model layer at a position on the user interface according to the corresponding location of the one or more schedules, schedule notes, floor plans, or identified tags in the base layer. (¶245 An output solution may be a graphical depiction of equipment location or types which are the result of the generative analysis process. It may be depicted as a layer on top of the input floor plan … an output may be displayed via a graphical user interface on a display or virtually. A solution may include a floor plan containing one or more areas of interest or disinterest.; Figs. 24D-G, FIg. 26, Fig. 28B; see above citations regarding the floor plans/tags)
Regarding Claim 2, Austern teaches:
updating the digital model based on a user input, wherein updating comprises: (¶225 receive, via a graphical user interface, information marking a plurality of areas of interest or disinterest in at least one room. In further embodiments the plurality of respective areas of interest or disinterest may have respective different functional requirements or different functional requirement types. For example, a first functional requirement may involve a lighting condition and a second functional requirement may involve sensor coverage. Or, two different functional requirements may be of the same type (e.g., lighting condition) but the requirements may vary (e.g., different lux levels.) These differing variables may be entered, for example, via a graphical user interface, via a rule, or via index.)
receiving a user input modifying at least one of the one or more schedules, schedule notes, floor plans, or identified tags; (¶225 receive, via a graphical user interface, information marking a plurality of areas of interest or disinterest in at least one room. In further embodiments the plurality of respective areas of interest or disinterest may have respective different functional requirements or different functional requirement types. For example, a first functional requirement may involve a lighting condition and a second functional requirement may involve sensor coverage. Or, two different functional requirements may be of the same type (e.g., lighting condition) but the requirements may vary (e.g., different lux levels.) These differing variables may be entered, for example, via a graphical user interface, via a rule, or via index.)
extracting, from the drawings, one or more updated identified tags; and (¶122 the floor plan may be represented as a hand-drawn or scanned image … A floor plan may also include other information associated with the plurality of rooms; ¶395 the semantic enrichment process may include accessing a floor plan.; ¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
associating the one or more updated identified tags with a corresponding location in the drawings; and (¶213 the input may include a sample, and the inferred output may include a classification of the sample (such as an inferred label, an inferred tag, and so forth); ¶250 outputting the solution may include an equipment placement location ... The output solution may be graphical in that it uses symbols to indicate electrical equipment locations and text may also be provided to illuminate the details of the electrical equipment and/or its installation. The output may be provided in a BIM model or CAD drawing, and may include semantic data related to electrical, HVAC and other equipment (e.g. block name, family type, label, or tag); ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
presenting, in the model layer on the user interface, one or more model objects representing each of the one or more updated identified tags. (See Figs. 24D-24G, and discussion thereof in the reference, for example ¶710 FIG. 24D is an illustration depicting an exemplary machine learning door analysis. Machine learning door analysis may be used to identify doors within a floor plan … Natural drawings 2453 may be original floor plans without any specific demarcation of doors ... Once machine learning detection or classification model 2459 is trained, it may receive an input natural drawing 2457. The input natural drawing 2457 may be a drawing for which a user wishes to have doors detected. After receiving the input natural drawing 2457, machine learning detection or classification model 2459 may detect and demarcate doors of the floor plan by one or more of the various methods described herein. In some embodiments, as shown in floor plan illustration 2461, machine learning door analysis may include highlighting the detected doors by, for example, placing a black square over the doors or otherwise demarcating the doors on the floor plan.; See also Fig. 26 and discussion of its user interface in ¶731)
Regarding Claim 3, Austern teaches:
wherein the one or more schedules, schedules notes, floor plans, or identified tags are extracted using an artificial intelligence (AI) agent. (¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079. As described herein, a semantic enrichment model may be a computational model, artificial intelligence model, or other model configured to receive floor plan data as input and output semantic designations for spaces within the floor plan. In some embodiments, the semantic enrichment model 1079 may also output a Room Semantic Designation 1081 and perhaps a Confidence Rating, as described herein.; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan. BIM may include an analysis of a BIM model (e.g., a three dimensional representation of a building with associated labels, BIM objects and parametric data) using a combination of geometric analysis, semantic analysis, and machine learning methods aimed at extracting specific information regarding the architecture of a floor plan. The results of a BIM analysis may include a set of room boundaries, names, numbers and descriptions of the rooms, walls, doors and windows associated with said rooms, materiality and description of any of these elements, etc.)
Regarding Claim 4, Austern teaches:
obtaining a tag dictionary including a list of potential tag identifiers; and (¶123 the classification tag; ¶155-156 a set of technical specifications may be stored in a data structure ... The data can contain textual, tabular, numeric or image information which describes the equipment ... The data may be accessible using a simple query; ¶202 generating a classification tag based on the first technical specification and the first equipment placement location.; ¶899 the requirements may define constraints such as areas on non-placement (e.g., stored as a dictionary containing the requirement type and a list of boundary points on which it applies); areas/paths/points of preferred placement of predefined devices (e.g., stored as a dictionary containing information of the requirement type, the geometry it applies on, and a list of device filters describing the subset of catalog items that should be placed)
extracting one or more identified tags using the tag dictionary. (¶123 the classification tag; ¶155-156 a set of technical specifications may be stored in a data structure ... The data can contain textual, tabular, numeric or image information which describes the equipment ... The data may be accessible using a simple query; ¶202 generating a classification tag based on the first technical specification and the first equipment placement location.)
Regarding Claim 5, Austern teaches:
identifying a floor in the building based on the project information; (¶122 the floor plan may be represented as a hand-drawn or scanned image … A floor plan may also include other information associated with the plurality of rooms; ¶395 the semantic enrichment process may include accessing a floor plan.; ¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
generating a corresponding floor in the digital model; and (¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079. As described herein, a semantic enrichment model may be a computational model, artificial intelligence model, or other model configured to receive floor plan data as input and output semantic designations for spaces within the floor plan.; ¶475 the system may use building information model (BIM) analysis to identify contours of a floor plan. BIM may include an analysis of a BIM model (e.g., a three dimensional representation of a building with associated labels, BIM objects and parametric data) using a combination of geometric analysis, semantic analysis, and machine learning methods)
associating in the digital model each of the one or more schedules, schedule notes, floor plans, or identified tags with a corresponding location on the floor. (¶213 the input may include a sample, and the inferred output may include a classification of the sample (such as an inferred label, an inferred tag, and so forth); ¶250 outputting the solution may include an equipment placement location ... The output solution may be graphical in that it uses symbols to indicate electrical equipment locations and text may also be provided to illuminate the details of the electrical equipment and/or its installation. The output may be provided in a BIM model or CAD drawing, and may include semantic data related to electrical, HVAC and other equipment (e.g. block name, family type, label, or tag); ¶667 identifying wall boundaries of a plurality of rooms using a machine learning method. As described herein, wall boundaries may be contours or other indicators of a space that show a room relative to other rooms within a floor plan. Identifying may include pointing, tagging, measuring, specifying, extracting, labeling, or any other means of identifying. In some embodiments, identifying may include highlighting or otherwise delineating identified boundaries on a floor plan.)
Regarding Claim 6, Austern teaches:
wherein the portion is of a first sheet of a plurality of sheets in the drawings, (Fig. 25A)
the system further comprising presenting, on the user interface, a sheet summary table listing the one or more identified tags associated with the first sheet and a number of instances of each identified tag located on the first sheet. (Fig. 25A)
Regarding Claim 7, Austern teaches:
communicating the digital model to a machine learning agent configured to use the digital model to determine at least one of the plurality of building equipment components capable of being used in the building. (Fig. 28C; ¶394 Once a floor plan has been analyzed, the resulting space data (including but not limited to geometric data, architectural feature data, and textual data) may be input into a semantic enrichment model 1079. As described herein, a semantic enrichment model may be a computational model, artificial intelligence model, or other model configured to receive floor plan data as input and output semantic designations for spaces within the floor plan. In some embodiments, the semantic enrichment model 1079 may also output a Room Semantic Designation 1081 and perhaps a Confidence Rating, as described herein.; ¶783 this may include semantic analysis of BIM object metadata and other property data which may include manufacturers, model number, BIM families, description, technical characteristics, classification tags and a variety of other data. In some embodiments, a machine learning model may be trained using a training data set of floor plans with geometric data representing furniture or architectural features as well sematic designations for those features. Based on the training data, a machine learning model may be developed to automatically associate geometric data with semantic designations. In some embodiments, the semantic designations may be determined based on property data or other metadata associated with the floor plan. For example, object property data of a BIM object, for example, may include a model number for an office chair, which the disclosed systems may associate with a “chair” semantic designation; ¶787 data regarding equipment associated with rooms)
Regarding Claim 8, Austern teaches:
associating a type of building equipment with at least one of the one or more schedules; (¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶168 FIG. 1 is a block diagram illustrating an example optimization process for identifying technical specifications and equipment locations, consistent with the disclosed embodiments)
associating at least one schedule note with the at least one of the one or more schedules; (¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶179 In an initial stage 110, an objective associated with a room 112 to be achieved using generative analysis may be defined. The objective may be based on a functional requirement associated with room 112, as described above. In the example depicted in FIG. 1, the objective may be associated with placement of a camera 114 and a functional requirement may require coverage region 116 of camera 114 to be maximized.)
calculating a performance metric for the type of building equipment based on the one or more associated schedules and the at least one schedule note; and (¶170 the generative analysis may run a plurality of simulations 122, 124 and 126 representing different option for placement of camera 114. … each of the simulations 122, 124, and 126 may then be assessed with respect to the functional requirements. For example, the generative analysis may determine that simulation 124 results in a coverage region 116 representing 65% of room 112, as shown in FIG. 1. In stage 130, the generative analysis may include running additional simulations 132, 134, and 136 based on one or more of the simulations from stage 120. For example, the generative analysis may include selecting the simulation in stage 120 with the greatest coverage)
selecting a building equipment component from a plurality of building equipment components based on the performance metric. (¶171 In stage 140, the generative analysis may determine an optimal solution which, in this example, may represent the maximum coverage percentage of 85%. This optimal solution may be output as a result of the generative analysis, as described in greater detail below. For example, the output may include a technical specification for camera 114 and a suggested location for camera 114 that maximizes coverage region 116 and at least partially complies with the functional requirements.)
Regarding Claim 9, Austern teaches:
identifying, for each of the one or more schedules extracted from the project information, essential schedule information; (¶502 receiving instructions to vary the equipment placement location may include receiving instructions to remove equipment. Removal may include deleting equipment from a floor plan. For example, the system may receive instructions to remove a piece, set, or group of equipment from a solution. Additionally, or alternatively, the system may receive instructions to remove equipment based on a parameter associated with the equipment.; ¶506 may include receiving instructions to lock a manually modified parameter associated with at least one piece of equipment ... This would prevent user input from being removed, ignored, or otherwise modified by the system.; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment)
reducing each of the one or more schedules in the digital model to the identified essential schedule information; and (¶502 receiving instructions to vary the equipment placement location may include receiving instructions to remove equipment. Removal may include deleting equipment from a floor plan. For example, the system may receive instructions to remove a piece, set, or group of equipment from a solution. Additionally, or alternatively, the system may receive instructions to remove equipment based on a parameter associated with the equipment.; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶706 using a geometric algorithm or artificial intelligence to automatically identify and remove gridlines, equipment, furniture, or other elements on a floorplan to improve generative analysis outcomes (e.g., to improve simulation performance). Additionally or alternatively, an algorithm may be configured to remove duplicate spaces to improve generative analysis outcomes)
updating the digital model with the one or more reduced schedules. (¶502 receiving instructions to vary the equipment placement location may include receiving instructions to remove equipment. Removal may include deleting equipment from a floor plan. For example, the system may receive instructions to remove a piece, set, or group of equipment from a solution. Additionally, or alternatively, the system may receive instructions to remove equipment based on a parameter associated with the equipment.; ¶563 generating an equipment schedule indicating an association of primary and auxiliary equipment based on the selected primary and auxiliary equipment; ¶707 updating a retention data structure based on an input. For example, the input may include instructions to add, delete, move or extend)
Regarding Claim 10, Austern teaches:
selecting, using the digital model, building equipment components for each for each of the one or more identified tags; and (¶731 enabling a user to select an equipment symbol for analysis. For example, a user may be presented with a graphical user interface on a physical or virtual display that enables the user to select an equipment symbol)
generating a quote for the selected building equipment components. (¶156 The data associated with a piece of equipment may contain its model, type, brand, series, price information, specific technical characteristics or properties (such a IR rating or weather resistance), a technical specification, a physical description or a picture of the object, or any other information relating to the equipment. The data may be accessible using a simple query; ¶185 the output may include textual information in addition to, or in place of, a graphical representation. For example, textual information may include information regarding the model, placement, price or parameters associated with the equipment.; ¶303 A bill of material may be a list of equipment models which may include the specific model, the quantity used, price information; examiner notes that accessing or displaying cost is equivalent to obtaining or providing a quote)
Regarding Claims 11-20:
Claims 11-20 are substantially similar to claims 1-10 respectively, and are rejected under the same grounds as those set forth for those claims above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20170147717 A1 discusses details regarding handling drawings and determining information from them for later display, but is silent regarding machine learning/artificial intelligence.
US 20190340558 A1 describes detailed analysis, including with various metrics, of a building and its associated processes, and utilizes machine learning algorithms to do so.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BIJAN MAPAR whose telephone number is (571)270-3674. The examiner can normally be reached Monday - Thursday, 11:00-8:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rehana Perveen can be reached at 571-272-3676. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BIJAN MAPAR/ Primary Examiner, Art Unit 2189
Prosecution Insights