Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over WI et al (US 20250191274) in view of CHEN et al (Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction) and FOLEY et al (US 20250209223).
As per claim 1, Wi teaches the claimed “method for a floor area estimator,” the method comprising: “receiving a three-dimensional (3D) representation of a building” (Wi, [0044]-[0049] - Polygons are triangles (mesh) that make up a 3D building model, and may include the texture and geometry data of the atlas image through UV mapping); “identifying one or more levels of the building” (Wi, [0051] - Here, the style-transfer-preprocessing unit 400 may perform preprocessing for setting the area to which style transfer is to be applied by generating a mask image through segmentation into a window and a wall of the building model image and generating a floor grid area based on the segmentation into the window and the wall, [0146]-[0151] - The floor grid area may correspond to an area representing each floor of a building, and may be estimated depending on the position relationship of the window and the wall); “performing a transformation to a 3D representation of each of the one or more levels to generate a 2D plan view projection of each of the one or more levels” (Wi, [0156] - That is, at step S130, a user-style image, the mask image of the window and the wall, the floor grid area, and the 2D building model image are input, the style of the 2D building model image is transferred, and the 2D building model image may be converted into 3D building model texture information). Furthermore, Chen teaches “processing, using an artificial neural network, the 2D plan view projection of each of the one or more levels to generate 2D semantically segmented masks of each of the one or more levels” (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes; page 11, Reconstruction with retrieved 3D CAD models - Each point cloud cluster is passed to PointNet (Figure 2) to obtain feature vectors, extracted from the second-to-last layer of the neural network, to represent each model object. The same process is repeated for objects detected in the point cloud scene… Example results for CAD modeling of indoor environments are shown in Figures 6 and 7, where building components such as floors and walls as well as furniture objects such as tables, chairs, and bookcases can be successfully retrieved). Noted: Chen’s 3D model of the floor can be projected into the floor plane to generate a 2D semantically segmented masks of the floor (e.g., Wi, [0138]-[0140] - Also, at step S120, the 3D building model texture may be projected to the 2D building model image at step S230… That is, at step S240, a facade image may be generated by projecting the 3D building model geometry data and the atlas image to the 2D building model image) for a purpose of applying Chen’s semantic segmentation on the 2D projected floor plan to calculate the area based on the fixed size of the grid such as “calculating a floor area for each of the one or more levels by quantifying segments in the 2D semantically segmented masks” (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes; page 11, Reconstruction with retrieved 3D CAD models - Each point cloud cluster is passed to PointNet (Figure 2) to obtain feature vectors, extracted from the second-to-last layer of the neural network, to represent each model object. The same process is repeated for objects detected in the point cloud scene… Example results for CAD modeling of indoor environments are shown in Figures 6 and 7, where building components such as floors and walls as well as furniture objects such as tables, chairs, and bookcases can be successfully retrieved) (see also Foley, [0272]-[0274] - Calculate the floor area, either net or Calculation normally starts with the entire floor plate. gross, whichever is applicable… Gross Floor Area may be considered a floor area within the inside perimeter of the outside walls of the building under consideration with no deductions for hallways, stairs, closets, thickness of interior walls, columns, elevator and building services shafts, or other features, but excluding floor openings associated with atriums and communicating spaces… Net Floor Area may be considered a floor area within the inside perimeter of the outside walls, or the outside walls and fire walls of a building, or outside and/or inside walls that bound an occupancy or incidental use area requiring the occupant load to be calculated using net floor area under consideration with deductions for hallways, stairs, closets, thickness of interior walls, columns, or other features). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by using a projected 2D floor plane to calculate the area of the floor of a building. The motivation is to decide the usable area of floor with different classified regions.
Claim 2 adds into claim 1 “performing a second transformation to the 3D representation of the building to generate a two-dimensional (2D) elevation view of a side or slice of the building” (Wi, [0138]-[0140] - Also, at step S120, the 3D building model texture may be projected to the 2D building model image at step S230… That is, at step S240, a facade image may be generated by projecting the 3D building model geometry data and the atlas image to the 2D building model image); and “processing, using a second artificial neural network, the 2D elevation view of the side or slice of the building to identify the one or more levels of the building” (Wi, [0044]-[0049] - Polygons are triangles (mesh) that make up a 3D building model, and may include the texture and geometry data of the atlas image through UV mapping); “identifying one or more levels of the building” (Wi, [0051] - Here, the style-transfer-preprocessing unit 400 may perform preprocessing for setting the area to which style transfer is to be applied by generating a mask image through segmentation into a window and a wall of the building model image and generating a floor grid area based on the segmentation into the window and the wall, [0146]-[0151] - The floor grid area may correspond to an area representing each floor of a building, and may be estimated depending on the position relationship of the window and the wall).
Claim 3 adds into claim 2 “wherein the second artificial neural network performs object detection to generate bounding boxes corresponding to the building levels or floor slabs, and wherein the 2D elevation view comprises a color element and a horizontal point density element, and wherein the second artificial neural network uses the color element and the horizontal point density element as inputs for two input channels” (Wi, [0138]-[0140] - Also, at step S120, the 3D building model texture may be projected to the 2D building model image at step S230… That is, at step S240, a facade image may be generated by projecting the 3D building model geometry data and the atlas image to the 2D building model image) (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes; page 11, Reconstruction with retrieved 3D CAD models - Each point cloud cluster is passed to PointNet (Figure 2) to obtain feature vectors, extracted from the second-to-last layer of the neural network, to represent each model object. The same process is repeated for objects detected in the point cloud scene… Example results for CAD modeling of indoor environments are shown in Figures 6 and 7, where building components such as floors and walls as well as furniture objects such as tables, chairs, and bookcases can be successfully retrieved). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by using a projected 2D floor plane to calculate the area of the floor of a building. The motivation is to decide the usable area of floor with different classified regions.
Claim 4 adds into claim 1 “wherein the artificial neural network performs semantic segmentation and classifies areas of the one or more levels into floor areas and non-floor areas” (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes; page 11, Reconstruction with retrieved 3D CAD models - Each point cloud cluster is passed to PointNet (Figure 2) to obtain feature vectors, extracted from the second-to-last layer of the neural network, to represent each model object. The same process is repeated for objects detected in the point cloud scene… Example results for CAD modeling of indoor environments are shown in Figures 6 and 7, where building components such as floors and walls as well as furniture objects such as tables, chairs, and bookcases can be successfully retrieved; Foley, [0272]-[0274] - Calculate the floor area, either net or Calculation normally starts with the entire floor plate. gross, whichever is applicable… Gross Floor Area may be considered a floor area within the inside perimeter of the outside walls of the building under consideration with no deductions for hallways, stairs, closets, thickness of interior walls, columns, elevator and building services shafts, or other features, but excluding floor openings associated with atriums and communicating spaces… Net Floor Area may be considered a floor area within the inside perimeter of the outside walls, or the outside walls and fire walls of a building, or outside and/or inside walls that bound an occupancy or incidental use area requiring the occupant load to be calculated using net floor area under consideration with deductions for hallways, stairs, closets, thickness of interior walls, columns, or other features). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by performing semantic segmentation and classifying areas of the one or more levels into floor areas and non-floor areas of a building. The motivation is to decide the usable area of floor with different classified regions.
Claim 5 adds into claim 1 “wherein the 2D plan view projection comprises a color element, a vertical point density element, and an item element” (Wi, [0138]-[0140] - Also, at step S120, the 3D building model texture may be projected to the 2D building model image at step S230… That is, at step S240, a facade image may be generated by projecting the 3D building model geometry data and the atlas image to the 2D building model image) (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes), and “wherein the artificial neural network uses the color element, the vertical point density element, and the item element as inputs for three input channels” (Foley, [0093]-[0095] - Any or all of the components in a user interface may be converted to a version that allows a user to modify an attribute of the components, such as the length, size, beginning point, end point, thickness, or other attribute. In some embodiments, a boundary may be treated as a component or a wall and manipulated in a similar manner… In general, a vertex is a data structure that can describe certain attributes, like the position of a point in a two-dimensional or three-dimensional space. It may also include other attributes, such as normal vectors, texture coordinates, colors, or other useful attributes; [0246] - At step 1004, a scale of components included in the design plan may be determined. The scale may be determined, for example via a scale indicator or ruler included in the design plan, or inclusion in the design plan of a component of a known dimension). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by performing semantic segmentation and classifying areas of the one or more levels into floor areas and non-floor areas of a building. The motivation is to decide the usable area of floor with different classified regions.
Claim 6 adds into claim 5 “generating the color element by identifying points of the 3D representation of each of the levels that are contained within a vertical column that are contained within vertical columns across each of the one or more levels” (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes), and “setting a color of an associated pixel in the 2D plan view projection as an average of those points” (Wi, [0138]-[0140] - Also, at step S120, the 3D building model texture may be projected to the 2D building model image at step S230… That is, at step S240, a facade image may be generated by projecting the 3D building model geometry data and the atlas image to the 2D building model image; Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by performing the 3D semantic representation of each of the levels of floor areas including a vertical column of a building. The motivation is to decide the usable area of floor with different classified regions.
Claim 7 adds into claim 5 “generating the vertical point density element by determining points of the 3D representation of each of the levels that are contained within vertical columns across each of the one or more levels, and setting an associated pixel to a grayscale value based on a number of points in each of the vertical columns” (Foley, [0093]-[0095] - Any or all of the components in a user interface may be converted to a version that allows a user to modify an attribute of the components, such as the length, size, beginning point, end point, thickness, or other attribute. In some embodiments, a boundary may be treated as a component or a wall and manipulated in a similar manner… In general, a vertex is a data structure that can describe certain attributes, like the position of a point in a two-dimensional or three-dimensional space. It may also include other attributes, such as normal vectors, texture coordinates, colors, or other useful attributes; [0246] - At step 1004, a scale of components included in the design plan may be determined. The scale may be determined, for example via a scale indicator or ruler included in the design plan, or inclusion in the design plan of a component of a known dimension). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by performing the 3D semantic representation of each of the levels of floor areas including the setting of a grayscale value based on a number of points in each of the vertical columns (e.g., Foley, [0246] - inclusion in the design plan of a component of a known dimension). The motivation is to decide the usable area of floor with different classified regions.
Claim 8 adds into claim 5 “generating the vertical point density element by determining points of the 3D representation of each of the levels that are contained within vertical columns across each of the one or more levels and that are less than or equal to a target height, and setting an associated pixel to a grayscale value based on a distance of a closest point to the target height” (Foley, [0246] - At step 1004, a scale of components included in the design plan may be determined. The scale may be determined, for example via a scale indicator or ruler included in the design plan, or inclusion in the design plan of a component of a known dimension) (Noted: Foley’s target height can be the height of the building) . Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by performing the 3D semantic representation of each of the levels of floor areas including the setting of a grayscale value based on a number of points in each of the vertical columns (e.g., Foley, [0246] - inclusion in the design plan of a component of a known dimension). The motivation is to decide the usable area of floor with different classified regions
Claim 9 adds into claim 1 “processing the 2D plan view with one or more additional artificial neural networks trained and employed to detect specific categories of areas to exclude from the 2D semantically segmented mask based on a set of exclusion rules” (Chen, page 8, Component Merging - The final results of object segmentation, classification, and merging from point clouds are visualized for two different indoor scenes in Figures 4 and 5. The subfigures shown are the: (i) panoramic image (ii) original point cloud; (iii) segmentation results (each segment is shown in a different color); (iv) classification results (yellow: ceiling, dark blue: floor, orange: wall, green: door, red: bookcase, pink: beam, light blue: column, gray: clutter; and (v) object merging visualized with bounding boxes; page 11, Reconstruction with retrieved 3D CAD models - Each point cloud cluster is passed to PointNet (Figure 2) to obtain feature vectors, extracted from the second-to-last layer of the neural network, to represent each model object. The same process is repeated for objects detected in the point cloud scene… Example results for CAD modeling of indoor environments are shown in Figures 6 and 7, where building components such as floors and walls as well as furniture objects such as tables, chairs, and bookcases can be successfully retrieved). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by excluding from the 2D semantically segmented mask based on a set of exclusion rules (e.g., Chen, page 8, Component Merging - wall, door, bookcase, …). The motivation is to decide the usable area of floor with different classified regions.
Claim 10 adds into claim 1 “generating inputs for a training dataset by performing a sliding window crop operation to a series of training representations to generate a set of input crops; generating outputs for the training dataset by performing the sliding window crop operation to a series of building models comprising building elements relevant to defining floor areas to generate a set of output crops (Foley, [0090] - At step 104, an AI engine may ascertain features included in the design plan, the AI engine may additionally ascertain that a feature is located within a particular set of boundaries or external to the set of boundaries. Features may include, by way of non-limiting example, one or more of: architectural aspects, fixtures, duct work, wiring, piping, or other item included in a two-dimensional reference submitted to be analyzed. The features and boundaries may be determined, for example, via algorithmically processing an input design plan image with a trained AI model. As a non-limiting example, the AI engine may process a raster file that is converted for output as an image file of a floorplan (as illustrated in FIG. 2B, a boundary is represented as line, a boundary may also be represented as a polygon, which may be a patterned polygon or other user discernable representation, such as a colored line etc.). Features may also be designated on a user interface); augmenting the training dataset by rotating and mirroring the input crops and the output crops to generate additional input-output pairs; and training the artificial neural network using the augmented training dataset” which Foley suggests in using AI engine to train floor building models to define the floor elements (Foley, [0089] - In still another aspect, in some embodiments, a controller may access data from various types of BIM and Computer Aided Drafting (CAD) design programs and import dimensional and shape aspects of select spaces or portions of the designs as they are related to a design plan). Thus, it would have been obvious, in view of Foley and Chen, to configure Wi’s method as claimed by collecting training dataset by performing the sliding window crop operation (e.g., Foley, [0090] - Features may also be designated on a user interface) a to a series of building models comprising building elements relevant to defining floor areas. The motivation is to decide the usable area of floor with different classified regions.
Claims 11-19 and 20 claim an apparatus and a non-transitory computer-readable storage medium based on the method of claims 1-10; therefore, they are rejected under a similar rationale.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHU K NGUYEN whose telephone number is (571)272-7645. The examiner can normally be reached M-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHU K NGUYEN/Primary Examiner, Art Unit 2616
Prosecution Insights