DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is final.
This action is in response to the amendments filed on 01/02/2026.
Claims 1-20 are pending and have been considered.
Claims 6, 8, and 18 have been amended.
Claims 1- 20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter, a judicial exception, an abstract idea (mental process), without significantly more. No arguments have been made regarding the rejection.
Claims 1,2, 4, 5, 11-13, 15, 20 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU.
Claims 6, 9, 10, 16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Diaz-Vilarino et al 3D Modeling of Building Indoor Spaces and Closed Doors from Imagery and Point Clouds, Sensors, 15, 3491-3512, 2015, hereinafter DIA
Claims 3, 14 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Roland et al US 10060730 B2, hereinafter ROL
Claims 7, 17 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Quintana, et al, Door detection in 3D coloured point clouds of indoor environments, Automation in Construction 85, 146-166, 2018, hereinafter QUI.
Claims 8, 18 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Goyal et al US 20230106339, hereinafter GOY
Information Disclosure Statement (IDS)
The information disclosure statement (IDS) submitted on 01/20/2026, 02/24/2026 has been considered and is in compliance with the provisions of 37 CFR 1.97.
Response to Amendments/Arguments
The Examiner thanks the Applicant for the Amendments and Arguments filed on 01/02/2026 which have been considered.
The objections to claims 6, 8, and 18 have been withdrawn.
There were no amendments or arguments addressing the 35 U.S.C. 101 rejection. Claims 1-20 remain rejected under 35 U.S.C. 101.
The arguments regarding 35 U.S.C. 103 rejection have been considered but have not been found persuasive. The Applicant argues that the CUI reference does not teach “projecting the set of points onto a two-dimensional (2D) plane corresponding to the wall”. The Examiner disagrees. In broadest reasonable interpretation of the claim, CUI teaches the same thing by disclosing : RANSAC detects vertical surfaces, vertical walls are detected Projecting points into a wall: “Horizontal cross sections of the point clouds at certain heights from the floor are used to detect openings by identifying the number of points. In order to detect doors, point clouds are sliced horizontally into a series of sections
at different heights, as shown in Fig. 3(a)… Some sections will have no points at the opening of a door, but others will have points on top of the door. This difference can be used to detect the locations of doors, as shown in Fig. 3(b). “ Each intersection of the horizontal slice with the wall, where the number of points are counted, represents a projection, which is describes as projection line by line, in increasing heights above floor, on the wall. Intersection of two planes (each horizontal plane and vertical wall plane) being a line. Thus, the method describes a projection of the point clouds, for each group of points in a slice, onto the vertical plane corresponding to the wall.
Claims 1-20 remain rejected under 35 USC 103.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter, a judicial exception (abstract idea, mental process) and without significantly more.
(S1) Prima facie, claims 1-20 are each directed to a statutory category of invention: process (Claims 1-11 directed to a method), machine (claims 12-19 directed to a system) and manufacture (claim 20 directed to a non-transitory computer readable medium).
(S2A1) Claims are analyzed under the Alice/Mayo framework to determine if directed to an ineligible judicial exception. Recitation of judicial exceptions are highlighted in bold font. Paraphrasing (in italics) is used to simplify referencing. Claims with similar limitations (even though not verbatim identical) and sharing the same rationale under Alice/Mayo steps S1 and S2, are grouped, as follows. The analysis is performed on a representative claim of the group; an additional analysis is performed if any claims in the group have additional limitations.
Claim 1, representative for claims 12, 20, recites
[0] at a device having a processor:
(a) identifying a set of points representing a three-dimensional (3D) appearance of a physical environment, wherein the set of points correspond to a wall in the physical environment; this step involves observation, evaluation, recognition, identification activities which can be performed in the mind. For example a person looking at a set of LED lights in a garden in the night, can identify the ones associated with a vertical plane corresponding to a wall, distinguishing it from another lighted shape, e.g. a lighted Christmas tree. Human binocular/stereo vision and processing in the brain allows such identification. This is a mental process (see MPEP 2106.04(a)(2) III)
(b) projecting the set of points onto a two-dimensional (2D) plane corresponding to the wall, wherein each point of the set of points is projected to a location on the 2D plane; again a step that can be performed in the mind; for example, if LEDs in previous description are off the plane, our brain is able to picture where the point would collapse to a 2D plane of a wall – a different example of this being done in household environment is to mentally project on the wall a piece of furniture in case we want to estimate if the a table for example would fit below a window in the (plane of the) wall, or if it will partly overlap. This is a mental process (see MPEP 2106.04(a)(2) III)
(c ) detecting a window, door, or opening based on the set of points projected onto the 2D plane; another step which can be performed in the mind. Humans routinely detect/identify windows, doors or openings while examining the projection in a 2D plane, for example a person can detect doors, windows, opening into corridors etc by looking at a drawing, a painting, or (flat) TV screen. This is a mental process (see MPEP 2106.04(a)(2) III).
(d) generating a 3D floor plan based on the detecting of the window, door, or opening. While in principle generating a 3D floor plan is something that an architect could do in the mind and using pen and paper, due to the interpretation that 3D generation is not mental process, the limitation is interpreted here as an additional element, which merely limits the abstract idea of analyzing and recognizing spatial features to a particular technological environment (i.e. the context of floor plan generation) – a field of use limitation.
Having these steps done by a processor (and claims 12 and 20 further reciting a non-transitory computer-readable storage medium) does not change the nature of the mental process, as the courts do not distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. As the Federal Circuit has explained, "[c]ourts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind." Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015). See also Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1318, 120 USPQ2d 1353, 1360 (Fed. Cir. 2016) (MPEP § 2106.04(a)(2), subsection III)
Thus, the limitations of claims 1, 12, 20 recite abstract ideas . In broadest reasonable interpretation and in view of the application specification as well as the guidance from MPEP 2106.04 II. B, the limitations are considered together as a single abstract idea for further analysis, as a process aimed at: “analyzing and recognizing spatial features of a building environment” .
This is a combination that, under its broadest reasonable interpretation covers performance of limitations expressing observation, evaluation, judgement and decision-making performing identification in the mind. Nothing in the claim elements precludes the steps from being practically performed mentally or manually by a human. These are Mental Processes – Concepts Performed in the Human Mind (MPEP § 2106.04(a)(2), subsection III and Mathematical Concept (see MPEP 2106.04(a)(2) I) ).
Accordingly, claims 1, 12, 20 recite an abstract idea.
(S2A2)/(S2B) The additional claim elements (d) are merely field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Therefore, it is concluded that claims 1, 12, 20 are ineligible.
Claim 2 (representative of claim 13) continues to recite the abstract idea of claim 1 (respectively 12) and further recites a mental process, highlighted in bold below:
wherein identifying the set of points that correspond to the wall comprises identifying points of the set of points within a threshold distance of the wall.
A person could in the mind apply the comparison of a distance to a plane of wall with a threshold. For example, a stone/brick wall may protruding – sticking out from the plane of the wall, up to a threshold distance, and still identified and associated with the wall.
The limitation further refines the abstract idea of the independent claim. There are no additional elements that impose a meaningful limitation on the judicial exception – the claim does not recite significantly more than the abstract idea itself.
Thus, claims 2 and 13 are ineligible.
Claim 3 (representative of claim 14) continues to recite the abstract idea of claim 1 (respectively 12) and further recites an abstract idea, highlighted in bold below:
wherein projecting the set of points onto the 2D plane comprises orthographic projection.
The limitation recited in the claim further refines the independent’s claim mental process of projection onto a 2D plane. An orthographic projection orthographic is at the same time a mathematical concept (classified in Math World under projective geometry). There are no additional elements that impose a meaningful limitation on the judicial exception – the claim does not recite significantly more than the abstract idea itself.
Thus, claims 3 and 14 are ineligible.
Claim 4 continues to recite the abstract idea of claim 1 and further recites an abstract idea, highlighted in bold below:
wherein points of the set of points are associated with semantic labels and the projecting comprises a generating a 2D semantic map based on the projecting.
The limitation recited in the claim further refines the independent’s claim mental process of projection onto a 2D plane. A semantic map is no more than a drawing with interpretations for the drawn elements. Points on the contour/perimeter of a window projected onto 2D, as a collection or generating a contour line are part of a semantic map delimiting a concept (i.e. a window, or any box in a patent drawing to which a meaning is associated). There are no additional elements that impose a meaningful limitation on the judicial exception – the claim does not recite significantly more than the abstract idea itself.
Thus, claim 4 is ineligible.
Claim 5 continues to recite the abstract idea of claim 1 and further recites an abstract idea, highlighted in bold below:
wherein points of the set of points are associated with color information and the projecting comprises a generating a 2D color map based on the projecting.
The limitation recited in the claim further refines the independent’s claim mental process of projection onto a 2D plane. A color map is no more than a drawing with interpretations for the drawn colored points/lines/contours. Points on the contour/perimeter of a window projected onto 2D, as a collection or generating a colored contour line are part of a color map delimiting an object (i.e. a window, or door, etc.). There are no additional elements that impose a meaningful limitation on the judicial exception – the claim does not recite significantly more than the abstract idea itself.
Thus, claim 5 is ineligible.
Claim 15 continues to recite the abstract idea of claims 1 and 12 and further recites the abstract idea, of claim 4 and 5 highlighted in bold below:
wherein points of the set of points are associated with semantic labels and the projecting comprises a generating a 2D semantic map based on the projecting.
wherein points of the set of points are associated with color information and the projecting comprises a generating a 2D color map based on the projecting.
As per the analysis of claims 4 and 5, the limitations recite only abstract ideas, and under similar reasoning as for claims 4 and 5, absent any additional element to integrate the abstract idea into a practical application or provide significantly more, the claim is found ineligible.
Claim 6 (representative of claim 16) continues to recite the abstract idea of claim 1 (respectively 12) and to refine it with further limitations reciting mathematical concepts, highlighted in bold below:
wherein points of the set points are associated with normalized distance information, wherein the set of points and the 2D plane are associated with positions with a common 3D coordinate system and the normalized distance information corresponds to distances of the points of the set of points the 2D plane in the common 3D coordinate system. Calculation of distances in 3D coordinate systems is a recitation of mathematical concepts. The claim does not recite additional elements that would integrate the abstract idea into a practical application, or provide significantly more.
Thus, claims 6 and 16 are ineligible.
Claim 7 (representative of claim 17) continues to recite the abstract idea of claim 1 (respectively 12) and to refine it with further limitations reciting mathematical concepts, highlighted in bold below:
wherein: projecting the set of points comprises generating a 2D semantic map, a 2D color map, and a 2D points distance map; and
detecting the window, door, or opening is based on the set of points is based on the 2D semantic map, the 2D color map, and the 2D points distance map.
Generating a 2D semantic map, a 2D color map and a 2D points distance map is something humans perform in the mind and often draw on paper with a pen – or colored pens in case of the colored map. Essentially it reflects a drawing representation, either with a label (Semantic) color or a margin (distance). In such maps the objects such as windows or doors or opening have their contours marked, with lines or areas marked by labels, colored differently then the rest, or having a marked margin. Humans in certain professions, such as architects, construction and facilities engineering, etc, routinely perform such recognition in the mind, while examining building drawings. Thus the claim recites and abstract idea, further refining the abstract idea of the independent claim.
The claim does not recite additional elements that would integrate the abstract idea into a practical application, or provide significantly more. Thus, claims 7 and 17 are ineligible.
Claim 8 (representative of claim 18) continues to recite the abstract idea of claim 1 (respectively 12)
wherein detecting the window, door, or opening comprises 2D orthographic detection of a bounding box corresponding to a boundary of the window, door, or opening in the 2D plane.
As indicated in the Claim Objections Section, ‘orthographic detection’ is interpreted as detection being performed in the orthographic /2D projection. The claim continues to recite the abstract idea of independent claim, refining it with a specific modality of detection which includes the use of delimiting ‘box’, which is a natural cognitive shortcut which humans use to simplify recognition in terms of simplified geometric primitives. In fact computer vision AI bounding boxes are explicitly used to imitate this natural cognitive shortcut, to provide a coarse but intuitive way to localize an object before refining its shape. Often also used in engineering drawings to mark an object of interest, exactly because the person who indicates the object can imagine the bounding box and the person to whom the message is addressed expects it and recognizes it as such.
The claim does not recite additional elements that would integrate the abstract idea into a practical application, or provide significantly more. Thus, claims 8 and 18 are ineligible.
Claim 9 (representative of claim 19) continues to recite the abstract idea of claim 1 (respectively 12) further reciting
wherein detecting the window, door, or opening comprises detecting whether the window or door is open or closed. This is a process that most people would recognize instantly of being a mental process of observation and judgement of the open/close status of the door. The claim does not recite additional elements that would integrate the abstract idea into a practical application, or provide significantly more. Thus, claims 9 and 19 are ineligible.
Claim 10 continues to recite the abstract idea of claim 1. It further recites:
(a) the set of points and the 2D plane are associated with positions with a common 3D coordinate system; and (abstract idea of associating geometrical/mathematical concepts, 2D plane as an abstract mathematical /geometrical concept, also 3D coordinate system.)
(b) generating the 3D floor plan comprises generating a representation of the wall based on the set of points and (additional elements, which merely limit the abstract idea of analyzing and recognizing spatial features to a particular technological environment (i.e. the context of floor plan generation) – a field of use limitation.)
( c) positioning a representation of the window, door, or opening based in the 3D floor plan based on the common 3D coordinate system. (abstract idea of assigning position in 3D coordinate system, a mathematical concept)
(S2A2)/(S2B) The additional claim elements (b) are merely field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Therefore, it is concluded that claim 10 is ineligible.
Claim 11 continues to recite the abstract idea of claim 1. It further recites:
wherein generating the 3D floor plan comprises generating representations of window, door, or opening on representations of multiple walls of the physical environment. (additional elements, which merely limit the abstract idea of analyzing and recognizing spatial features to a particular technological environment (i.e. the context of floor plan generation) – a field of use limitation.)
(S2A2)/(S2B) The additional claim elements are merely field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Therefore, it is concluded that claim 11 is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
i. Determining the scope and contents of the prior art.
ii. Ascertaining the differences between the prior art and the claims at issue.
iii. Resolving the level of ordinary skill in the pertinent art.
iv. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims that share substantially similar limitations (even though not verbatim) are grouped and analyzed together; the analysis is done on the claim with most comprehensive limitations.
Claims 1,2, 4, 5, 11-13, 15, 20 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU.
Regarding Claim 1 (representative for claims 12, 20) CUI discloses
identifying a set of points representing a three-dimensional (3D) appearance of a physical environment, wherein the set of points correspond to a wall in the physical environment; projecting the set of points onto a two-dimensional (2D) plane corresponding to the wall, wherein each point of the set of points is projected to a location on the 2D plane; detecting a window, door, or opening based on the set of points projected onto the 2D plane; and {3119 right col, bottom]: 1) Horizontal and Vertical Plane Detection Using the dominant directions, the point clouds are transferred into a Manhattan coordinate frame. As the original data has no normal, they are approximated by the principal component analysis (PCA) using K-nearest neighbors then used to determine the rotation parameters [43]. Finally, the point clouds are transformed into the Manhattan frame. To extract walls from noisy point clouds, the RANSAC algorithm [28] is used to generate initial vertical planar surfaces…3220 right col 2) Opening Detection: For indoor modeling, doors and windows are considered as openings in segmented walls…The extraction of windows is similar to that of doors; Fig 3. Fig. 3. Detection of doors. (a) Split point clouds from different heights (red circles are locations of doors). (b) Point clouds on top of door openings. (c) Conversion of projected points into a binary image. (d) Detected polygons in 3-D representing doors. }. In broadest reasonable interpretation projecting the set of points onto a two-dimensional (2D) plane corresponding to the wall, wherein each point of the set of points is projected to a location on the 2D plane is taught by CUI disclosing : RANSAC detects vertical surfaces, vertical walls are detected. Projecting points into a wall: “Horizontal cross sections of the point clouds at certain heights from the floor are used to detect openings by identifying the number of points. In order to detect doors, point clouds are sliced horizontally into a series of sections at different heights, as shown in Fig. 3(a)… Some sections will have no points at the opening of a door, but others will have points on top of the door. This difference can be used to detect the locations of doors, as shown in Fig. 3(b). “ More clearly: each intersection of the horizontal slice with the wall, where the number of points are counted, represents a projection, which is describes as projection line by line, in increasing heights above floor, on the wall. Intersection of two planes (each horizontal plane and vertical wall plane) being a line. Thus, the method describes a projection of the point clouds, for each group of points in a slice, onto the vertical plane corresponding to the wall.
CUI does not explicitly disclose the use of a computer, and does not emphasize explicitly the generation of floorplan on detecting the window door or opening, although discloses 3-D reconstruction of indoor environment.
However GAU discloses:
a non-transitory computer-readable storage medium; and one or more processors {[Abstract] In some embodiments, a system is described comprising a memory that stores computer executable components, and a processor that executes the computer executable components stored in memory}
generating a 3D floor plan based on the detecting of the window, door, or opening. {(74) In various embodiments, a floorplan model can be a simplified representation of surfaces (e.g., walls, floors, ceilings, etc.), portals (e.g., door openings) and/or window openings associated with an interior environment. A floorplan model can contain locations of boundary edges for each given surface, portal (e.g., door opening), and/or window opening. (75) In various embodiments, a floorplan model generated by the 3D model generation component 118 can be a schematic floorplan of an architectural structure (e.g., house), a schematic floorplan of an interior space of an architectural structure (e.g., a house), etc. For example, the 3D model generation component 118 can generated a floorplan model of an architectural structure by employing identified walls associated with the derived 3D data 116 derived from 2D images captured of the architectural structure. In some implementations, the 3D model generation component 118 can employ common architectural notation to illustrate architectural features of an architectural structure (e.g., doors, windows, fireplaces, length of walls, other features of a building, etc.). n another example, a floorplan model can comprise a series of lines in 3D space which represent intersections of walls and/or floors, outlines of doorways and/or windows, edges of steps, outlines of other objects of interest (e.g., mirrors, paintings, fireplaces, etc.).}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI with GAU. One would have been motivated to do so, in order to obtain the advantage of obtaining and visualize the entire floor plan of the 3D room after obtaining the 2D wall projections based on windows doors or openings. Both CUI and GAU are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI and GAU would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable. Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU.
Regarding claim 4 CUI/GAU discloses the limitations of claim 1. CUI further discloses:
wherein points of the set of points are associated with semantic labels and the projecting comprises a generating a 2D semantic map based on the projecting. {see at least [3118] left col, bottom] 1) an automatic semantic operator, which allows individual rooms to be modeled, is proposed to semantically label the point clouds of indoor environment with a combination of visibility analysis and physical constraints of the structural elements; and 2) a multi-label graph cut is implemented to automatically 3-D reconstruct indoor models of multi-floors and complicated connections from MLS point clouds. The reconstructed results include both volumetric solid models and surface models, which have detailed geometric, topological and semantic information of indoor space and is suitable for both 3-D visualization and calculating purposes. [3128 right col, middle] For window and door detections, the method only relies on the geometric and semantic information of the point clouds. }
Regarding claim 11 CUI/GAU discloses the limitations of claim 1, CUI further discloses:
wherein generating the 3D floor plan comprises generating representations of window, door, or opening on representations of multiple walls of the physical environment.
{see at least Fig. 12. Results of the qualitative and quantitative evaluation. (a) Doors (green) and windows (red) combined with detected wall points on the first floor. (e) Visualization of recall and precision errors for the first floor measured with buffer size at 10 cm. }
PNG
media_image1.png
255
1254
media_image1.png
Greyscale
Figure above, fragment from Fig 2 in CUI shows windows and doors on multiple walls.
Regarding claim 5 CUI/GAU discloses the limitations of claim 1. GAU further discloses:
wherein points of the set of points are associated with color information and the projecting comprises a generating a 2D color map based on the projecting. {The stitching component 508 can further perform reverse-projecting of color data from colored point clouds or depth maps to create a single 2D panoramic image.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with further teaching of GAU. One would have been motivated to do so, in order to obtain the advantage of obtaining color maps which aid in segmentation of wall elements such as windows, doors and openings and help creating the floorplan.
Both CUI/GAU are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI and GAU would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU.
Regarding claims 2, 13 CUI/GAU discloses the limitations of claim 1, 12. GAU further discloses
wherein identifying the set of points that correspond to the wall comprises identifying points of the set of points within a threshold distance of the wall.{ FIG. 11 is a flowchart representation of an exemplary method according to some embodiments, the method generating and providing measurements of the wall structure based on 2D and 3D representations and measurements of 3D bounding boxes, the 3D bounding boxes representing measurements of corresponding objects.} identifying points of a wall comprising identification of points of a set of points within a threshold distance of the wall is interpreted as generating and providing measurements for the wall structure from representations of 3D bounding box, and associating the ‘3rd’ dimension of 3D with the width of the box around the wall corresponding to the distribution of points away from the wall, at a certain distance from the wall, thus the threshold distance is interpreted as being of a value equal to the ‘width’ value of the 3D bounding box dimension that in principle collapses from 3D to 2D at the surface of a perfect planar wall.
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with further teaching of GAU. One would have been motivated to do so, in order to cope with imperfections of a wall (not a perfect ‘ideal’ smooth vertical plane and also to imperfection of the measurement which may have errors placing pints in front or behind a 2D plane.
Both CUI/GAU are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI and GAU would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU.
Regarding claim 15 CUI/GAU discloses the limitations of claim 1, 4, 5, 12.
Claim 15 combines the limitations of claim 4 and 5.
wherein points of the set of points are associated with semantic labels and color information and the projecting comprises a generating a 2D semantic map and a 2d color map based on the projecting.
Based on the analysis for claims 4 and 5, which were found obvious in over CUI in view of GAU, claim 15 is also determined obvious over CUI in view of GAU.
Claims 6, 9, 10, 16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Diaz-Vilarino et al 3D Modeling of Building Indoor Spaces and Closed Doors from Imagery and Point Clouds, Sensors, 15, 3491-3512, 2015, hereinafter DIA
Regarding claim 6, 16 CUI/GAU discloses the limitations of claim 1, 12. DIA further discloses:
wherein points of the set points are associated with normalized distance information, wherein the set of points and the 2D plane are associated with positions with a common 3D coordinate system and the normalized distance information corresponds to distances of the points of the set of points to the 2D plane in the common 3D coordinate system. {[3496] The origin of the local coordinate system is settled in the origin of one of the laser scanner positions; [3498] In order to simplify the subsequent generation of a number of orthoimages corresponding to walls, the procedure starts with a point cloud rotation, in a way that floor and ceiling are parallel to the x-y plane and walls are parallel to either x-axis or y-axis, if possible … The procedure continues with a segmentation of the point cloud based on a seeded region-growing method [29], by which planar surfaces that form the indoor space are detected;
Fig 6 (fragment below) Figure 6. Deviation of the data to the fitted planes for (b) Colorbars are coded in centimeters } Distances interpreted as deviations in cm.
PNG
media_image2.png
382
1226
media_image2.png
Greyscale
Both x and y axis in centimeters, showing distances
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with teaching of DIA. One would have been motivated to do so, in order have the advantage of easier recognition of windows, from the ,opre distinguishable contours .
Both CUI/GAU and DIA are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI/GAU and DIA would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of DIA
Regarding claim 9, 19 CUI/GAU discloses the limitations of claim 1, 12. DIA further discloses:
wherein detecting the window, door, or opening comprises detecting whether the window or door is open or closed. {[3493 bottom] Other approaches are based on the use of color information. An illustration of such methods is the recognition and labeling of windows from thermal colored 3D point clouds [20]. As the material and the thermal conductivity is different in walls and in windows, a temperature map is used for detecting windows and labelling them as closed, open or damaged.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with teaching of DIA. One would have been motivated to do so, in order to allow both situations to be recognized, when windows or doors are closed, as well as when these are open.
Both CUI/GAU and DIA are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI/GAU and DIA would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of DIA
Regarding claim 10 CUI/GAU discloses the limitations of claim 1. DIA further discloses:
wherein: the set of points and the 2D plane are associated with positions with a common 3D coordinate system; and generating the 3D floor plan comprises generating a representation of the wall based on the set of points and positioning a representation of the window, door, or opening based in the 3D floor plan based on the common 3D coordinate system. {[3496] The origin of the local coordinate system is settled in the origin of one of the laser scanner positions; [3498] In order to simplify the subsequent generation of a number of orthoimages corresponding to walls, the procedure starts with a point cloud rotation, in a way that floor and ceiling are parallel to the x-y plane and walls are parallel to either x-axis or y-axis, if possible … The procedure continues with a segmentation of the point cloud based on a seeded region-growing method [29], by which planar surfaces that form the indoor space are detected; [3503 below fig 12] Final step consists on projecting the rectangles detected in the orthoimages on the 3D building envelope in order to get the 3D coordinates of the closed doors.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with teaching of DIA. One would have been motivated to do so, in order to have the advantage of using a known coordinate system which allows placement of a wall and elements of a wall at a distances that reflect planarity of the wall, windows and doors, as the points in the plane have specific relationships in the coordinate system.
Both CUI/GAU and DIA are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI/GAU and DIA would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of DIA
Claims 7, 17 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Quintana, et al, Door detection in 3D coloured point clouds of indoor environments, Automation in Construction 85, 146-166, 2018, hereinafter QUI.
Regarding claim 7, 17 CUI/GAU discloses the limitations of claim 1, 12. CUI/GAU does not disclose, however QUI discloses:
wherein: projecting the set of points comprises generating a 2D semantic map, a 2D color map, and a 2D points distance map; and detecting the window, door, or opening is based on the set of points is based on the 2D semantic map, the 2D color map, and the 2D points distance map. {[p. 146 left col, low] integrates both geometric and colour information, provided by a calibrated set of 3D laser scanner and a colour camera; [p. 148 top] Fig. 1. Illustration of the process used to construct the 3D voxel space and labels. Fig. 2. Front view of the labelled voxels of walls. Wall voxels can be labelled as Occupied, Occluded or Opening.}
PNG
media_image3.png
325
1289
media_image3.png
Greyscale
[p. 150 left col, middle] The output of the room scanning is composed of (1) a dense 3D coloured point cloud; (2) a labelled voxel model with associated 3D points from the point cloud; and (3) a 3D boundary model of the room composed of planar rectangular patches (and their associated voxels) representing the walls, ceilings and floors. …Our door detection process considers the labelling and coloured 3D points associated with the voxels of each wall rectangular segment.; [p. 152 right col.] Door recognition is carried out in two stages, using a wall's labelled voxels and its orthoimage }
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with teaching of QUI. One would have been motivated to do so, in order to have the advantage of improved detection using multiple sources of information, color, distance and labeling, which offers a superior detection.
Both CUI/GAU and DIA are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI/GAU and QUI would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable.
Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of QUI
Claims 8, 18 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Goyal et al US 20230106339, hereinafter GOY
Regarding claims 8, 18 CUI/GAU discloses the limitations of claim 1, 12. CUI further teaches:
wherein the projecting comprises orthographic projection; { “Horizontal cross sections of the point clouds at certain heights from the floor are used to detect openings by identifying the number of points. In order to detect doors, point clouds are sliced horizontally into a series of sections at different heights, as shown in Fig. 3(a)… Some sections will have no points at the opening of a door, but others will have points on top of the door. This difference can be used to detect the locations of doors, as shown in Fig. 3(b). “ More clearly: each intersection of the horizontal slice with the wall, where the number of points are counted, represents a projection, which is describes as projection line by line, in increasing heights above floor, on the wall. Intersection of two planes (each horizontal plane and vertical wall plane) being a line. Thus, the method describes a projection of the point clouds, for each group of points in a slice, onto the vertical plane corresponding to the wall. Horizontal plane and vertical plane are perpendicular. The projection is thus orthographic.
CUI/GAU teaches the orthographic projection does not disclose, however does not explicitly detect windows by the bounding box, instead, it uses the orthographic projection to aggregate the projected lines from orthographic projection, however GOY discloses:
and detecting the window, door, or opening comprises detection, based on the orthographic projection, of a bounding box corresponding to a boundary of the window, door, or opening in the 2D plane. { {[0103] The door placement is carried out by object detection module 134 using the following equations…where C.sub.BBI is a centroid of a bounding box of door detection (returned by door detection) in the corresponding 2D image,
}
PNG
media_image4.png
421
309
media_image4.png
Greyscale
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with teaching of GOY. One would have been motivated to do so, in order to facilitate the detection of windows, doors or opening in a limited area of the frame which is isolated (bounding box) for the purpose of more detailed analysis for efficient processing. Furthermore a POSITA would have used a bounding box to limit the location where an opening is detected. Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of GOY.
Claims 3, 14 are rejected under 35 U.S.C. 103 as being unpatentable over of Cui et al , Automatic 3-D Reconstruction of Indoor Environment With Mobile Laser Scanning Point Clouds et al, IEEE Journal of Selected Topics in Applied Earth Observation And Remote Sensing. Vol 12, No 8, 2019, hereinafter CUI in view of Gausebeck US 11094137 B2, hereinafter GAU, in further view of Roland et al US 10060730 B2, hereinafter ROL.
Regarding claims 3, 14 CUI/GAU discloses the limitations of claim 1, 12. CUI/GAU do not disclose, however ROL discloses:
wherein projecting the set of points onto the 2D plane comprises orthographic projection. { (48) The next step is to reduce the collection of 3D points to a collection of 2D points. This can be done by projecting all points onto a plane using an orthogonal projection.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of CUI/GAU with ROL. One would have been motivated to do so, in order to obtain the advantage of using a well known and common technique in projective geometry which allows the representation of a dispersion of points onto the plane that best match the 3D distribution. Both CUI/GAU and ROL are in the same art, and implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Since the elements disclosed by CUI and GAU would function in the same manner in combination as they do in their separate embodiments, the results of the combination would be predictable. Accordingly, the claimed subject matter would have been obvious over CUI in view of GAU in further view of ROL.
Prior art made of record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
CN 110100217 A
CN 111612806
CA 3090629 A1
US 12014120 B2
US 20190026956 A1
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADRIAN STOICA whose telephone number is (571) 272-3428. The examiner can normally be reached Monday to Friday, 9 a.m. -5 p.m. PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached on (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S./Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights