DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1 and 11 have been amended and claims 5, 8, 15, and 18 have been cancelled; as a result, claims 1-4, 6, 7, 9-14, 16, 17, 19, and 20 are currently pending in the present application, with claims 1 and 11 being independent.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 20 March 2024 has been considered by the examiner.
Response to Arguments
Applicant’s arguments, see page 7, filed 15 July 2026, with respect to duplicate claim notification of claims 4, 5, 14, and 15, along with accompanying amendments received on the same day, have been fully considered and are persuasive. Claims 5 and 15 have been cancelled. The duplicate claim notification/warning has been withdrawn.
Applicant’s arguments, see page 7, filed 15 July 2026, with respect to the claim objections of claims 1 and 11, along with accompanying amendments received on the same day, have been fully considered and are persuasive. The objections to the claims have been withdrawn.
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on the previous combination of references as applied in the most recent Office Action.
The examiner agrees that the previous rejection, as set forth in the most recent Office Action, mailed on 15 January 2026, does not appear to explicitly teach the claimed subject matter. However, Sato et al (US PG Publication 2013/0194276), in combination with the previous references does teach the newly amended subject matter as set forth in the rejection below. It is also noted, that the claims are being afforded their broadest reasonable interpretation in light of the corresponding disclosure and are not limited by the examples provided by the applicant in their remarks.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 6, 9-13, 16, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al. (“Automatic Analysis and Sketch-Based Retrieval of Architectural Floor Plans”, 2014) in view of Stockton (“Building Designs by Stockton”, 2021) in further view of Wessel et al. (“The Room Connectivity Graph: Shape Retrieval in the Architectural Domain”, 2008) further in view of Sato et al. (US PG Publication 2013/0194276).
Regarding claim 1, Ahmed teaches a system for architectural floorplan image analytics (see for instance, abstract), comprising:
a database for storing a floorplan image (see for instance, page 98, section 4.1. Floorplan Analysis Evaluation”, page 99, section 5. “Conclusion and future work”, paragraph 2, and fig. 1.); and
a processor in communication with the database, the processor configured to perform the steps of (The system is implemented on a computer, and while not explicitly stated by Ahmed, it would have been obvious to a person of ordinary skill in the art on the effective filing date of the invention, that a computer would comprise a processor in communication with a database and the processor would perform various functions):
retrieving the floorplan image from the database (see for instance, page 98, section 4.1. Floorplan Analysis Evaluation and fig. 1. “Our system is evaluated using a data set containing original floor plan images. This data set was introduced by Macé et al. (2010) and contains the floor plan images”, see for instance, page 98, section 4.1. Floorplan Analysis Evaluation, paragraph 1);
applying object detection to the floorplan image to identify one or more floors of the floorplan image;
applying segmentation to the one or more floors of the floorplan image to identify one or more entities of the one or more floors of the floorplan image (see for instance, figs. 1, 2, and 5. “Floor plans contain information that collectively help an architect to express the actual dynamics of the building...One of the key points of the proposed method is its fine segmentation of different types of information available in floor plans, e.g., walls, symbols, text, etc”, see for instance, page 94, section 3.1.1. Information Segmentation”, paragraph 1. “To detect the actual bounds of rooms, the image with the closed gaps is inverted and connected component analysis is performed on it. All of the very small connected components are removed, whereas each of the remaining connected component is referred as room. The detected rooms can be found in Fig. 2b.3.”, see for instance, page 95, section 3.1.3. “Semantic Analysis”, paragraph 3 and fig. 2);
extracting text from the floorplan image (see for instance, fig. 1 and page 94, section 3.1.1. “Information Segmentation”. First, text/graphics segmentation is performed using the methods presented by Ahmed et al. (2011b). This method is based on method by Tombre et al. (2002) with number of improvements specifically floor the floor plans. Text/graphics segmentation separate the text from the graphics in the floor plan image”, see for instance, page 94, section 3.1.1. “Information Segmentation”, paragraph 2);
preforming named entity recognition on the extracted text (“After detection of rooms the next step is to define there functions like WC, Living room, etc. In order to find the function of each room, the text layer from the information segmentation as well as the connected component of the room is used. In particular, all text components which lie in the boundary of a room are taken into account. After extraction of the room text, horizontal and vertical smearing is performed on the extracted text to merge the neighboring characters, resulting in the bounds for words. Using the bounding boxes all the words are rotated to a horizontal direction and OCR is performed on them”, see for instance, page 95, section 3.1.3 Semantic Analysis, paragraph 3);
associating the one or more identified entities with one or more corresponding recognized entities (“The OCR result is then compared to rooms title dictionary and the closest title according to the Levenshtein distance is assigned to the room”, see for instance, page 95, section 3.1.3 Semantic Analysis, paragraph 3);
generating one or more nodes corresponding to to the one or more identified entities (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, page 96, section 3.3. “Graph Structure”, paragraph 1), each of the one or more nodes defining an entity type, a dimension, and a floor of the one or more identified entities (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, Ahmed, page 96, section 3.3. “Graph Structure”, paragraph 1); and
creating edges between nodes having a connective relationship (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, page 96, section 3.3. “Graph Structure”, paragraph 1).
While Ahmed teaches segmenting different types of information for floor plans for a given floor, such as walls, symbols, text, etc, Ahmed does not appear to teach applying object detection to the floorplan image to identify one or more floors of the floorplan image. In addition, Ahmed does not teach that each of the one or more nodes defines an entity type, a dimension, and a floor of the one or more identified entities.
In the same art of floor plans, Stockton teaches that a floorplan image can contain multiple floors, see for instance, Stockton, page 1.
It would have been obvious to one of ordinary skill in the art having the teachings of Ahmed and Stockton in front of them before the effective filing date of the claimed invention to incorporate multiple floors on a plan as taught by Stockton into Ahmed’s architectural floor plan analysis system, as having plan have multiple floors, such as described by Stockton was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Ahmed.
The modification of Ahmed with Stockton would have explicitly allowed the floor plan discussed in Ahmed to contain multiple floors.
The motivation for combining Ahmed with Stockton would have been to improve the user experience, enhance functionality by allowing for the use of known floor plan types, and a extend the floor plan of Ahmed to allow for multiple floors.
While Ahmed in view of Stockton teach segmenting different types of information for floor plans for a given floor, such as walls, symbols, text, etc, they do not appear to explicitly teach applying object detection to the floorplan image to identify one or more floors of the floorplan image. In addition, Ahmed in view of Stockton do not teach that each of the one or more nodes defines an entity type, a dimension, and a floor of the one or more identified entities.
In the same art of architecture, Wessel teaches the extraction of room connectivity graphs consists of three steps, the detection of building stories, the determination of story rooms, and finally the determination of doors and windows connecting these rooms, see for instance, section 3 ‘Room Connectivity Graph Extraction’. Nodes can have multiple attributes, such as area, façade, and extent, see for instance, section 3.3 Room detection.
It would have been obvious to one of ordinary skill in the art having the teachings of Ahmed, Stockton, and Wessel in front of them before the effective filing date of the claimed invention to incorporate detecting of building stories as taught by Wessel into Ahmed’s architectural floor plan analysis system, as detecting the number of floors, such as described by Wessel was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Ahmed and Stockton.
The modification of Ahmed and Stockton with Wessel would have explicitly allowed applying object detection to the floorplan image to identify one or more floors of the floorplan image.
The motivation for combining Ahmed and Stockton with Wessel would have been to improve the user experience, enhance functionality, and to use references familiar to Ahmed (see section 2.4 of Ahmed).
While Ahmed in view of Stockton in further view of Wessel teach a node having multiple attributes including a level, unit, zone, or room, area, and extent, they do not appear to explicitly teach that each of the one or more nodes defines an entity type, a dimension, and a floor of the one or more identified entities.
In the same art of floor plans, Sato teaches generating graph information by defining pieces of space formed by three-dimensional structures as nodes and defining edges for linking the defined nodes, see for instance, paragraph 68. Attributes of the nodes are shown in Fig. 8, which shows (1) 3D space coordinate values (m) of starting point and (2) lengths (m) in axis directions (3D directions) of space corresponding to each node, (3) a value represented by furniture, etc, (4) weight refers to, for example a value obtained by subtracting the volume of furniture or the like from the volume of space corresponding to each node, see for instance, paragraphs 75 and 76.
It would have been obvious to one of ordinary skill in the art having the teachings of Ahmed, Stockton, Wessel, and Sato in front of them before the effective filing date of the claimed invention to incorporate multiple attribute nodes as taught by Sato into Ahmed’s architectural floor plan analysis system, as having a node represent multiple attributes, such as dimensions, location, furniture present, and weight, such as described by Sato was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Ahmed, Stockton, and Wessel.
The modification of Ahmed, Stockton, and Wessel with Sato would have explicitly allowed each of the one or more nodes to define an entity type, a dimension, and a floor of the one or more identified entities.
The motivation for combining Ahmed, Stockton, and Wessel with Sato would have been to improve the user experience, enhance functionality, and to further expand on the multi-attribute nodes of Ahmed in view of Stockton in further view of Sato.
Regarding claim 2, Ahmed in view of Stockton in view of Wessel in further view of Sato teach the system of claim 1 and further teach wherein the processor applies optical character recognition to floorplan image to extract the text therefrom (First, text/graphics segmentation is performed using the methods presented by Ahmed et al. (2011b). This method is based on method by Tombre et al. (2002) with number of improvements specifically floor the floor plans. Text/graphics segmentation separate the text from the graphics in the floor plan image”, see for instance, Ahmed, page 94, section 3.1.1. “Information Segmentation”, paragraph 2 “After extraction of the room text, horizontal and vertical smearing is performed on the extracted text to merge the neighboring characters, resulting in the bounds for words. Using the bounding boxes all the words are rotated to a horizontal direction and OCR is performed on them. The OCR result is then compared to rooms title dictionary and the closest title according to the Levenshtein distance is assigned to the room”, see for instance, Ahmed, page 95, section 3.1.3., Semantic Analysis, paragraph 3). The motivation to combine Ahmed, Stockton, Wessel, and Sato is the same as that which was set forth in claim 1.
Regarding claim 3, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 1, and further teach wherein the processor applies a bounding box to each of the one or more floors of the floorplan image (“After extraction of the room text, horizontal and vertical smearing is performed on the extracted text to merge the neighboring characters, resulting in the bounds for words. Using the bounding boxes all the words are rotated to a horizontal direction and OCR is performed on them. The OCR result is then compared to rooms title dictionary and the closest title according to the Levenshtein distance is assigned to the room”, see for instance, Ahmed, page 95, section 3.1.3., Semantic Analysis, paragraph 3. Thick lines can be used to construct the boundary of the building, see for instance, Ahmed, page 94, section 3.1.1 Information Segmentation, paragraph 4). The motivation to combine Ahmed, Stockton, Wessel, and Sato is the same as that which was set forth in claim 1.
Regarding claim 6, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 1 and further teach wherein the processor generates a multi-attributed graph for each of the one or more identified floors of the floorplan image (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, Ahmed, page 96, section 3.3. “Graph Structure”, paragraph 1). The motivation to combine Ahmed, Stockton, Wessel, and Sato is the same as that which was set forth in claim 1.
Regarding claim 9, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 1 and further teach wherein the edges include associated edge attributes, including a connectivity type (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, Ahmed, page 96, section 3.3. “Graph Structure”, paragraph 1). The motivation to combine Ahmed, Stockton, and Wessel is the same as that which was set forth in claim 1.
Regarding claim 10, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 9 and further teach wherein the edges indicate a directional connective relationship between adjoining rooms of a floor or a vertical connective relationship between vertically aligned rooms of different floors (“The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, Ahmed, page 96, section 3.3. “Graph Structure”, paragraph 1). The motivation to combine Ahmed, Stockton, Wessel, and Sato is the same as that which was set forth in claim 1.
Regarding claim 11, claim 11 is the method claim corresponding to the system claim 1 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 1.
Regarding claim 12, claim 12 recites substantially similar subject as to that which is recited in claim 2 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 2.
Regarding claim 13, claim 13 recites substantially similar subject as to that which is recited in claim 3 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 3.
Regarding claim 16, claim 16 recites substantially similar subject as to that which is recited in claim 6 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 6.
Regarding claim 19, claim 19 recites substantially similar subject as to that which is recited in claim 9 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 9.
Regarding claim 20, claim 20 recites substantially similar subject as to that which is recited in claim 10 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 10.
Claim(s) 4, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al. (“Automatic Analysis and Sketch-Based Retrieval of Architectural Floor Plans”, 2014) in view of Stockton (“Building Designs by Stockton”, 2021) in further view of Wessel et al. (“The Room Connectivity Graph: Shape Retrieval in the Architectural Domain”, 2008) further in view of Sato et al. (US PG Publication 2013/0194276), as applied to claims 1 and 11 above, in further view of Dodge et al. (“Parsing Floor Plan Images”, 2017).
Regarding claim 4, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 1, but do not appear to explicitly teach wherein the processor extracts entity size information from the extracted text and associates the extracted entity size information with a recognized entity.
In the same art of floor plan analysis, Dodge teaches architectural floor plans are scaled drawings of apartments or building layouts. They contain structural and semantic information, e.g., room types and sizes, and the locations of doors, windows, and fixtures...In this paper we instead focus on readily available floor plan images from real estate websites”, see for instance, page 358, section 1 Introduction, paragraph 1. Having extracted the sizes of walls from OCR, we are able to place furniture items into the model at the correct scale, see for instance, page 458, section 1 Introduction, paragraph 1. We combine three methods to extract geometric and semantic information: Wall segmentation, object detection, and optical character recognition (OCR), see for instance, page 358, section 3 Parsing floor plans, paragraph 1. Our input images may contain both English and Japanese text. We use the Google vision API for text detection and character recognition, which handles multiple languages”, see for instance, page 359, “Optical Character Recognition”, paragraph 1. Room sizes are read with OCR in the input image (left) and propagated to unlabeled rooms (right), see for instance, page 361, fig. 4. For our applications, the most important text is the room size, see for instance, page 361, Text detection performance, paragraph 1. “For each segmented room, we query the text information for the Japanese room measurement unit (Jo). We compute the relationship between the room size in physical units and pixels to compute the pixel density (in pixel/J). This pixel density can be used to compute an estimate of the area of rooms that are not labeled with physical units, and well as can be used to compute wall length”, see for instance, page 361, section 4.1 Applications: 3D Model Creation and Furniture Fitting, paragraph 2. From the floor plan we extract a parsed representation of wall locations, objects, and size information, see for instance, page 361, section 5 Conclusion, paragraph 2.
It would have been obvious to one of ordinary skill in the art having the teachings of Ahmed, Stockton, Wessel, Sato, and Dodge in front of them before the effective filing date of the claimed invention to incorporate detecting entity size as taught by Dodge into Ahmed’s architectural floor plan analysis system, as detecting the room size, such as described by Dodge was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Ahmed, Stockton, Wessel, and Sato.
The modification of Ahmed, Stockton, Wessel, and Sato with Dodge would have explicitly allowed the processor to extract entity size information from the extracted text and associates the extracted entity size information with a recognized entity.
The motivation for combining Ahmed, Stockton, Wessel, and Sato with Dodge would have been to improve the user experience, enhance functionality, and allowed the OCR described in Ahmed to explicitly extract the room size.
Regarding claim 14, claim 14 recites substantially similar subject as to that which is recited in claim 4 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 4.
Claim(s) 7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al. (“Automatic Analysis and Sketch-Based Retrieval of Architectural Floor Plans”, 2014) in view of Stockton (“Building Designs by Stockton”, 2021) in further view of Wessel et al. (“The Room Connectivity Graph: Shape Retrieval in the Architectural Domain”, 2008) further in view of Sato et al. (US PG Publication 2013/0194276), as applied to claims 6 and 16 above, in further view of Khosravan et al. (US Patent 11,830,135).
Regarding claim 7, Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the system of claim 6, and further teach wherein the processor merges the multi-attributed graphs for each of the one or more identified floors of the floorplan image to generate a combined multi-attributed graph corresponding to all identified floors of the floorplan image (“The extracted semantics are represented as a graph G = (V,E). The vertices V have a type Tvertex reflecting a level, unit, zone, or room. The edges E also have different types Tedge indicating if the vertices are connected directly or are just adjacent, both of these relations are symmetric”, see for instance, Ahmed, page 96, section 3.3. “Graph Structure”, paragraph 1. “Future work should also consider building units that connect different stories. By that, the room connectivity graphs of each story will be interlinked by edges representing staircases and elevator shafts”, see for instance, Wessel, page 80, section 6 Conclusion, paragraph 2). The motivation to combine Ahmed, Stockton, and Wessel is the same as that which was set forth in claim 1.
While Ahmed in view of Stockton in further view of Wessel in further view of Sato teach the broadest reasonable interpretation of claim 7, Khosravan is being brought into the rejection to explicitly teach the implementation of a multi-attributed graph corresponding to multiple floors.
In the same art of floor plans, Khosravan teaches a multi-attributed graph corresponding to multiple floors of a floor plan, see for instance, column 22, lines 36-67 and fig. 2E.
It would have been obvious to one of ordinary skill in the art having the teachings of Ahmed, Stockton, Wessel, Sato, and Khosravan in front of them before the effective filing date of the claimed invention to incorporate as taught by Khosravan into Ahmed’s architectural floor plan analysis system, as detecting the room size, such as described by Khosravan was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Ahmed, Stockton, and Wessel.
The modification of Ahmed, Stockton, Wessel, and Sato with Khosravan would have explicitly allowed the processor to merge the multi-attributed graphs for each of the one or more identified floors of the floorplan image to generate a combined multi-attributed graph corresponding to all identified floors of the floorplan image.
The motivation for combining Ahmed, Stockton, Wessel, and Sato with Khosravan would have been to improve the user experience, enhance functionality, and allowed the future work described by Wessel to be implemented in the system.
Regarding claim 17, claim 17 recites substantially similar subject as to that which is recited in claim 7 and is accordingly rejected using substantially similar rationale as to that which was set forth with respect to claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Pizarro et al. teaches “Automatic Floor Plan Analysis and Recognition. Pizarro further teaches implementing OCR to recognize room size and place further scaled to the scene, see for instance, page 11, paragraph 2. Multi-modal information of a floor plan, such as room structure, type, symbols, text, and scale to recognize and reconstruct its elements, see for instance, page 13, paragraph 2. Yolov4 is employed to detect the ROIs alongside the text, number, and symbols containing semantic and contextual information like room types, dimensions, or areas, see for instance, page 13, paragraph 2. Floor plans might also include outer and inner walls, windows, furniture, dimension lines, grids, text, or icons, alongside the constraints and relationships between them, see for instance, page 1, paragraph 1.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J COBB whose telephone number is (571)270-3875. The examiner can normally be reached Monday - Friday, 11am - 7pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J COBB/Primary Examiner, Art Unit 2615
