DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to the communication dated 12/24/2025
Claims 1-12, 14-18 are presented for examination
Information Disclosure Statement
The IDS dated 11/04/2021 and the IDS dated 8/29/2023 have been reviewed. See attached.
Drawings
The drawings dated 03/26/2021 have been reviewed. They are accepted.
Specification
The abstract dated 03/26/2021 has been reviewed. It has 72 words and 6 lines and no legal phraseology. It is accepted.
Finality
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Response to Arguments - 112
Applicant’s arguments, see Page 10, filed 12/24/2025, with respect to the rejection of claims 10 and 14 under 112 have been fully considered and are persuasive. The rejection of claims 10 and 14 under 112 has been withdrawn.
Response to Arguments - 101
Applicant’s arguments, see Pages 10-16, filed 12/24/2025, with respect to the rejection of claims 1-12 and 13-18 under 101 have been fully considered and are persuasive. The rejection of claims 1-12 and 13-18 under 101 has been withdrawn.
The newly amended claim limitations, particularly the use of a machine learning model capable of accurately recognizing colorless, texture-less features in architectural imagery, something typical ML-based image processing systems struggle with, to generate filtered drawings without warping or loss of geometric detail, a major drawback of previous drawing filtering systems, successfully integrate the claims into a practical application.
Further, the claims are significantly akin to Research Corp. Techs. v. Microsoft Corp., 627 F.3d 859, 868-69, 97 USPQ2d 1274, 1380 (Fed. Cir. 2010); as described in MPEP 2106.04(a)(2)(III)(A) and 2106.05(a)(I). Research Corp. Techs. describes “a claim to a method for rendering a halftone image of a digital image by comparing, pixel by pixel, the digital image against a blue noise mask, where the method required the manipulation of computer data structures (e.g., the pixels of a digital image and a two-dimensional array known as a mask) and the output of a modified computer data structure (a halftoned digital image), Research Corp. Techs., 627 F.3d at 868, 97 USPQ2d at 1280" as being patent eligible under 101. This is analogous to the data structure manipulation and filtered image generation present in the current claims.
Additionally, sufficient evidence to show that the ordered combination of additional elements (transforming a vector image file into an agnostic format, generating a raster image from this agnostic representation, simplifying the raster image with a conditional generative adversarial neural network trained to discriminate between the indoor architectural features and the outdoor architectural features, converting the simplified raster image back into a vector format, performing additional processing, and finally generating an image from the further processed vector format data) is conventional was not found. See MPEP 2106.05(I)(B): "BASCOM provides another example of how courts conduct the significantly more analysis, and of the critical importance of considering the additional elements in combination. In this case, the Federal Circuit vacated a judgment of ineligibility because the district court failed to properly perform the second step of the Alice/Mayo test when analyzing a claimed system for filtering content retrieved from an Internet computer network. BASCOM Global Internet v. AT&T Mobility LLC, 827 F.3d 1341, 119 USPQ2d 1236 (Fed. Cir. 2016). The Federal Circuit agreed with the district court that the claims were directed to the abstract idea of filtering Internet content, and then walked through the district court’s analysis in part two of the Alice/Mayo test, noting that:
• The district court properly identified the additional elements in the claims, such as a "local client computer," "remote ISP server," "Internet computer network," and "controlled access network accounts" (827 F.3d at 1349, 119 USPQ2d at 1242);
• The district court properly considered the additional elements individually, for example by consulting the specification, which described each of the additional elements as "well-known generic computer components" (827 F.3d at 1349, 119 USPQ2d at 1242); and
• The district court should have considered the additional elements in combination, because the "inventive concept inquiry requires more than recognizing that each claim element, by itself, was known in the art" (827 F.3d at 1350, 119 USPQ2d at 1242).
Based on this analysis, the Federal Circuit concluded that the district court erred by failing to recognize that when combined, an inventive concept may be found in the non-conventional and non-generic arrangement of the additional elements, i.e., the installation of a filtering tool at a specific location, remote from the end-users, with customizable filtering features specific to each end user. 827 F.3d at 1350, 119 USPQ2d at 1242."
Response to Arguments - 103
Applicant's arguments filed 12/24/2025 have been fully considered but they are not persuasive.
Applicant argues that there is an inconsistency between translations of the Cheng_2020 and that the reference does not teach simplifying interior features.
Examiner responds by explaining that, firstly, the limitations argued, namely the simplification of interior features specifically, does not rely on Cheng_2020, and thus the argument about whether or not a particular translation of Cheng_2020 teaches this feature is moot.
Secondly, the applicant is required to provide evidence of such a translation issue for a request of this type to be proper, as per MPEP 2120 (II) “Examiners may rely on a machine translation of a foreign language document unless the machine translation is not of sufficient quality to be adequate evidence of the contents of the document. See In re Orbital Technologies Corporation, 603 Fed. App’x 924, 932 (Fed. Cir. 2015). A request by the applicant for the examiner to obtain a human language translation should be granted if the applicant provides evidence (e.g., a translation inconsistent with the machine translation) showing the machine translation does not accurately represent the document’s contents.” The applicant has provided no such evidence; while previous arguments allege that such a difference between different translations of the reference exists, mentioning alternative Google Patents and Espacenet translations, neither of which have been provided to the office.
Applicant argues that no prior art teaches the newly amended claim elements.
Examiner responds by explaining that the previously cited prior art references teach these features, in particular…
Cheng_2020 teaches generate a raster image, ([Page 7 Par 5] “S101: …raw building geometry data, and rasterizes a two-dimensional grid image…”) whereby a coordinate system of the agnostic representation in model space is mapped and scaled to an image space; ([Fig. 3, 7, 9])
…
vectorize the simplified geometry in a coordinate system of the image space to generate a vectorized simplified geometry; ([Page 7 Par 5] “S101: …from the two-dimensional grid image to extract a two-dimensional vector geographic boundary,” [Page 8 Par 1] “S202: Obtains contour line vector data, of a building…”)
…
map the vectorized simplified geometry ([Page 7 Par 5] “S101: …from the two-dimensional grid image to extract a two-dimensional vector geographic boundary,” [Page 8 Par 1] “S202: Obtains contour line vector data, of a building…”) into the model space ([Page 4 Par 8] “S402: All contour line segments in the contour line vector data correspondingly generate a wall surface, and corresponding wall surfaces are connected to form a complete closed wall surface structure, according to the positions; of each contour line segment in the contour line vector data.”)
apply morphological buffering ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”)
([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”)
([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”)
Ahmed makes obvious
compare the ([Page 857 Col 2 Par 2] “The rooms are finally labeled by using the text labels which have been extracted during text/graphics segmentation. Therefore we perform OCR2 on the texts which are inside the detected rooms. If there is no text found in the room then it is marked as unknown room. In the case of two text labels, only this one is chosen which is closer to the center of the room.”)
The process of creating the final image in Ahmed involves comparing the processed segmented geometry to the original floorplan imagery to determine labels for the rooms, then using these labels to produce a labeled room detection output image. As this output would visually appear as a labeled version of figure 6b, it can be seen from a comparison with the original imagery found in figure 2a that the geometric relationship between the rooms is preserved.
PNG
media_image1.png
413
323
media_image1.png
Greyscale
PNG
media_image2.png
452
534
media_image2.png
Greyscale
Arandiga_1998 makes obvious perform operations to generate an approximate reduced representation; apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; the approximate reduced representation([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )
Applicant argues that because Ahmed allegedly removes both external and interior walls, it does not teach wherein the simplified geometry includes the indoor architectural features and excludes the outdoor architectural features;
Examiner responds by explaining that, regardless of whether or not both indoor and outdoor walls are removed in Ahmed, the claims do not require any specific interior elements, let alone walls specifically. The term “indoor architectural features” is very broad and does not exclude the retained interior floors for the rooms as depicted in Ahmed. Further, the applicant’s arguments that such indoor and outdoor features are limited to exclusively “building and store units” and “parking spaces, parking lots, roads, sidewalks and vacant land,” respectively, is not required by the claims. Even if the description of such features in the specification were to be improperly read into the claims, going against standard examining procedure and office guidance, the specification uses language that makes it clear that the listed feature examples are meant to be read as a non-exhaustive list of examples and not a limited set of the only possible embodiments ([Par 36] “The facility may include indoor components (e.g. a building with multiple units) and outdoor components (e.g. parking lots, roadways, sidewalks, etc.).”) With this in mind, the removal of the external walls and retainment of the interior floors during in Ahmed reads on a geometry that “includes the indoor architectural features and excludes the outdoor architectural features;”
Further, as to the applicant’s arguments that no external features at all are described in Ahmed, it can be clearly seen and would have been immediately obvious to one of ordinary skill in the art that the outside walls of the depicted building are examples of “outdoor architectural features” (see [Figure 2] of Ahmed) Further, Ahmed explicitly mention the removal of “external walls” ([Page 866 Col 1 Par 1] “External walls are removed by successively applying erosion and dilation with a 3×3 square mask.”)
PNG
media_image3.png
746
577
media_image3.png
Greyscale
Applicant argues that because Ahmed teaches the removal of external walls initially for the purposes of improving text detection, its use to teach simplifying the image to include the indoor architectural features and exclude the outdoor architectural features is improper.
Examiner responds by explaining that, firstly, a prior art reference need not teach a certain element for the exact same purpose as the claims to be valid prior art. Regardless of whether or not it was for the purpose of improving text detection. Ahmed nevertheless teaches the removal of external walls from the original floorplan. Secondly, the text extraction image of Figure 2b is clearly not the result of the described external wall removal; Ahmed explains the result of this step as [Page 866 Col 1 Par 1] “External walls are removed by successively applying erosion and dilation with a 3×3 square mask. Note, that this process not only removes the external wall components but also the main title text of floor plan, which is not needed during this step. After the removal of external walls from the floor plan image, the remaining image contains only the text, medium lines, and thin lines.” As can be seen from Figure 2b, neither the medium or thin lines are present. Further, it can be seen in Figure 6b, which is relied on as an example of the simplified geometry, that the simplified geometry includes only the interior floors and does not include the outside walls.
PNG
media_image4.png
513
301
media_image4.png
Greyscale
PNG
media_image5.png
462
546
media_image5.png
Greyscale
Applicant argues that because figures 2 and 3 of the application are not the same as the figures of Ahmed, Ahmed fails to teach the simplified geometry includes the indoor architectural features and excludes the outdoor architectural features,
Examiner responds by explaining that the figures do not need to be the same for the reference to read on the claims. Further, a comparison of the figures of Ahmed, particularly a comparison of Figure 6b to the original geometry shown in Figure 2a, shows a similar transformation that results in geometry that “includes the indoor architectural features and excludes the outdoor architectural features”
PNG
media_image1.png
413
323
media_image1.png
Greyscale
PNG
media_image2.png
452
534
media_image2.png
Greyscale
Further, in response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Further, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., that text found in the original geometry cannot be considered as part of the comparison to identify a subset of indoor features) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
(1) Claims 1-3, 5, and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998
Claim 1. Cheng_2020 teaches a system for geometry simplification ([Page 7 Par 4] “… simplifying the … model of the building…”) and filtering ([Fig. 4, Page 4 Par 6] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details” [Examiner’s note: the word filter is interpreted by its plain meaning, to remove impurities or details resulting in a reduced final product, i.e. a product containing fewer details]) in architectural drawings ([Fig. 3, 6, and 9] clearly show architectural drawings)([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building” [Examiner’s note: this geographic boundary representation is interpreted as a form of vector image]) contained in a computer-readable design file ([Page 8 Par 3] “… S301: Importing the original building geometry data, and acquiring the outline surface, contour of the building from the building top bird's eye view, which is the outer contour; of the building model”) into an agnostic representation (S101: “...extract a two-dimensional vector geographic boundary …” [Examiner’s note: in the context of the disclosure of Cheng_2020 this extraction of a two-dimensional vector is interpreted as being equivalent to extracting a representation of an image in vector format that is not filetype or system dependent, in other words an agnostic representation of vector imagery, the vector imagery representing the geometry]), the vector imagery ([Page 7 Par 6] “…two-dimensional vector geographic boundary…” [Examiner’s note: this geographic boundary representation is interpreted as a form of vector image]) generate a raster image ([Page 7 Par 5] “S101: …raw building geometry data, and rasterizes a two-dimensional grid image…”) whereby a coordinate system of the agnostic representation in model space is mapped and scaled to an image space ([Fig. 3, 7, 9]), generate a simplified geometry from the raster image by ([Fig. 4, Page 7 Par 5] “S101: …geometrically simplifies, importing raw building geometry data, and rasterizes a two-dimensional grid image…” [Examiner’s note: as indicated in the sequence of images in Fig. 4, importing of raw geometry and producing a raster image from that geometry is interpreted as occurring before simplification.]), ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) in the raster image; ([Fig. 4, Page 7 Par 5] “S101: …geometrically simplifies, importing raw building geometry data, and rasterizes a two-dimensional grid image…” [Examiner’s note: as indicated in the sequence of images in Fig. 4, importing of raw geometry and producing a raster image from that geometry is interpreted as occurring before simplification.]), vectorize the simplified geometry in a coordinate system of the image space ([Page 7 Par 5] “S101: …from the two-dimensional grid image to extract a two-dimensional vector geographic boundary,” [Page 8 Par 1] “S202: Obtains contour line vector data, of a building…”), map the vectorized simplified geometry ([Page 7 Par 5] “S101: …from the two-dimensional grid image to extract a two-dimensional vector geographic boundary,” [Page 8 Par 1] “S202: Obtains contour line vector data, of a building…”) into the model space ([Page 4 Par 8] “S402: All contour line segments in the contour line vector data correspondingly generate a wall surface, and corresponding wall surfaces are connected to form a complete closed wall surface structure, according to the positions; of each contour line segment in the contour line vector data.”) ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”)
Cheng_2020 does not explicitly teach a system comprising a processor, a memory operably coupled to the processor, and having stored thereon processor-executable instructions that, when executed, cause the processor to perform operations, the imagery depicting indoor architectural features and outdoor architectural features and lacking at least one of color and texture; a conditional generative adversarial neural network trained to discriminate between indoor features and outdoor features wherein the simplified geometry includes the indoor architectural features and excludes the outdoor architectural features; perform operations to generate an approximate reduced representation; apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; compare the approximate reduced representation to original imagery to identify a subset of the original imagery for the indoor architectural features; and generate a filtered image with the subset of the original imagery, wherein geometric relationships between the indoor architectural features in the original imagery are preserved in the filtered image.
Gupta_2006 makes obvious ([Abstract] “We propose a method for indoor versus outdoor scene classification using a probabilistic neural network (PNN). The scene is initially segmented (unsupervised) using fuzzy C-means clustering (FCM) and features based on color, texture, and shape are extracted from each of the image segments” [Section 1 Page 1 Col 1 Par 1] “Some examples of such local features are the presence of trees, water bodies, exterior of buildings, sky in an outdoor scene and the presence of straight lines or regular flat-shaded objects or regions such as walls, windows, artificial man-made objects in an indoor scene.” [Figures 8- 13] show raster images characterized as being outdoor or indoor based in part on detected architectural features, like the exteriors of buildings or interior walls. ([Figure 5] Shows various segmented representations of the input image; each can be considered a simplified instance of the original image, with each containing only the information relevant to that segment. In other words, unimportant information is filtered out and only important features are kept)
Gupta_2006 is analogous art because it is within the field of neural network-based image processing, particularly image processing involving the differentiation between outdoors and indoor images. It would have been obvious to one of ordinary skill in the art to combine Gupta_2006 with Cheng_2020 before the effective filing date in order to improve the filtering and simplification process by enabling awareness of the types of objects and features in an image, ensuring important, necessary features are not over-simplified or entirely filtered out when simplifying/filtering. An essential part of this differentiation, particularly when mainly concerned with the interior geometry of a single building and none of its surroundings, as in Cheng_2020, would be discern outdoor features from indoor features. As suggested by Gupta_2006, ([Section 1 Page 1 Col 1 Par 1] “Classification of a scene as belonging to indoor or outdoor is a challenging problem in the field of pattern recognition. This is due to the extreme variability of the scene content and the difficulty in explicitly modeling scenes with indoor and outdoor content. Such a classification has applications in content-based image and video retrieval from archives, robot navigation, large-scale scene content generation and representation, generic scene recognition, and so forth.”) Gupta_2006 solves this indoor/outdoor classification challenge by introducing an image segmentation feature, which allows for greater flexibility and accuracy ([Section 1 Page 1 Col 1 Par 1] “In this work, we represent the image as a collection of segments that can be of arbitrary shape. From each segment color, texture, and shape features are extracted. Therefore, the problem of indoor versus outdoor scene classification is a feature set classification problem where the number of feature vectors in the feature set is not constant, as the number of segments in an image varies” [Section 1 Page 1 Col 2 Par 1] “Hence we propose a modified probabilistic neural network that can handle variability in the feature set dimension”) Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020 with Gupta_2006 would result in a more robust geometry filtering and simplification system that avoids filtering out or over simplifying important details
The combination of Cheng_2020 and Gupta_2006 does not explicitly teach a system comprising a processor, a memory operably coupled to the processor, and having stored thereon processor-executable instructions that, when executed, cause the processor to perform operations, the imagery depicting indoor architectural features and outdoor architectural features and lacking at least one of color and texture; a conditional generative adversarial neural network; wherein the simplified geometry includes the indoor architectural features and excludes the outdoor architectural features; perform operations to generate an approximate reduced representation; apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; compare the approximate reduced representation to original imagery to identify a subset of the original imagery for the indoor architectural features; and generate a filtered image with the subset of the original imagery, wherein geometric relationships between the indoor architectural features in the original imagery are preserved in the filtered image.
Ahmed makes obvious ([Page 867 Col 2 Par 3] “Our system is evaluated using a data set containing original floor plan images. This data set was introduced in [18] and contains the floor plan images from the period of more the ten years. The size of each floor plan image in the data set is 2479 × 3508. All floor plans are binarized to ensure that only structural information of the floor plans is used for the analysis (and not the color information).” [Page 865 Col 2 Par 2 – Page 866 Col 1 Par 1] “The information segmentation process starts with wall detection followed by text/graphics segmentation. The initial wall detection is needed because external walls are sometimes marked as a text, creating errors during the structural analysis” [Figure 2] Shows an original floor plan image including outer walls and exterior features)
PNG
media_image6.png
783
614
media_image6.png
Greyscale
([Page 865 Col 2 Par 2 – Page 866 Col 1 Par 1] “The information segmentation process starts with wall detection followed by text/graphics segmentation… External walls are removed by successively applying erosion and dilation with a 3×3 square mask. Note, that this process not only removes the external wall components but also the main title text of floor plan, which is not needed during this step. After the removal of external walls from the floor plan image, the remaining image contains only the text, medium lines, and thin lines.” [Figure 6] Shows side by side images of a floorplan before final processing and after final processing. Note the lack of external details that were present in (a) in the final result (b))
PNG
media_image7.png
315
348
media_image7.png
Greyscale
([Page 857 Col 2 Par 2] “The rooms are finally labeled by using the text labels which have been extracted during text/graphics segmentation. Therefore we perform OCR2 on the texts which are inside the detected rooms. If there is no text found in the room then it is marked as unknown room. In the case of two text labels, only this one is chosen which is closer to the center of the room.”)
Ahmed is analogous art because it is within the field of architectural plan processing. It would have been obvious to one of ordinary skill in the art to combine it with Cheng_2020 and Gupta_2006 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better analyze the contents of depictions of architectural features. As noted by Ahmed, architectural drawing frequently include text descriptions and labels that inform the exact relationships between features. These text descriptions, however, can be difficult to accurately extract from the drawings. ([Page 864 Col 2 Par 2-3] “The pattern recognition community has already put a lot of effort into the segmentation of text and graphics. Several different methods have been proposed to work for different purposes [4]. For the purpose of technical drawings, [5] proposed a method to extract text strings from mixed text/graphics images … However, a drawback of these methods is that many text components touching graphics are marked as a graphical component rather than as text. In most technical drawings, images, text, and graphics overlay, which especially holds for map images.”) To this end. Ahmed presents a method to efficiently extract and separate text and graphics from floorplan imagery ([Page 864 Col 2 Par 4] “In this work, we decided to adopt the approach of [6], because no color information is available. In addition, we introduce some improvements to take advantage from specific properties of architectural floor plans.” [Page 864 Col 1 Par 3] “The approach described in this paper improves previously introduced approaches at several parts. It starts with a fine segmentation of different types of information available in the floor plans, so that only the required information is used by each step. Besides several improvements of the extraction and segmentation methods applied, we introduce the removal of components outside the outer walls, which significantly improves the performance.”) Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020 and Gupta_2006 with Ahmed would result in a system that is capable of processing architectural imagery more accurately and efficiently.
The combination of Cheng_2020, Gupta_2006, and Ahmed does not explicitly teach a system comprising a processor, a memory operably coupled to the processor, and having stored thereon processor-executable instructions that, when executed, cause the processor to perform operations, a conditional generative adversarial neural network; generate an approximate reduced representation of data and apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; the approximate reduced representation;
Bailey_2003 makes obvious a system comprising a processor, a memory operably coupled to the processor, and having stored thereon processor-executable instructions that, when executed, cause the processor to perform operations, ([Page 4 Par 9] “In another example implementation, a computing system may include one or more processors and one or more memories, wherein the computing system is configured to perform operations) a conditional generative adversarial neural network; ([Page 7 Par 17] “The CGAN generator is further conditioned on observations…” [Par 92] “In some implementations, the one or more training images may become the input for training, validating, and testing deep learning neural networks (actions 408, 410, 412 in FIG. 4). In the example of FIG. 4, one or more of a “Vanilla” Generative Adversarial Network (GAN), a Conditional GAN (CGAN), and/or a latent space representation may be trained.” [Par 101] “For example, a conditional GAN (e.g., CGAN 600) may allow parametric control over synthetically generated data so that the result satisfies the supplied parametric values.” [Page 18 Par 93] " A Generative Adversarial Network (GAN) may generally include a class of neural networks used in unsupervised machine learning. Example applications may include, but are not limited to, ... drug prediction to treat a certain disease, retrieving images containing a given pattern, etc.")
Bailey_2003 is analogous art because it is within the field of image processing integrated with machine learning. It would have been obvious before the filing date to combine Bailey_2003 with Cheng_2020, Gupta_2006, and Ahmed in order to improve system performance by introducing a neural network capable of handling geometric data much more efficiently with a minimal loss of accuracy, as well as efficiently generating training data for those models ([Page 2 Par 4-5] “In contrast, another type of modeling approach, called object- based modeling (OBM), has the advantage of generating more realistic geobodies by directly building the geological objects … with specified distribution of geometric parameters of channels such as width, length, ... However, the conditioning to well observation, particularly for dense well locations, as well as other types of data measurements and constraints such as seismic data, becomes very difficult since object-modeling process may not converge and becomes extremely slow due to the use of MCMC (Markov Chain Monto Carlo). In some implementations and as will be discussed in greater detail below, object- based modeling can be embedded into neural networks as a tool to generate various geological templates (e.g., training data or training images) to drive the modeling process to build geologically realistic models and honor various subsurface constraints by generating many realizations than contain patterns that ensemble the templates in very efficient manner.”) Although the techniques discussed in Bailey_2003 are applied to the field of geological analysis, it would be appreciated by one of ordinary skill in the art that the geometry processing and training data generation techniques described in the disclosure would have wide-ranging application to many other fields under the umbrella of image processing, such as architectural data processing, as in Cheng_2020. Overall, one of ordinary skill in the art would recognize that combining Cheng_2020, Gupta_2006, and Ahmed with Bailey_2003 would result in a more efficient and accurate neural network system that was easier to train.
The combination of Cheng_2020, Gupta_2006, Ahmed, and Bailey_2003 fails to make obvious generate an approximate reduced representation of data and apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; the approximate reduced representation
Arandiga_1998 makes obvious generate an approximate reduced representation of data and apply reduction techniques to the approximate reduced representation to generate an approximate reduced representation; the approximate reduced representation([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )
Arandiga_1998 is analogous art because it is within the field of data compression/reduction with particular application to images. It would be obvious to one of ordinary skill in the art to combine Arandiga_1998 with the previous combination of Cheng_2020, Gupta_2006, Ahmed and Bailey_2003 in order to better simplify processing and make design processing faster by reducing the amount of data that needs to be processed. ([Page 160 Par 2] “Multiscale techniques do have an important role in numerical analysis. A wavelet type decomposition of a function is used to reduce the cost of many numerical algorithms either by applying it to the numerical solution operator to obtain an approximate sparse form [4, 14, 21, 2] or by applying it to the numerical solution itself to obtain an approximate reduced representation in order to solve for fewer quantities”) Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020, Gupta_2006, Ahmed and Bailey_2003 with Arandiga_1998 would result in a more efficient system overall by reducing the amount of data that needs to be processed.
Claim 5. The elements of claim 5 are substantially the same as those of claim 1. Therefore, the elements of claim 5 are rejected due to the same reasons as outlined above for claim 1.
Claim 2. Cheng_2020 teaches that the agnostic representation is a generic geometry collection of the vector imagery in the model space. ([Page 7 Par 5] “S101: …from the two-dimensional grid image to extract a two-dimensional vector geographic boundary” [Examiner’s note: as discussed earlier this extracted vector geographic boundary is interpreted as being a filetype and system agnostic representation of the geometry, in other words a generic geometry collection]).
Claim 8. The elements of claim 8 are substantially the same as those of claim 2. Therefore, the elements of claim 8 are rejected due to the same reasons as outlined above for claim 2.
Claim 3. Cheng_2020 teaches that the vector imagery is a visual representation of any one of: a facility and an architectural structure ([Page 7 Par 5] “S101: extract a two-dimensional vector geographic boundary; of a building.”)
Claim 7. The elements of claim 7 are substantially the same as those of claim 3. Therefore, the elements of claim 7 are rejected due to the same reasons as outlined above for claim 3.
(2) Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1)
Claim 16. Cheng_2020 teaches A computer-implemented method for geometry simplification ([Page 7 Par 4] “… simplifying the … model of the building…”) and filtering ([Fig. 4, Page 4 Par 6] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details” [Examiner’s note: the word filter is interpreted by its plain meaning, to remove impurities or details resulting in a reduced final product, i.e. a product containing fewer details]) comprising: generating a raster image from a vector image ([Page 7 Par 5] “S101: …raw building geometry data, and rasterizes a two-dimensional grid image…” [Examiner’s note: in the context of the present invention the ‘raw building geometry data’ would be a vector image]), contained in a computer- readable design file, ([Page 8 Par 3] “… S301: Importing the original building geometry data, and acquiring the outline surface, contour of the building from the building top bird's eye view, which is the outer contour; of the building model”) the vector image ([Page 7 Par 5 “S102: …two-dimensional vector geographic boundary…” [Examiner’s note: this geographic boundary representation is interpreted as a form of vector image]) ([Fig. 4, Page 7 Par 5] “S101: …geometrically simplifies, importing raw building geometry data, and rasterizes a two-dimensional grid image…” [Examiner’s note: as indicated in the sequence of images in Fig. 4, importing of raw geometry and producing a raster image from that geometry is interpreted as occurring before simplification.]), ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) in the raster image; ([Page 7 Par 5] “S101: …raw building geometry data, and rasterizes a two-dimensional grid image…” [Examiner’s note: in the context of the present invention the ‘raw building geometry data’ would be a vector image]), ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”)
Cheng_2020 does not explicitly teach imagery depicting indoor architectural features and outdoor architectural features and lacking at least one of color and texture; processing data by a conditional generative adversarial neural network trained to discriminate between indoor features and outdoor features wherein the simplified image includes the indoor architectural features and excludes the outdoor architectural features; comparing the simplified image to original imagery to identify a subset of the original imagery for the indoor architectural features; and generating a filtered image with the subset of the original imagery wherein geometric relationships between the indoor architectural features in the original imagery are preserved in the filtered image.
Gupta_2006 makes obvious ([Abstract] “We propose a method for indoor versus outdoor scene classification using a probabilistic neural network (PNN). The scene is initially segmented (unsupervised) using fuzzy C-means clustering (FCM) and features based on color, texture, and shape are extracted from each of the image segments” [Section 1 Page 1 Col 1 Par 1] “Some examples of such local features are the presence of trees, water bodies, exterior of buildings, sky in an outdoor scene and the presence of straight lines or regular flat-shaded objects or regions such as walls, windows, artificial man-made objects in an indoor scene.” [Figures 8- 13] show raster images characterized as being outdoor or indoor based in part on detected architectural features, like the exteriors of buildings or interior walls. ([Figure 5] Shows various segmented representations of the input image; each can be considered a simplified instance of the original image, with each containing only the information relevant to that segment. In other words, unimportant information is filtered out and only important features are kept)
Gupta_2006 is analogous art because it is within the field of neural network-based image processing, particularly image processing involving the differentiation between outdoors and indoor images. It would have been obvious to one of ordinary skill in the art to combine Gupta_2006 with Cheng_2020 before the effective filing date in order to improve the filtering and simplification process by enabling awareness of the types of objects and features in an image, ensuring important, necessary features are not over-simplified or entirely filtered out when simplifying/filtering. An essential part of this differentiation, particularly when mainly concerned with the interior geometry of a single building and none of its surroundings, as in Cheng_2020, would be discern outdoor features from indoor features. As suggested by Gupta_2006, ([Section 1 Page 1 Col 1 Par 1] “Classification of a scene as belonging to indoor or outdoor is a challenging problem in the field of pattern recognition. This is due to the extreme variability of the scene content and the difficulty in explicitly modeling scenes with indoor and outdoor content. Such a classification has applications in content-based image and video retrieval from archives, robot navigation, large-scale scene content generation and representation, generic scene recognition, and so forth.”) Gupta_2006 solves this indoor/outdoor classification challenge by introducing an image segmentation feature, which allows for greater flexibility and accuracy ([Section 1 Page 1 Col 1 Par 1] “In this work, we represent the image as a collection of segments that can be of arbitrary shape. From each segment color, texture, and shape features are extracted. Therefore, the problem of indoor versus outdoor scene classification is a feature set classification problem where the number of feature vectors in the feature set is not constant, as the number of segments in an image varies” [Section 1 Page 1 Col 2 Par 1] “Hence we propose a modified probabilistic neural network that can handle variability in the feature set dimension”) Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020 with Gupta_2006 would result in a more robust geometry filtering and simplification system that avoids filtering out or over simplifying important details
The combination of Cheng_2020 and Gupta_2006 does not explicitly teach imagery depicting indoor architectural features and outdoor architectural features and lacking at least one of color and texture; a conditional generative adversarial neural network; wherein the simplified image includes the indoor architectural features and excludes the outdoor architectural features; comparing the simplified image to original imagery to identify a subset of the original imagery for the indoor architectural features; and generating a filtered image with the subset of the original imagery wherein geometric relationships between the indoor architectural features in the original imagery are preserved in the filtered image.
Ahmed makes obvious imagery depicting indoor architectural features and outdoor architectural features and lacking at least one of color and texture; ([Page 867 Col 2 Par 3] “Our system is evaluated using a data set containing original floor plan images. This data set was introduced in [18] and contains the floor plan images from the period of more the ten years. The size of each floor plan image in the data set is 2479 × 3508. All floor plans are binarized to ensure that only structural information of the floor plans is used for the analysis (and not the color information).” [Page 865 Col 2 Par 2 – Page 866 Col 1 Par 1] “The information segmentation process starts with wall detection followed by text/graphics segmentation. The initial wall detection is needed because external walls are sometimes marked as a text, creating errors during the structural analysis” [Figure 2] Shows an original floor plan image including outer walls and exterior features)
PNG
media_image6.png
783
614
media_image6.png
Greyscale
([Page 865 Col 2 Par 2 – Page 866 Col 1 Par 1] “The information segmentation process starts with wall detection followed by text/graphics segmentation… External walls are removed by successively applying erosion and dilation with a 3×3 square mask. Note, that this process not only removes the external wall components but also the main title text of floor plan, which is not needed during this step. After the removal of external walls from the floor plan image, the remaining image contains only the text, medium lines, and thin lines.” [Figure 6] Shows side by side images of a floorplan before final processing and after final processing. Note the lack of external details that were present in (a) in the final result (b))
PNG
media_image7.png
315
348
media_image7.png
Greyscale
comparing the simplified image to original imagery to identify a subset of the original imagery for the indoor architectural features; and generating a filtered image with the subset of the original imagery wherein geometric relationships between the indoor architectural features in the original imagery are preserved in the filtered image. ([Page 857 Col 2 Par 2] “The rooms are finally labeled by using the text labels which have been extracted during text/graphics segmentation. Therefore we perform OCR2 on the texts which are inside the detected rooms. If there is no text found in the room then it is marked as unknown room. In the case of two text labels, only this one is chosen which is closer to the center of the room.”)
Ahmed is analogous art because it is within the field of architectural plan processing. It would have been obvious to one of ordinary skill in the art to combine it with Cheng_2020 and Gupta_2006 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better analyze the contents of depictions of architectural features. As noted by Ahmed, architectural drawing frequently include text descriptions and labels that inform the exact relationships between features. These text descriptions, however, can be difficult to accurately extract from the drawings. ([Page 864 Col 2 Par 2-3] “The pattern recognition community has already put a lot of effort into the segmentation of text and graphics. Several different methods have been proposed to work for different purposes [4]. For the purpose of technical drawings, [5] proposed a method to extract text strings from mixed text/graphics images … However, a drawback of these methods is that many text components touching graphics are marked as a graphical component rather than as text. In most technical drawings, images, text, and graphics overlay, which especially holds for map images.”) To this end. Ahmed presents a method to efficiently extract and separate text and graphics from floorplan imagery ([Page 864 Col 2 Par 4] “In this work, we decided to adopt the approach of [6], because no color information is available. In addition, we introduce some improvements to take advantage from specific properties of architectural floor plans.” [Page 864 Col 1 Par 3] “The approach described in this paper improves previously introduced approaches at several parts. It starts with a fine segmentation of different types of information available in the floor plans, so that only the required information is used by each step. Besides several improvements of the extraction and segmentation methods applied, we introduce the removal of components outside the outer walls, which significantly improves the performance.”) Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020 and Gupta_2006 with Ahmed would result in a system that is capable of processing architectural imagery more accurately and efficiently.
The combination of Cheng_2020, Gupta_2006, and Ahmed does not explicitly teach a conditional generative adversarial neural network;
Bailey_2003 makes obvious a conditional generative adversarial neural network ([Page 7 Par 17] “The CGAN generator is further conditioned on observations…” [Par 92] “In some implementations, the one or more training images may become the input for training, validating, and testing deep learning neural networks (actions 408, 410, 412 in FIG. 4). In the example of FIG. 4, one or more of a “Vanilla” Generative Adversarial Network (GAN), a Conditional GAN (CGAN), and/or a latent space representation may be trained.” [Par 101] “For example, a conditional GAN (e.g., CGAN 600) may allow parametric control over synthetically generated data so that the result satisfies the supplied parametric values.” [Page 18 Par 93] " A Generative Adversarial Network (GAN) may generally include a class of neural networks used in unsupervised machine learning. Example applications may include, but are not limited to, ... drug prediction to treat a certain disease, retrieving images containing a given pattern, etc.")
Bailey_2003 is analogous art because it is within the field of image processing integrated with machine learning. It would have been obvious before the filing date to combine Bailey_2003 with Cheng_2020, Gupta_2006, and Ahmed in order to improve system performance by introducing a neural network capable of handling geometric data much more efficiently with a minimal loss of accuracy, as well as efficiently generating training data for those models ([Page 2 Par 4-5] “In contrast, another type of modeling approach, called object- based modeling (OBM), has the advantage of generating more realistic geobodies by directly building the geological objects … with specified distribution of geometric parameters of channels such as width, length, ... However, the conditioning to well observation, particularly for dense well locations, as well as other types of data measurements and constraints such as seismic data, becomes very difficult since object-modeling process may not converge and becomes extremely slow due to the use of MCMC (Markov Chain Monto Carlo). In some implementations and as will be discussed in greater detail below, object- based modeling can be embedded into neural networks as a tool to generate various geological templates (e.g., training data or training images) to drive the modeling process to build geologically realistic models and honor various subsurface constraints by generating many realizations than contain patterns that ensemble the templates in very efficient manner.”) Although the techniques discussed in Bailey_2003 are applied to the field of geological analysis, it would be appreciated by one of ordinary skill in the art that the geometry processing and training data generation techniques described in the disclosure would have wide-ranging application to many other fields under the umbrella of image processing, such as architectural data processing, as in Cheng_2020. Overall, one of ordinary skill in the art would recognize that combining Cheng_2020, Gupta_2006, and Ahmed with Bailey_2003 would result in a more efficient and accurate neural network system that was easier to train.
Claim 17. Cheng_2020 teaches that the vector imagery is a visual representation of any one of: a facility and an architectural structure ([Page 7 Par 5] “S101: extract a two-dimensional vector geographic boundary; of a building.”)
(3) Claims 4, 6, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998, and Berkner_2003 (EP 1329847 A1)
Claim 4. Cheng_2020 teaches wherein the ([Fig.4] the first image from the left in Fig. 4 is interpreted as the original vector imagery. Despite the flattening of the third dimension, it is clearly shown to be within the same coordinate system as the simplified geometry, best represented by the 4th image from the left in Fig. 4, [Page 7 Par 6] “S102:…two-dimensional vector geographic boundary…” [Examiner’s note: this geographic boundary representation is interpreted as an image vector])
Arandiga_1998 makes obvious wherein the approximate reduced representation (Introduction Par 3 “...obtain an approximate reduced representation…”)
The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 does not explicitly teach wherein data is the inverse transform of other data.
Berkner_2003 makes obvious wherein data is the inverse transform of other data ([Page 22 Par 1] “…performing an inverse transform…”)
Berkner_2003 is analogous art because it is within the field of image processing. It would have been obvious before the filing date to combine the previous combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 with Berkner_2003 to produce a method that better reduces and simplifies the original image by using improved compression techniques. ([Page 25 Par 7] “The value of header-based processing is demonstrated in the example of creating a good 128x128 thumbnail representation of 1024x1024 image. An image analysis process described herein is the one for automatic cropping and scaling as described above. The complexity of processed data compared to traditional image processing of a JPEG 2000 image and a raster image is listed in Table 3. The advantage over an image in JPEG 2000 form is that only 1/1000 of the data must be used by the segmentation algorithm and less than 1/2 of data must be decoded.” [Page 25 Table])
PNG
media_image8.png
200
696
media_image8.png
Greyscale
Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 with Berkner_2003 would result in an image processing system with a much more robust compression mechanism, reducing the amount of data that needs to be processed and therefore making the system more efficient.
Claim 12. The elements of claim 12 are substantially the same as those of claim 4. Therefore, the elements of claim 12 are rejected due to the same reasons as outlined above for claim 4.
Claim 6. Cheng_2020 teaches ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”)
Arandiga_1998 makes obvious ([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )
The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 fails to make obvious further comprising storing data in a database.
Berkner_2003 makes obvious further comprising storing data in a database. ([Page 11 Par 2] “The techniques described herein have applications in areas such as, but not limited to, display-adaptive image representations, digital video surveillance, image database management.”)
Berkner_2003 is analogous art because it is within the field of image processing. It would have been obvious before the filing date to combine the previous combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 with Berkner_2003 to produce a method that better reduces and simplifies the original image by using improved compression techniques. ([Page 25 Par 7] “The value of header-based processing is demonstrated in the example of creating a good 128x128 thumbnail representation of 1024x1024 image. An image analysis process described herein is the one for automatic cropping and scaling as described above. The complexity of processed data compared to traditional image processing of a JPEG 2000 image and a raster image is listed in Table 3. The advantage over an image in JPEG 2000 form is that only 1/1000 of the data must be used by the segmentation algorithm and less than 1/2 of data must be decoded.” [Page 25 Table])
PNG
media_image8.png
200
696
media_image8.png
Greyscale
Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998with Berkner_2003 would result in an image processing system with a much more robust compression mechanism, reducing the amount of data that needs to be processed and therefore making the system more efficient.
(4) Claims 11 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998 and REVIT
Claim 11. Cheng_2020 teaches ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) and the ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”)
Gupta_2006 makes obvious wherein the ([Abstract] “We propose a method for indoor versus outdoor scene classification using a probabilistic neural network (PNN). The scene is initially segmented (unsupervised) using fuzzy C-means clustering (FCM) and features based on color, texture, and shape are extracted from each of the image segments” [Section 1 Page 1 Col 1 Par 1] “Some examples of such local features are the presence of trees, water bodies, exterior of buildings, sky in an outdoor scene and the presence of straight lines or regular flat-shaded objects or regions such as walls, windows, artificial man-made objects in an indoor scene.” [Figures 8- 13] show raster images characterized as being outdoor or indoor based in part on detected architectural features, like the exteriors of buildings or interior walls. ([Figure 5] Shows various segmented representations of the input image; each can be considered a simplified instance of the original image, with each containing only the information relevant to that segment. In other words, unimportant information is filtered out and only important features are kept)
Bailey_2003 makes obvious a conditional generative adversarial network ([Page 7 Par 17] “The CGAN generator is further conditioned on observations…”)
The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 does not explicitly teach data according to class labels or layers in the design file
REVIT makes obvious data according to class labels or layers in the design file ([Page 73 Par 8] “When you import or link a file to a Revit project, you can query the file for information about its objects. This allows you to determine the identity of an object and the layer on which it resides. You can also hide the object’s layer or delete it.”)
REVIT is analogous art because it is within the field of CAD for architectural design. It would have been obvious to combine REVIT with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 before the effective filing date in order to improve the modelling experience by automating certain tasks, making the architectural design process easier and faster to use without sacrificing the power of the editor ([Page 10 Par 8-9] “A fundamental characteristic of a building information modeling application is the ability to coordinate changes and maintain consistency at all times. You do not have to intervene to update drawings or links. When you change something, Revit Architecture immediately determines what is affected by the change and reflects that change to any affected elements. Revit Architecture uses 2 key concepts that make it especially powerful and easy to use. The first is the capturing of relationships while the designer works. The second is its approach to propagating building changes. The result of these concepts is software that works like you do, without requiring entry of data that is unimportant to your design.”) Overall, one of ordinary skill in the art would have recognized that combining REVIT with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 would result in a system that is faster and easier to use.
Claim 15. Cheng_2020 teaches ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) the original geometry in the vector imagery ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Fig. 4, Page 4 Par 6] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details” [Examiner’s note: the word filter is interpreted by its plain meaning, to remove impurities or details resulting in a reduced final product, i.e. a product containing fewer details]) and the buffered ([Fig. 4, Page 4 Par 6] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details” [Examiner’s note: the word filter is interpreted by its plain meaning, to remove impurities or details resulting in a reduced final product, i.e. a product containing fewer details])
Ahmed makes obvious wherein comparing to the original imagery ([Page 857 Col 2 Par 2] “The rooms are finally labeled by using the text labels which have been extracted during text/graphics segmentation. Therefore we perform OCR2 on the texts which are inside the detected rooms. If there is no text found in the room then it is marked as unknown room. In the case of two text labels, only this one is chosen which is closer to the center of the room.”)
Arandiga_1998 makes obvious wherein processing the approximate reduced representation ([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” ) ([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )
The combination of The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 does not explicitly teach wherein processing imagery comprises limiting the subset of the imagery to the intersection of a first set of imagery and other imagery/data
REVIT makes obvious wherein processing imagery comprises limiting the subset of the imagery to the intersection of a first set of imagery and other imagery/data ([Page 1499 Par 7] “The datum now displays only in views whose cutting plane intersects the selected scope box. If a view's cutting plane lies outside the scope, the associated datum does not display in the view”)
REVIT is analogous art because it is within the field of CAD for architectural design. It would have been obvious to combine REVIT with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 before the effective filing date in order to improve the modelling experience by automating certain tasks, making the architectural design process easier and faster to use without sacrificing the power of the editor ([Page 10 Par 8-9] “A fundamental characteristic of a building information modeling application is the ability to coordinate changes and maintain consistency at all times. You do not have to intervene to update drawings or links. When you change something, Revit Architecture immediately determines what is affected by the change and reflects that change to any affected elements. Revit Architecture uses 2 key concepts that make it especially powerful and easy to use. The first is the capturing of relationships while the designer works. The second is its approach to propagating building changes. The result of these concepts is software that works like you do, without requiring entry of data that is unimportant to your design.”) Overall, one of ordinary skill in the art would have recognized that combining REVIT with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 would result in a system that is faster and easier to use.
(5) Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to REVIT
Claim 18. Cheng_2020 teaches ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”) and the ([Page 7 Par 6] “S102: … three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building.”)
Gupta_2006 makes obvious wherein the ([Abstract] “We propose a method for indoor versus outdoor scene classification using a probabilistic neural network (PNN). The scene is initially segmented (unsupervised) using fuzzy C-means clustering (FCM) and features based on color, texture, and shape are extracted from each of the image segments” [Section 1 Page 1 Col 1 Par 1] “Some examples of such local features are the presence of trees, water bodies, exterior of buildings, sky in an outdoor scene and the presence of straight lines or regular flat-shaded objects or regions such as walls, windows, artificial man-made objects in an indoor scene.” [Figures 8- 13] show raster images characterized as being outdoor or indoor based in part on detected architectural features, like the exteriors of buildings or interior walls. ([Figure 5] Shows various segmented representations of the input image; each can be considered a simplified instance of the original image, with each containing only the information relevant to that segment. In other words, unimportant information is filtered out and only important features are kept)
Bailey_2003 makes obvious the conditional generative adversarial network ([Page 7 Par 17] “The CGAN generator is further conditioned on observations…”)
The combination of Cheng_2020, Gupta_2006, Ahmed, and Bailey_2003 fails to make obvious data according to class labels or layers in the design file
REVIT makes obvious data according to class labels or layers in the design file ([Page 73 Par 8] “When you import or link a file to a Revit project, you can query the file for information about its objects. This allows you to determine the identity of an object and the layer on which it resides. You can also hide the object’s layer or delete it.”)
REVIT is analogous art because it is within the field of CAD for architectural design. It would have been obvious to combine REVIT with Cheng_2020, Gupta_2006, Ahmed, and Bailey_2003 before the effective filing date in order to improve the modelling experience by automating certain tasks, making the architectural design process easier and faster to use without sacrificing the power of the editor ([Page 10 Par 8-9] “A fundamental characteristic of a building information modeling application is the ability to coordinate changes and maintain consistency at all times. You do not have to intervene to update drawings or links. When you change something, Revit Architecture immediately determines what is affected by the change and reflects that change to any affected elements. Revit Architecture uses 2 key concepts that make it especially powerful and easy to use. The first is the capturing of relationships while the designer works. The second is its approach to propagating building changes. The result of these concepts is software that works like you do, without requiring entry of data that is unimportant to your design.”) Overall, one of ordinary skill in the art would have recognized that combining REVIT with Cheng_2020, Gupta_2006, Ahmed, and Bailey_2003would result in a system that faster and easier to use.
(6) Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998 and Harper_2005 (US 20050231502 A1).
Claim 9. Harper_2005 makes obvious that the raster image has dimensions of 2048 x 2048 pixels. (Par 155: “…in contemporary GPUs, images beyond 2048 X 2048 are generally too big.”)
Harper_2005 is analogous art because it is within the field of image processing. It would have been obvious before the filing date to combine Harper_2005 with the previous combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 in order to further speed up image processing, regardless of physical hardware limitations (i.e. single core processors) ([Par 8] “In one general embodiment of the invention, software will exploit a selected processor in the system to compose a graph-like description of an image task…. Furthermore, having a graph-like description of the overall image task allows use of an optimizing compiler to reduce the necessary resources for the overall image task. This compiling function is especially useful since the node programs will generally run on a processor other than that which runs the compiler.” [Par 9] “. Alternatively, the graph may be optimized by a compiler in distinct pieces, as the graph is created. The purpose of optimizing is to minimize memory usage and CPU or GPU time or otherwise gain efficiency when the image is computed.” [Par 11] “Applying these techniques in the contemporary graphics context is highly efficient and allows developers to write filters by expressing the operations to be performed on an element (e.g. pixel) or elements without concern for the specific hardware in a system--that will be accounted by the compiler. In addition, having created an API and efficient processing infrastructure for deployment in a multi-processor system, many embodiments also include functionality to exploit that the API on single processor systems.”) Overall, one of ordinary skill in the art would recognize that combining Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 with Harper_2005 would make the image processing system significantly more efficient, particularly in the case of weaker hardware.
(7) Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998, Harper_2005 (US 20050231502 A1) and Sankar_2014 (US 20140320661 A1).
Claim 10. Cheng_2020 teaches wherein generating the raster image ([Page 7 Par 5] “S101: …raw building geometry data, and rasterizes a two-dimensional grid image…”)
The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, Arandiga_1998, and Harper_2005 does not explicitly teach processing data comprises a rigid transformation represented as a 3x3 invertible matrix.
Sankar_2014 makes obvious processing data comprises a rigid transformation represented as a 3x3 invertible matrix. ([Par 44] “The indoor scene capture system then calculates a 2D rigid body transformation…” [Examiner’s note: all 2D rigid body transformations are 3x3 invertible matrices])
Sankar_2014 is analogous art because it is within the field of architectural data processing. It would have been obvious to one of ordinary skill in the art to combine it with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, Arandiga_1998, and Harper_2005 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to enable easier capture of data relating to the geometry and floorplan of the structures by integrating scanning. While Sankar_2014 highlights the benefits of image-based data capture for indoor spaces, it notes that prior approaches suffer from expensive, bulky hardware and/or require tedious manual adjustment. ([Par 2-5] “The desire to enhance and improve the notion of visual presence has subsequently fueled a sizable body of work around interactive visual tours. However, most of these approaches rely on specialized data acquisition equipment and complex and time-consuming off-line processing pipelines, making them inaccessible to the casual user. … Brooks was one of the first to propose a system to build rapid visual prototypes of buildings for architectural use. (Brooks, F., "WalkThrough--A Dynamic Graphics System for Simulating Virtual Buildings," Proceedings of I3D'86, 1987, pp. 9-21.) More recently, Uyttendaele used an omnidirectional video to create indoor virtual tours. (Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S., Szeliski, R., and Hartley, R., "Image-Based Interactive Exploration of Real-World Environments," IEEE Computer Graphics and Applications, 2004, 24: pp. 52-63.) Similar approaches are used in Google's Streetview and Art Project. (Anguelov, D., Dulong, C., Filip, D., Frueh, C., Lafon, S., Lyon, R., Ogale, A., Vincent, L., and Weaver, J., "Google Street View: Capturing the World at Street Level," June 2010, Computer, 43(6): pp. 32-38; Google Inc., Google Art Project, 2011, http://www.googleartproject.com.) These approaches require sophisticated omnidirectional camera rigs and several hours of offline processing. … Kim employed a Manhattan-world assumption to acquire indoor floor plans in real-time. (Kim, Y. M., Dolson, J., Sokolsky, M., Koltun, V., Thrun, S., "Interactive Acquisition of Residential Floor Plans," Proceedings of ICRA, 2012, pp. 3055-3062.) Kim's approach is hardware-intensive, requiring the user to carry a Kinect camera, a projector, a laptop, and a special input device while capturing data around the house. The recent shift of imaged-based systems to the mobile phone platform is exemplified by the mobile Photosynth application that creates panoramic images in real time on a smartphone. (Microsoft Corporation, Photosynth, 2011, http://photosynth.net/.) MagicPlan is a commercial floor plan generation app available for the iPhone. (Sensopia Inc., MagicPlan, 2011, http://www.sensopia.com.) By marking floor corners in the room via an augmented reality interface, MagicPlan is able to estimate dimensions of the room and generate a corresponding floor plan. MagicPlan reconstructs rooms individually and then has a user manually assemble them to form a complete floor plan.”) To this end, Sankar_2014 presents a method for a fully automated, smartphone-based architectural capture system ([Abstract] “An indoor scene capture system is provided that, with a handheld device with a camera, collects videos of rooms, spatially indexes the frames of the videos, marks doorways between rooms, and collects videos of transitions from room to room via doorways. The indoor scene capture system may assign a direction to at least some of the frames based on the angle of rotation as determined by an inertial sensor (e.g., gyroscope) of the handheld device. The indoor scene capture system marks doorways within the frames of the videos. For each doorway between rooms, the indoor scene capture system collects a video of transitioning through the doorway as the camera moves from the point within a room through the doorway to a point within the adjoining room.”) Overall, one of ordinary skill in the art would have recognized that combining Sankar_2014 with Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, Arandiga_1998, and Harper_2005 would result in a system that allowed for inexpensive, easy to use architectural data capture, ultimately resulting in the easier generation of initial geometry, making the process as a whole of producing filtered architectural imagery significantly easier to use.
(8) Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng_2020 (CN 110827402 A), in view of Gupta_2006 in further view of Improved Automatic Analysis of Architectural Floor Plans (Hereinafter Ahmed) as well as Bailey_2003 (WO 2019118658 A1) in addition to Arandiga_1998 and Gloudemans_2009 (US 20090128667 A1).
Claim 14. Cheng_2020 teaches ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”) the original geometry in the vector imagery ([Page 7 Par 6] “S102: Performs three-dimensional reconstruction, to reconstruct a reconstructed three-dimensional building structure, based on the two-dimensional vector geographic boundary, and reconstructing a wall and roof; of a building”) ([Page 7 Par 6] “…two-dimensional vector geographic boundary…” [Examiner’s note: this geographic boundary representation is interpreted as a form of vector image]) ([Fig. 4, Page 8 Par 5] “S302: Uses an edge extraction method on the … grid data to directly obtain contour line vector data, containing fewer details”)
Ahmed makes obvious wherein comparing to the original imagery ([Page 857 Col 2 Par 2] “The rooms are finally labeled by using the text labels which have been extracted during text/graphics segmentation. Therefore we perform OCR2 on the texts which are inside the detected rooms. If there is no text found in the room then it is marked as unknown room. In the case of two text labels, only this one is chosen which is closer to the center of the room.”)
Arandiga_1998 makes obvious ([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )([Introduction Par 3] “...obtain an approximate reduced representation…”, [Page 174 Par 3] “…multiresolution representations can then be used to reduce the cost of a numerical algorithm or to compress the information in the discrete set for purposes of storage or transmission…” )
The combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 does not explicitly teach wherein processing comprises removing line segments from imagery that do not sufficiently intersect other imagery
Gloudemans_2009 makes obvious wherein processing comprises removing line segments from imagery that do not sufficiently intersect other imagery ([Par 4] “…automatically detecting the at least one line segment in the mask image, automatically determining that the portion of the at least one line segment is occluded by the object, and automatically removing...” [Par 6] “…at least one processing facility: a) automatically finds the at least one line segment, b) removes the at least one line segment, c) after the removing, detects blobs in the image, d) determines if the blobs meet at least one specified criterion…”)
Gloudemans_2009 is analogous art because it is within the field of image processing. It would have been obvious before the filing date to combine Gloudemans_2009 with the previous combination of Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 to better handle extraneous architectural drawing data. As noted by Gloudemans_2009, creating an image that captures only desired or important image elements can be complicated by foreground entities obscuring the desired background elements ([Par 2] “Due to safety and practical considerations, cameras cannot always be positioned in locations which capture the action from a desired viewpoint. As a result, the images obtained may not capture the most important action, for instance, due to one player occluding another.”) To solve this issue, Gloudemans_2009 introduces a comprehensive occlusion management system ([Par 4] “One embodiment involves finding lines in an image, such as lines on a field in a sporting event, and removing any occluding objects such as players on the field. For example, a method for automatically finding and repairing lines in image data of an event includes obtaining an image of the event from a camera, where the event includes a scene having at least one line segment which includes a portion which is occluded by an object, and the object has a width which is greater than a width of the portion.” [Fig. 5d-5j]) While Gloudemans_2009 applies this functionality to the context of sports broadcasts, it would be obvious to one of ordinary skill in the art that these techniques could be easily applied to architectural matters, such as automatically recognizing and removing powerlines or fenceposts from architectural drawings. Overall, one of ordinary skill in the art would have recognized that combining Cheng_2020, Gupta_2006, Ahmed, Bailey_2003, and Arandiga_1998 with Gloudemans_2009 would result in a system that removes unimportant image elements more accurately and automatically.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael P Mirabito whose telephone number is (703)756-1494. The examiner can normally be reached M-F 10:30 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at (571) 272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.P.M./ Examiner, Art Unit 2187
/EMERSON C PUENTE/ Supervisory Patent Examiner, Art Unit 2187