DETAILED ACTION
Claims 1-22 are presented for examination. This action is made in response to the claims filed August 15, 2023.
Claims 1, 4, 6-8, 14, and 16-20 are objected to for informalities.
Claims 1-22 are rejected under 35 USC 112(b) as indefinite.
Claims 1-22 are rejected under 35 USC 112(a) as lacking enablement and best mode.
Claims 1-4 and 6-22 are rejected under 35 USC 103 as unpatentable over Sun in view Chang.
Claim 5 is rejected under 35 USC 103 as unpatentable over Sun in view of Chang, and Czerniawski.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 4, 6-8, 14, and 16-20 are objected to because of the following informalities:
EXAMINER’S NOTE: While the preamble is often given no weight in interpretation, the claims explicitly bring the limitations of the preamble into the claims. Accordingly, the features of the preamble are given patentable weight and are expected to be consistent and definite when considered with the other elements of the claims.
Claims 1 and 18 recite, “construction levels.” There is no standard for what a construction level is in the specification, and it is a not a term of art. The Applicant is advised to amend the claims to remove the term.
Claims 1 and 18 recite, “factoring in dynamic data and measuring parameters […] and generating standardized geo-encoding output” and also recite, “the dynamic data and measuring parameters at least comprise aerial digital imagery of a geographic area, geo location parameter values for locations in the geographic area and/or construction parameter values and/or measurement-based exposure parameter values and/or protection parameter values.” In substance, this is a list of alternatives and should be rewritten as such for the purposes of clarity. Further, the final “comprises” term does not even clarify that it is the method that further comprises the elements that follow. The Applicant is advised to drop the list of parameters to the body of the claim. Placing all of these elements in a preamble that is incorporated in substance into the claim merely serves to obfuscate the claims. However, if the Applicant insists that the elements inexplicably and confusingly remain in a list in the preamble, The Applicant is advised to restructure the preamble. This includes removing the “and/or” language (which is identical to “or” in patent construction), removing the “at least” language (which the term, “comprises,” already indicates), formatting the list of alternative parameters as a list with clear indentation, and clarifying that it is the recited method that comprises the steps in the body of the claim that follows the preamble.
An example amendment could include:
“A method, implemented by processing circuitry of a digital platform, for automated standardized location digital twins of physical constructions factoring in dynamic data and measuring parameters at different construction levels and generating standardized geo-encoding output, the dynamic data and measuring parameters comprise:
aerial digital imagery of a geographic area;
geo location parameter values for locations in the geographic area;
construction parameter values;
measurement-based exposure parameter values; or
protection parameter values,
the method comprising:
capturing […];
displaying […]; […]”
Claims 1 and 18 recite, “the geo location parameters being technically measurable parameters indicating at least a latitude, longitude, elevation, surface area and/or soil conditions of the sub area and/or the property asset of interest.” This uses the same “and/or” language, which adds unnecessary language to the claim and confounds the clarity of the claim. Also, the “at least” language is superfluous and only serves to obfuscate the substance of the claim.
Claims 1 and 18 recite, “identifying a sub-area by indicating the construction identification by setting polygon-shaped boundaries around the physical construction at the geographic area to generate a digital 2-dimensional construction lay-out on the digital imagery, wherein assembling the digital 2-dimensional construction lay-out comprises assigning one or more floor levels providing a 3-dimensional volumetric construction lay-out on the digital imagery.” This clearly represents more than one method step, and the arrangement of the steps makes no sense. “Identifying to generate” appears to be a deliberate effort to confuse those who would read and attempt to interpret the claim. The Applicant is advised to separate this element into at least two separate gerund verb steps of identifying and generating. This will also help clarify the qualification of the “assembling” element in dependent claims like claim 2.
Claims 1 and 18 recite, “and/or” in many different limitations. “And/or” is also recited in claims 4, 7-8, 14, 17, and 19. Please amend to replace them with “or.”
Claims 1 and 18 recite, “at least” in many different limitations. “At least” is also recited in claims 6, 11, 16, and 19. Please amend to replace them with “a” or a similar concise term to reduce the verbosity of the claim.
Claims 1 and 18 recite, “the image data.” There is no primary antecedent basis for this feature.
Claim 7 recites, “parameter values are auto-allocated per building of a complex or construction pro-rata.” The specification does not clarify what pro-rata means in this context. Without this standard, it is impossible to determine what the claim actually entails.
Claim 11 recites, “wherein each location digital twin of a physical property asset includes at least a standardized set of asset risk parameters including geo location parameters and measurement-based risk relevant data.” It is unclear how a location can include a standardized set of parameters. Locations are locations. Images are images. Data is data. The claim does not make clear in which of these dimensions, if any, the recited inclusions applies.
Claim 20 recites, “a boundary.” It is unclear how this relates to the identified “polygon-shaped boundaries” of claim 18 from which claim 20 was likely intended to depend, whether directly or indirectly through a likely dependence of claim 19.
Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-22 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
EXAMINER’S NOTE: While the preamble is often given no weight in interpretation, the claims explicitly bring the limitations of the preamble into the claims. Accordingly, the features of the preamble are given patentable weight and are expected to be consistent and definite when considered with the other elements of the claims.
Claims 1 and 18 recite, “capturing and displaying a digital imagery of a geographic area including a location of a physical construction of interest on a display of a user interface.” However, there is no indication how one captures digital imagery using a user interface. The Applicant is advised to limit each limitation to one gerund form action to clarify the actions taken. Also, this same limitation claims a further action, “wherein at least some of the geo location parameters are extracted from the image data by image recognition.” This should also be a separate gerund form action step, for example, a further “extracting” step. The Applicant should reformat this limitation as three different steps for clarity. The current formatting of the claims confounds the metes and bounds of the claims.
Claims 1 and 18 recite, “technically measurable.” “Technically” is a relative term with no clear discernible scope. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “identifying a sub-area by indicating the construction identification.” There is no primary antecedent basis for “the construction identification.” This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “capturing and displaying a digital imagery of a geographic area” and “identifying a sub-area by indicating the construction identification.” It is unclear what the relationship is between the recited “geographic area,” “construction identification” and “sub-area.” This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “generate a digital 2-dimensional construction lay-out on the digital imagery, wherein assembling the digital 2-dimensional construction lay-out comprises assigning one or more floor levels providing a 3-dimensional volumetric construction lay-out on the digital imagery,” The assembling appears intended to refer to the generation in the prior clause, but it is unclear from the difference in the language. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “identifying a sub-area by indicating the construction identification by setting polygon-shaped boundaries around the physical construction at the geographic area to generate a digital 2-dimensional construction lay-out on the digital imagery.” This does not make sense. For purposes of compact prosecution, this limitation will be interpreted to mean that a boundary is determined around the “physical construction” in a 2-D rendering. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “wherein assembling the digital 2-dimensional construction lay-out comprises assigning one or more floor levels providing a 3-dimensional volumetric construction lay-out on the digital imagery.” It is unclear how assembling a 2-D construction layout includes assigning one or more floor levels to provide a 3-D construction layout on the digital imagery. Providing 3-D floor data does not assemble a 2-D construction layout. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recites, “generating a digital, polygon-based voxel layout providing the 3-dimensional representation of the digital 2-dimensional construction lay-out by encoding irregular voxel grids with non-uniform space partitioning.” The recited “3-dimensional representation of the digital 2-dimensional construction lay-out by encoding irregular voxel grids with non-uniform space partitioning.” Has no corresponding primary antecedence. Further, the relationship between the recited “irregular voxel grids” and the previously recited “polygon-shaped boundaries” is unclear. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 1 and 18 recite, “wherein the digital input imagery and the assembled polygon-based voxel layout is bridged by a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout.” “Is bridged by” is not clear claim language. Bridging is not only broad, but it is unclear what elements bridging entailed. Looking to pages 20-21 of the Applicant’s Specification in the Application, there is no guidance as to the metes and bounds of this term. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 3 recites, “further generating the standardized location digital twin output by a predefined location digital twin format.” Outputting by a format makes no sense. Formats do not output. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 4 recites, “further assigning via the user interface constructions and/or specifiable locations of third-party constructions and exposure measurands.” It is unclear what this assigning means. Assigning to what? Is it assigning a variable? Assigning necessarily requires a relationship that is not stated in the claim. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 5 recites, “further assigning fire protection rating values of partition walls.” It is unclear what this assigning means. Assigning to what? Is it assigning a variable? Assigning necessarily requires a relationship that is not stated in the claim. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 6 recites, “wherein the construction lay-out is at least partially auto-populated by a hazard exposure value or hazard protection value.” However, it is unclear what it means to auto-populate a layout by a value.” The specification provides no clarification for what this means. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 7 recites, “wherein loss scenarios are automatically provided where parameter values are assignable at construction level” It does not mean anything to automatically provide loss scenarios where parameter values are assignable at construction level. Again, it is unclear what a construction level is. There is no basis in the specification for this construction level of the claimed automatic provision. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 7 recites, “parameter values are automatically updated based on the dynamic input data.” There is no primary antecedent basis in the claims for the feature, “dynamic input data.” This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 8 recites, “wherein based on the dynamic data and measuring parameters and the generated location digital twin critical equipment and/or critical process flows are automatically indicated on the construction lay-out.” The claim recites based on the [A and B] and the [C] and/or the [D] are automatically indicated on the construction lay-out. This makes it unclear what elements (e.g., [A], [B], [C], or [D]) form the basis of the determination and what elements are the result of the determination. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 9 recites, “wherein the loss scenarios are indicated dynamically on the digital imagery and construction lay-out.” There is no primary antecedent basis for the claimed “loss scenarios.” It is possible that the Applicant intended the dependency chain to extend from claim 9 to claim 8 and further to claim 7, but claim 8 depends from claim 1. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claim 10 recites, “wherein the location digital twin is indicated as an aerial overview of the geographic area including the sub-area around the physical property asset at the location of interest and the asset risk parameters.” It is unclear what it means that “the location digital twin is indicated as an aerial overview of the geographic area including […] the asset risk parameters.” This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 10, 14, and 15 recite “the asset risk parameters,” but there is no primary antecedent basis for this feature. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 13, 14, and 15 recite “the/additional measurement-based risk relevant data.” There is no primary antecedence for these features. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims.
Claims 19-22 recite, “The digital platform for automated risk analysis according to The digital platform for automated risk analysis according to” It seems that the Applicant intended to create a dependency relationship between claims, but the Applicant repeated the dependency language and also failed to establish, upon which claims, the claims 19-22 depend. This makes it impossible for the person of ordinary skill in the art to discern the metes and bounds of the claims. Because the claims have deficient dependencies, any art rejection would be speculative. Therefore, no art will be applied in this action to these claims.
Dependent claims that depend from the rejected claims are rejected for at least the same reasons.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
EXAMINER’S NOTE: It is clear from the language of the claim that the Applicant lifted the claim language referencing machine learning architecture from the secondary art reference of record, NPL: “Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation” by Chang et al. Specifically, the language was taken from the Abstract and 4.1.3 on Page 4.
Enablement
Enablement of Any Implementation
Claims 1-22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention.
Claims 1 and 18 recite,
wherein the digital input imagery and the assembled polygon-based voxel layout is bridged by a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout
The only references to this feature in the Applicant’s specification include
PAGES 20-21
As an embodiment variant, to overcome technical challenges and limitations of the prior art systems, the invention can e.g. be based on digital, polygon- based voxel layout, providing a novel 3D representation that can encode irregular voxel grids with non-uniform space partitioning. To bridge between the digital input imagery and the assembled polygon-based voxel layout, a pointer-based cross-modal modules can e.g. be used in a generative adversarial layout network. The pointer module can be used not only for message passing, but also as a decoder to output probability over a dynamic set of valid boundary conditions. The main components of this embodiment variant are: (i) a new 3D location digital twin with a polygon-based voxel layout; (ii) a graph-conditioned generative adversarial network using GNN and pointer-based cross-modal module; (iii) an automated pipeline to assemble and generate valid volumetric construction levels and floors through simple graphical interaction; and (iv) a synthetic dataset that can e.g. contain known volumetric constructions and their corresponding layouts, which allow a new way of automated error detection of generated location digital twin, which no known prior art system is able to provide.
The following will demonstrate that the machine learning technology described was relatively new at the time of filing, complicated to effectively deploy (if it was even satisfactorily completed in the art yet), and that a person of ordinary skill in the art would require undue experimentation to realize the terms of the claim from the Applicant’s specification, if the person of ordinary skill in the art were even able to do so at all with infinite experimentation. Analysis is provided in line with the Wands factors of MPEP 2164.
(A) The breadth of the claims: With respect to the features at issue, the claim claims all machine learning arrangements of a graph-conditioned generative adversarial network using GNN and pointer-based cross-modal “module.” There are significant issues with the clarity of this statement, but it is clear that the claims broadly claim a set of machine learning models that are complicated to connect and have the effect the Applicant intends.
(B) The nature of the invention: The invention is a method and system for automated standardized location digital twins of physical construction factoring in dynamic data and measuring parameters at different construction levels and generating standardized geo-encoding output. The invention and corresponding art should therefore be characterized using digital twins for geographic mapping. The feature at issue of the claims recites a complex arrangement of neural networks that would require significant understanding of the machine learning to implement.
(C) The state of the prior art: A prior art search revealed that the language of the claim was likely taken from a particular inventive entity, represented in NPL: “Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation” by Chang et al. (Chang). The inventors worked for Autodesk, a leader in the art of modeling buildings for display, as the claimed feature is used in the claim, and the paper was published only months prior to this application. This reference is a representative effort on the leading edge of the art. This publication demonstrates the difficulty in constructing and executing the model that the Applicant attempted to describe with no technical specification. The paper even goes so far as to say in its conclusion that,
In this paper, we try to provide a novel pipeline, Building-GAN, to improve the efficiency on a realistic professional task, volumetric design in the architectural and construction industry. We invent a 3D representation, voxel graph, to represent building designs, and design a genera tor with a cross-modal pointer module to connect the program graph and voxel graph. Our extensive evaluations, including user testing and user study, show that architects can create numerous valid and valuable designs by interacting with Building-GAN. Future works include enforcing the constraints, such as connectivity, TPR, and FAR, as well as extending the voxel graph for non-cuboid geometries. We will release our code, model, and dataset, and invite the research community to work together on design-related problems in the industries.
Essentially, the complicated system was still a work in progress at the time of filing and was not ready for reliable deployment, as in the Applicant’s claim, at the time of filing. This means that the state of the art did not include a completed implementation of the system the Applicant has attempted to claim with essentially no description. That is, the subject matter the Applicant, at best, referenced from this paper, is the leading edge of research, a level of extraordinary skill, rather than the actual accepted state of the art at the time.
(D) The level of one or ordinary skill in the art: A person of ordinary skill in the art has moderate knowledge of digital twins, perhaps a B.S. or M.S. level computer science worker and the use of at least some models to relate the digital twins to the real world. This is also qualified by the fact that the claims also relate to insurance risk, geographic factors, and catastrophes, meaning some of the expertise of the person of ordinary skill in the art of the claims is also allocated to business and geology, qualifying and somewhat diminishing the focus of the person of ordinary skill in the art on machine learning architecture. By contrast, the authors of the Chang reference are PhDs on the cutting edge of digital twin architecture for construction working for leading players in the art, Autodesk Research in the US and the Obayashi AI Design Lab in Japan. The recitations of the Chang reference represent extraordinary skill in the art, that the identified person of ordinary skill in the art did not possess at the time of filing and could not replicate without undue experimentation. That is, the Applicant cannot rely on the disclosure of the paper, and its implementation, being within the knowledge of the level of one of ordinary skill in the art at the time of filing.
(E) The level of predictability of the art: As illustrated in the Chang reference, the work is incomplete, despite a complex and state of the art architecture. If the feature at issue is considered, the level of predictability was low at the time of the filing date.
(F) The amount of direction provided by the inventor: The inventor has provided almost no direction, as indicated from the excerpt from the Applicant’s specification pages 20-21 quoted above. The detail was barely enough to bring up the Chang reference in a search, let alone act as a template from which a person of ordinary skill in the art could make and use the claimed “pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout” without undue experimentation.
(G) The existence of working examples: The Applicant has provided no working examples. In fact, the Applicant did not even take the time to copy the work of the Applicant into the specification to demonstrate at least one incomplete (i.e., still flawed according to the conclusion of the paper) implementation of the network the Applicant attempted to claim.
(H) The quantity of Experimentation needed to make or use the invention based on the content of the disclosure: The Chang reference provides a “simplified” overview of the architecture used as in FIG. 4 on page 6:
PNG
media_image1.png
195
667
media_image1.png
Greyscale
The identified person of ordinary skill in the art lacked the skills to interpret and implement this architecture at the time of filing without undue experimentation without reference to this paper. This is such a complex architecture that it took a team of experts with extraordinary knowledge in the art to assemble a model that sort of works.
Given the state of the art at the time of filing, the limited and diverse experience of the person of ordinary skill in the art, the lack of guidance provided in the specification, and the complicated relationships between elements of the claims when actually implemented, the identified person of ordinary skill in the art, approaching the problem with the specification as a reference (without the Chang reference to which the Applicant clearly referred without any implementation detail), would not be able to make and use the invention with infinite experimentation, let alone undue experimentation.
To summarize, the person of ordinary skill in the art is someone with familiarity with digital twins for the purpose of geo-mapping and assessing insurance risk in catastrophes. Even the reference from which the “inventor” of the instant application lifted the language of the claim considered the architecture unfinished and provided significant detail and architecture that the specification lacks. The Applicant’s specification, therefore, enables essentially none of the elements of the model architecture that the specification only briefly describes in passing. Even if the technology had already been developed to the extent that it could support the features of the Applicant’s claim, and even if the Applicant clearly lifted language from the Chang publication that the language of the specification would potentially lead a person of ordinary skill in the art to find, the identified person of ordinary skill in the art could not likely make and use the invention without undue experimentation based on the teachings of the specification and the background knowledge of an ordinarily skilled person. Further, even if the technology had already been developed to the extent that it could support the features of the Applicant’s claim, and even if the Applicant clearly lifted language from the Chang publication that the language of the specification would potentially lead a person of ordinary skill in the art to find, the Applicant cannot rely on material absent from the specification to enable the invention.
Accordingly, not even a single implementation of the identified claim feature is enabled.
Enablement of the Full Scope of Claimed Implementations
Additionally, even if the Applicant’s specification enabled even a single implementation of “a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout,” there are so many possible combinations of machine learning architectures that could include a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout, that the Applicant could not possibly enable the full scope of possible architectures. At the time of filing, the Chang reference demonstrates that the team of cutting-edge experts in the art could not confidently deploy a single implementation that meets the standards of the research team. Not even the team of cutting-edge experts certainly did not enable the full range of possible architectures that are covered by the claim language.
Accordingly, the full scope of the identified claim features is not enabled.
Best Mode
Claims 1-22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the best mode contemplated by the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s) has not been disclosed. Evidence of concealment of the best mode is based upon the use of the exact unique language of the Chang reference in the claims and written description without including the architecture and implementation details in the written description that could have been included with simple copy and paste operations.
Specifically, MPEP 2165.03(II) provides two components for best mode analysis:
(A) Determine whether, at the time the application was filed, the inventor knew of a mode of practicing the claimed invention that the inventor considered better than any other.
Claims 1 and 18 recite,
wherein the digital input imagery and the assembled polygon-based voxel layout is bridged by a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout
The Chang reference recites at Page 4.1.3 Pointer-based Cross-Modal Module,
To bridge between the program graph and the voxel graph, we introduce a pointer-based cross-modal module. Inspired by the application [21, 16] of the Pointer Network [24] in natural language processing and mesh generation tasks, we construct a pointer module to achieve message passing between the voxel nodes and all the program nodes on the same story. We cannot use a fixed length output to model program type distribution since 1) different stories can have different numbers of program nodes to choose from, for example, one floor has five rooms and another one has seven rooms; and 2) if there are two program nodes with the same program type, we want to differentiate between the two nodes, such as two restrooms in the same floor. […] We experiment different ways to integrate the pointer module. It can be placed after every several message passing steps in voxel GNN.
This paper with this language was published on April 27, 2021, a little more than six months before the Applicant filed the priority document of the instant application on November 5, 2021.
Based on the use by the Applicant in the Applicant’s claims of this exact language that is unique to the Chang reference published prior to the priority date of the claims, it is clear that the Applicant knew of the modes presented in the reference and that the verbatim use of the language demonstrates that the inventor considered the modeling arrangement to be the best for use in the Applicant’s claims.
(B) Compare what was known in (A) with what was disclosed – is the disclosure adequate to enable one skilled in the art to practice the best mode?
A good example of the architecture provided in the Chang reference from which the Applicant lifted the claim language is the, albeit simplified, overview of the architecture in FIG. 4 on page 6 of the Chang reference:
PNG
media_image1.png
195
667
media_image1.png
Greyscale
As demonstrated in the enablement rejections, the Applicant provided very little information about this architecture (see the quotation presented in the enablement rejection above from the Applicant’s specification Pages 20-21). As demonstrated, when compared to what is known in the Chang reference, the disclosure (or lack thereof) in the Applicant’s specification is wholly inadequate to enable one skilled in the art to practice the best mode of the claimed “invention.”
Overall Assessment
The first inquiry (A) is answered in the affirmative, and the second inquiry (B) is answered in the negative based on reasons to support the conclusion that the specification is non-enabling with respect to the best mode. Despite lifting the claim language of the claimed machine learning component from Chang, the Applicant provided none of the potentially enabling disclosure of Chang, nor any other implementation of a working architecture for the claimed machine learning system. Therefore, it is clear that, having referenced the Chang paper (i.e., having used language taken verbatim therefrom), the Applicant deliberately concealed/omitted any mode of the invention, let alone the best mode of the invention known to the inventor at the time of filing. Assessing the adequacy of the disclosure in this regard, the information contained in the specification disclosure is insufficient to enable a person skilled in the relevant art to make and use the best mode of the claims. Therefore, the Applicant failed to disclose, in the Applicant’s specification, the best mode the inventor contemplated of the claimed invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 and 6-22: Sun and Chang
Claims 1-4 and 6-22 are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0165616 A1 to Sun et al. (Sun) in view of NPL: “Building GAN: Graph-Conditioned Architectural Volumetric Design Generation” to Chang et al. (Chang).
Claims 1 and 18
Regarding claim 18, Sun teaches:
A digital platform for automated standardized location digital twins of physical constructions factoring in dynamic data and measuring parameters at different construction levels and generating standardized geo-encoding output, the dynamic data and measuring parameters at least comprise aerial digital imagery of a geographic area, geo location parameter values for locations in the geographic area and/or construction parameter values and/or measurement-based exposure parameter values and/or protection parameter values, the digital platform comprising: processing circuitry configured to (Sun [0140] 6) “The present invention transforms models of water events, produced by its own or acquired from various sources, into computer services (a.k.a. web services, software services, etc.) so that one or more capabilities of the water models can be accessed through a prescribed interface. For example, the present invention “live” stream KMLs for consumption by software such as Google Earth (Pro edition or others.) On the side of consuming devices, a piece of software uses “network links” to access this server and request data in KML format. Various machines, such as those with Google Earth (Pro edition or others) installed, can directly access, interpret, and manipulate the data of the water model. This method of web service is particularly effective and efficient to make water models accessible timely and by large numbers of machines. Besides KML, various formats and protocols exist for web services including Open Geo spatial Consortium's WMS format, an open standard for streaming location information. One mechanism to construct such services is using an “application server,” such as ArcGIS Server, to transform the water models into various streaming services.” – This teaches a digital platform for digital twins to be executed on processing circuitry based on dynamic data to produce standardized geo-encoding output, such as KML. Also, see paragraphs [0006], [0030], [0062], and [0065] for the rest of the alternatively claimed input.)
capture and display a digital imagery of a geographic area including a location of a physical construction of interest on a display of a user interface, wherein at least some of the geo location parameters are extracted from the image data by image recognition, the geo location parameters being technically measurable parameters indicating at least a latitude, longitude, elevation, surface area and/or soil conditions of the sub area and/or the property asset of interest, (Sun See FIGs. 3 to 7 (Shown below)– These show the capture and display of captured imagery that include a location of interest associated with the input. [0062], [0065] “obtaining from satellite imagery analysis” [0072] “Observation of flooding event are frequently available as satellite imagery or aerial photos. Not only the spatial extent of flooding can be delineated based on such data, but also other information such as water flow volume, depth information, etc. can be estimated when combining with extra data such as terrain characteristics.” – the input includes digital imagery [0060] – Soil data at the location [0063] – The State Plane Coordinate System includes equivalents of latitude and longitude within its system.
PNG
media_image2.png
998
806
media_image2.png
Greyscale
PNG
media_image3.png
1011
806
media_image3.png
Greyscale
PNG
media_image4.png
1042
816
media_image4.png
Greyscale
PNG
media_image5.png
1004
801
media_image5.png
Greyscale
PNG
media_image6.png
997
799
media_image6.png
Greyscale
identify a sub-area by indicating the construction identification by setting polygon-shaped boundaries around the physical construction at the geographic area to generate a digital 2-dimensional construction lay-out on the digital imagery, (Sun [0180] “The present invention produces, distributes, and acquires virtual perimeters of events for automatic machine action. A virtual perimeter can be produced based on models of events and other relevant information. In its simplest form, it is the boundary of an impacted area by an event. In a more detailed description, a virtual perimeter is just a set of spatial selecting criteria that can be used to define an impacted area. The present invention publishes virtual perimeters through machine services. The production and distribution can be requested by another machine over network. Once a virtual perimeter is produced or acquired, the present invention uses it to perform various machine actions including selecting features, placing objects, manipulating autonomous moving objects, identifying advertisement targets based on spatial and temporal factors of the perimeter, pushing advertisement, sending information, notification, or other automatic machine actions, and timing actions.” – Virtual perimeters are drawn around buildings and subareas.)
merge the digital imagery of the geographic area with a 2-dimensional geographic or topographic digital event foot-print of a selected natural catastrophic event, (Sun [0063] “The present invention also uses various data and utilizes any interested data with location information to serve various purposes, such as, locating and identifying features within an inundation area. Some examples of such data are: roads; bridges; infrastructures; mobile objects; demographic data; building footprints; building characteristics; time zones systems; and State Plane Coordinate System (SPCS) coverage, etc.” – The digital imagery of the geographic area is merged with a 2-D geographic or topological digital event footprint of a selected natural catastrophic event, e.g., an inundation event. [0072] “Observation of flooding event are frequently available as satellite imagery or aerial photos. Not only the spatial extent of flooding can be delineated based on such data, but also other information such as water flow volume, depth information, etc. can be estimated when combining with extra data such as terrain characteristics.” – The merged data representation includes digital imagery.)
assemble the digital 2-dimensional construction lay-out by graphically assigning hierarchic-structured levels comprising at least complexes and/or buildings and/or compartments via the graphical user interface, (Sun [0063] “The present invention also uses various data and utilizes any interested data with location information to serve various purposes, such as, locating and identifying features within an inundation area. Some examples of such data are: roads; bridges; infrastructures; mobile objects; demographic data; building footprints; building characteristics; time zones systems; and State Plane Coordinate System (SPCS) coverage, etc.” – The Images of buildings and complexes are assembled to be displayed inline with the other information.)
extend the digital 2-dimensional construction lay-out by graphically assigning at least physical construction characteristic parameters and/or fire protection parameters and/or fire detection parameters and/or water source parameters and/or hazard control and protection parameters, and (Sun [0057] “The US GEOLOGICAL SURVEY (USGS) National Water-Use Information System (NWIS) provides the nation's water-use information aggregated at the county, state, and national levels through Web Interface and Web Services. The present invention system accesses such web services to retrieve water data via automated methods and parse the results to be used in hydrologic analyses and processes. Peak flow data retrieved from the NWIS are used in frequency analysis to develop flood flow event such as 100-year event flow to create Federal Emergency Management Agency (FEMA) flood hazard areas. Instant stream flow is used in hydraulic models to generate water elevation surface models, based on which to produce inundation models.” – This provides water source data for the catastrophe emulation. [0117] “First Impacting Stage is a threshold water stage (water height) measured at a location (reference location that is usually different from a location of interest) when water starts to “touch” a location or a structure of interest. This concept can have various and practical usages. For example, for a location or a structure such as an intersection or a building. First Impacting Stage can be 24 feet, meaning if the water stage reaches 24 feet at “the reference point”, the water would start to affect the location or the structure. People can simply pay attention to what stage it is at the reference point, and instantly know whether that stage would mean danger. A community can set up its own “reference point” a location and in advance model flooding events of various staves to determine the First Impacting stage of all the locations of interests. By monitoring and communicating the actual measurement during an event in a timely fashion, citizens would easily understand the meaning of the message and get proper warnings. For communities, this is a “cheap” way to set up a flood monitoring system. It can be as simple as a painted ruler at the river bank or bridge, and a webcam points to it and lively broadcasts stage. Communities does not need to have sophisticated and expensive equipment to measure water stage to get a flood warning in advance. A person does not even have to look at a map to understand or predict the danger. A radio message would suffice.” – This provides hazard control and protection parameters for the catastrophe emulation.)
generate a location digital twin based on the digital imagery and the combined digital 2-dimensional construction having a standardized output for the construction based on the dynamic data and measuring parameter values. (Sun See Figs. 3-7 and [146]-[0166] mentioning 2-D presentation. For a standardized output see [0128]-[0144] “The present invention transforms water models into proper formats … for software such as Google Earth.” And for the measuring parameter values, Again See paragraphs [0046], [0066], [0098], [0102], and [0117])
Sun teaches combining outputs of models and images for 2-D and 3-D emulations, but does not explicitly teach, but Sun in view of Chang teaches:
identify a sub-area by indicating the construction identification by setting polygon-shaped boundaries around the physical construction at the geographic area to generate a digital 2-dimensional construction lay-out on the digital imagery, wherein assembling the digital 2-dimensional construction lay-out comprises assigning one or more floor levels providing a 3-dimensional volumetric construction lay-out on the digital imagery, (Chang Page 3 “Given a building datum, we first construct 2D program graphs for each story. Each program node feature includes the program type and the story level. Here, we consider 6 program types: lobby/corridor, restroom, stairs, elevator, office, and mechanical room. A program edge shows the two programs are connected by a door or opening. To construct the 3D program graph, we stack all 2D program graphs and chain the stairs and elevators, since they are the only paths for moving vertically. In practice, the 3D program graph also represents the circulation of the building.” – One or more floor designs, each having a 2-D layout, are piked on each other to make 3-D building renderings.)
generate a digital, polygon-based voxel layout providing the 3-dimensional representation of the digital 2-dimensional construction lay-out by encoding irregular voxel grids with non-uniform space partitioning, (Chang Page 2, Left Column, Last Paragraph – Right Column, First Paragraph “To overcome these challenges and limitations, we propose voxel graph, a novel 3D representation that can encode irregular voxel grids with non-uniform space partitioning. To bridge between the input program graph and the output voxel graph, we design a pointer-based cross-modal modules in our generative adversarial graph network. The pointer module can be used not only for message passing, but also as a decoder to output probability over a dynamic set of valid programs.” – A polygon-based voxel layout that provides a 3-D rendering of a 2-D construction layout is generated by encoding irregular voxel grids with non-uniform space partitioning.)
wherein the digital input imagery and the assembled polygon-based voxel layout is bridged by a pointer-based cross-modal module using a generative adversarial layout network and graphical neural network for passing messages between the digital input imagery and the assembled polygon-based voxel layout, and (Chang Page 4, 4.1.3 Pointer-based Cross-Modal Module “After processing the program graph with the program GNN, the final embedding of program nodes can be viewed as the virtual ”blueprint” of a design. Therefore, it is necessary to ”look” at this blueprint to generate the output. To bridge between the program graph and the voxel graph, we introduce a pointer-based cross-modal module. Inspired by the application [21, 16] of the Pointer Network [24] in natural language processing and mesh generation tasks, we construct a pointer module to achieve message passing between the voxel nodes and all the program nodes on the same story. We cannot use a fixed length output to model program type distribution since 1) different stories can have different numbers of program nodes to choose from, for example, one floor has five rooms and another one has seven rooms; and 2) if there are two program nodes with the same program type, we want to differentiate between the two nodes, such as two restrooms in the same floor.” – The claim lifted this language from this excerpt of the paper. This is the exact network that the patent discusses.)
wherein the polygon-based voxel layout combines voxel-based and imagery-based layouts by encoding voxels into polygon-shaped graph nodes, (Chang Page 2, Left Column, Last Bullet – Right Column, First Paragraph “Volumetric images (usually defined as 3D regular grids with uniformly discretized voxels) have the closest structural similarity to rectangular buildings than other 3D representations, such as point clouds or meshes. However, it is not computational and memory efficient to use this dense representation for polygonal rooms. Moreover, there are voxels within the regular grid but not in the irregular valid design space that take unneeded memory and computation. To overcome these challenges and limitations, we propose voxel graph, a novel 3D representation that can encode irregular voxel grids with non-uniform space partitioning. To bridge between the input program graph and the output voxel graph, we design a pointer-based cross-modal modules in our generative adversarial graph network. The pointer module can be used not only for message passing, but also as a decoder to output probability over a dynamic set of valid programs.” – The polygon-based voxel layout combines imagery-based layouts by encoding voxels into polygon-shaped graph nodes.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claims to modify the generic 3-D rendering of models used with FEA in Sun by the machine learning methods of Chang because the person of ordinary skill in the art would be motivated by the aim of Sun to make realistic emulations/digital twins to look to Chang, which provides models that generate realistic 3D volumetric designs and outperforms previous methods and baselines. (Sun [0152] “4. The current preferred preference for creating visualization of Federal Emergency Management Agency (FEMA) Flood Insurance Map (FIRM) products, inundation maps, multi-frequency products, or other data, is to create detailed and photo realistic 3D visualization. To accomplish this, the current preference is to use software such as Google Earth (Pro edition or others), which provides photo realistic buildings, trees, etc. in 3D.” [0154] “Similarly, the present invention transforms various inundation models and products, including those maps and libraries created and hosted by government agencies and their associates (e.g. NWS, USGS, and State of North Carolina FIMAN system) into various forms and formats for human (and machine) to consume.” [0162] “The present invention currently prefers producing detailed 3D and photo realistic visualizations of water events. It accomplishes this by using software such as Google Earth (Pro, VR, or another edition. Configuration includes turning on “Photo realistic atmosphere” in Preference and “Detailed 3D” layer in the Default Layers.)”; Chang Abstract “Our generator is a cross-modal graph neural network that uses a pointer mechanism to connect the input program graph and the output voxel graph, and the whole pipeline is trained using the adversarial framework. The generated designs are evaluated qualitatively by a user study and quantitatively using three metrics: quality, diversity, and connectivity accuracy. We show that our model generates realistic 3D volumetric designs and outperforms previous methods and baselines.)
Regarding claim 1, claim 1 recites the method features that the system of claim 18 is configured to execute and is rejected for the same reasons as claim 18.
Claim 2
Regarding claim 2, Sun in view of Chang teaches claim 1, and further teaches:
further assembling the digital 2-dimensional construction lay-out by graphically assigning one or more floor levels providing a 3- dimensional volumetric construction lay-out on the digital imagery. (Chang Page 2, Left Column, Last Paragraph – Right Column, First Paragraph “To overcome these challenges and limitations, we propose voxel graph, a novel 3D representation that can encode irregular voxel grids with non-uniform space partitioning. To bridge between the input program graph and the output voxel graph, we design a pointer-based cross-modal modules in our generative adversarial graph network. The pointer module can be used not only for message passing, but also as a decoder to output probability over a dynamic set of valid programs.” See Also FIGs. 1 and 3 (both shown below)– This further assembles the digital 2-D construction lay-out by graphically assigning one or more floor levels providing a 3-D volumetric construction lay-out on the digital imagery.)
PNG
media_image7.png
390
290
media_image7.png
Greyscale
PNG
media_image8.png
175
554
media_image8.png
Greyscale
Claim 3
Regarding claim 3, Sun in view of Chang teaches claim 1, and further teaches:
further generating the standardized location digital twin output by a predefined location digital twin format, the format being interchangeable for all possible location digital twins generated. (Sun See FIGs. 3-7 -This shows a standardized location digital twin output , which is interchangeable for all possible location digital twins generated.
PNG
media_image2.png
998
806
media_image2.png
Greyscale
PNG
media_image3.png
1011
806
media_image3.png
Greyscale
PNG
media_image4.png
1042
816
media_image4.png
Greyscale
PNG
media_image5.png
1004
801
media_image5.png
Greyscale
PNG
media_image6.png
997
799
media_image6.png
Greyscale
Claim 4
Regarding claim 4, Sun in view of Chang teaches claim 1, and further teaches:
further assigning via the user interface constructions and/or specifiable locations of third party constructions and exposure measurands. (Sun [0165] “The present invention adds, reduces, modifies, or eliminates information as visual elements in the final output to deliver “just the right amount” of information to the viewer. It adds visual objects, animated or stationary, to further enrich the representation. For example, the present invention adds vehicles, planes, and ships to a video. It also has the flexibility to add existing or planned structures and infrastructures, such as bridges, buildings, shopping malls, wetlands, restoration sites, designing scenarios, etc. These “graphical objects” can be acquired if they exist already, or created using various software packages, such as SketchUp. The present invention performs this task of modification before, during or after the video is recorded. For example, during post processing of a recorded video clip, the present invention often adds in 2D or 3D objects, such as a building.” – The user interface is used to assign constructions and/or specifieble locations of third party constructions and exposure measurands, such as the amount of building inundation.)
Claim 6
Regarding claim 6, Sun in view of Chang teaches claim 1, and further teaches:
wherein the construction lay-out is at least partially auto-populated by a hazard exposure value or hazard protection value. (Sun [0122]-[0123] “The Threshold Events and associated characteristics are valuable for various purposes such as determining historical inundation of a lo cation or a structure. For example, once a first impacting stage is determined at a nearby water gauge, the present invention queries the historical records of that gauge to extract records that are above the threshold the First Impacting Stage. The present invention then determines how many times the location or structure were inundated, which is a valuable piece of information various purposes including setting insurance premium and risk communication. Other characteristics of First Impacting Events can serve similar purposes. Threshold events can serve as unique “risk scores” for a flooding source for any location or structures affixed to that location. – Risk scores are determined.)
Claim 7
Regarding claim 7, Sun in view of Chang teaches claim 1, and further teaches:
wherein loss scenarios are automatically provided where parameter values are assignable at construction level and/or parameter values are auto-allocated per building of a complex or construction pro-rata and/or parameter values are automatically updated based on the dynamic input data. (Sun [0122]-[0123] “The Threshold Events and associated characteristics are valuable for various purposes such as determining historical inundation of a lo cation or a structure. For example, once a first impacting stage is determined at a nearby water gauge, the present invention queries the historical records of that gauge to extract records that are above the threshold the First Impacting Stage. The present invention then determines how many times the location or structure were inundated, which is a valuable piece of information various purposes including setting insurance premium and risk communication. Other characteristics of First Impacting Events can serve similar purposes. Threshold events can serve as unique “risk scores” for a flooding source for any location or structures affixed to that location. – Risk score parameter values are determined for each structure.)
Claim 8
Regarding claim 8, Sun in view of Chang teaches claim 1, and further teaches:
wherein based on the dynamic data and measuring parameters and the generated location digital twin critical equipment and/or critical process flows are automatically indicated on the construction lay-out. (Sun [0205] “A navigational system on a vehicle can suggest a different route to the driver to consider avoiding inundated areas. Similar, a control system on a self-driving driverless, or unmanned vehicle can automatically decide and take a detour to avoid certain areas such as an inundated area.” – Based on the parameters and the models,critical process flows are indicated, including alternative routes to avoid inundated areas.)
Claim 9
Regarding claim 9, Sun in view of Chang teaches claim 8, and further teaches:
wherein the loss scenarios are indicated dynamically on the digital imagery and construction lay-out. (Sun [0157] “The present invention adds various data layers, either static or dynamic, to overlay with water models. For example, the present invention adds one or multiple water models as well as other types of data such as satellite imagery and landmarks. The present invention utilizes static data files, or dynamic web services, by which data is “streamed” to a machine with a display screen to create visualizations. [0181] “The present invention adds various data layers, either static or dynamic, to overlay with water models. For example, the present invention adds one or multiple water models as well as other types of data such as satellite imagery and landmarks. The present invention utilizes static data files, or dynamic web services, by which data is “streamed” to a machine with a display screen to create visualizations.“ – Loss scenarios are dynamically indicated on the digital imagery and construction lay-out.)
Claim 10
Regarding claim 10, Sun in view of Chang teaches claim 1, and further teaches:
wherein the location digital twin is indicated as an aerial overview of the geographic area including the sub-area around the physical property asset at the location of interest and the asset risk parameters. (Sun [0006] “wherein the location digital twin is indicated as an aerial overview of the geographic area including the sub-area around the physical property asset at the location of interest and the asset risk parameters.” [0072] “wherein the location digital twin is indicated as an aerial overview of the geographic area including the sub-area around the physical property asset at the location of interest and the asset risk parameters.” See Also FIGs. 3-7, illustrating the risk to buildings by illustrating inundation – The location digtial twin is indicated as an aerial overview of the geographic area inclding the sub-area around the physical property asset at the location of interest and the asset risk parameters.)
Claim 11
Regarding claim 11, Sun in view of Chang teaches claim 1, and further teaches:
wherein each location digital twin of a physical property asset includes at least a standardized set of asset risk parameters including geo location parameters and measurement-based risk relevant data. (Sun [0009] “Another objective of the present invention is to produce various information products and analytics of various forms. Some examples are: forecast; notification; visualization; fly-over video; base flood elevation; probability; insurance premium rating; virtual perimeter; and integrated analytical report with multiple model outputs on a single map.” – The digital twin also incudes risk parameters for different geo location positions with associated scores and even risk premium ratings.)
Claim 12
Regarding claim 12, Sun in view of Chang teaches claim 1, and further teaches:
wherein structural characteristics of the property asset are extracted from the image data by image recognition and visualized in the image. (Sun [0095] “The present invention takes topographic data, hydrographic data, land use data, stage and flow model (information) to generate computational elements to be used to create various 3D hydraulic models. The present invention executes 3D hydraulic models to produce outputs of the models and reads, extracts, and converts water surface profiles to be assigned to the computational elements. The water surface elevations at the centroids of computational elements are spatially distributed to create water surface elevation model, such as water surface TIN. The preset invention determines the spatial extent of inundated area using various methods such as Gridded Inundation Model (GIM) or Triangular Irregular Network Model (TIN) mentioned above.” [0114] “In this process, the present invention considers of the (bare earth) elevation of the location. This is the “base” First Impacting Frequency that can be consistently determined and compared for any location. It considers the water surface elevation and the bare-earth elevation of terrain. But it does not factor in structure elevation. For example, water of a 1% flood event reaches the bare-earth elevation of a location, but if the structure at the location is elevated, the 1% event flood water would not affect the structure, meaning the First Impacting Frequency is lower than 1% (Describing in another way, the 1% frequency is impacting the location, but not necessarily impacting the structure at the location yet.) To determine First Impacting Characteristics for a structure at a location, structure characteristics such as elevation of the structure and Lowest Floor Elevation, if available, are factored in to produce a more accurate determination of a First Impacting Frequency.” – The inundation of buildings is determined from image interpretation.)
Claim 13
Regarding claim 13, Sun in view of Chang teaches claim 1, and further teaches:
wherein image data of the geographic area, the geo location parameters for locations in the geographic area and the measurement-based risk relevant data for locations in the geographic area are provided by a database by an application programming interface from an external data platform or processing tool. (Sun [0057] “c) The US GEOLOGICAL SURVEY (USGS) National Water-Use Information System (NWIS) provides the nation's water-use information aggregated at the county, state, and national levels through Web Interface and Web Services. The present invention system accesses such web services to retrieve water data via automated methods and parse the results to be used in hydrologic analyses and processes. Peak flow data retrieved from the NWIS are used in frequency analysis to develop flood flow event such as 100-year event flow to create Federal Emergency Management Agency (FEMA) flood hazard areas. Instant stream flow is used in hydraulic models to generate water elevation surface models, based on which to produce inundation models.”
Claim 14
Regarding claim 14, Sun in view of Chang teaches claim 1, and further teaches:
wherein the asset risk parameters of the digital twin are augmented by user input via the user interface, wherein the user input comprises additional geo location parameters, additional measurement-based risk relevant data and/or property-specific information data about the property asset of interest. (Sun [0172] “The water model altitude: Absolute, Clamped to Ground, Clamped to Sea Floor, Relative to Ground, Relative to Sea Floor. Currently, the preferred is to use the ABSOLUTE. Among various ways recording a video in Google Earth (Pro or others), using a previously recorded tour is the current preference. First step is to create a “path” by using the “Add a Path” function in Google Earth (Pro or others). This pre-defined path is then used to create a “tour” in Google Earth (Pro or others) which automatically simulating flying over according to the defined path. This preferred method would not require user input during recording. An alternative workflow is to directly record movement controlled by your mouse, keyboard, or other controlling devices.” [0193] “The present invention acquires key information by querying and retrieving from data source. Such information include elevation, building characteristics (e.g. type of building, basement existence, foundation, etc.) Based on acquired information, analytics necessary for rating risk or insurance premium can be produced. The present invention acquires said key information based on user inputs through a system such as a mobile app or a web browser. The preset invention acquires features with coordinates by querying an existing data source, by analyzing available information, or letting user input such information including digitization on screen as points, lines, or polygons. The present invention produces analytics such as Lowest Adjacent Grade and Highest Adjacent Grade based on information acquired based on the feature with coordinates.” – User input can be taken for go location parameters, measurement-based risk relevant data, or property-specific information.)
Claim 15
Regarding claim 15, Sun in view of Chang teaches claim 1, and further teaches:
wherein the asset risk parameters are presented as a location score card for the measurement-based risk relevant data of the property asset of interest. (Sun [0193] “The present invention acquires key information by querying and retrieving from data source. Such information include elevation, building characteristics (e.g. type of building, basement existence, foundation, etc.) Based on acquired information, analytics necessary for rating risk or insurance premium can be produced. The present invention acquires said key information based on user inputs through a system such as a mobile app or a web browser. The preset invention acquires features with coordinates by querying an existing data source, by analyzing available information, or letting user input such information including digitization on screen as points, lines, or polygons. The present invention produces analytics such as Lowest Adjacent Grade and Highest Adjacent Grade based on information acquired based on the feature with coordinates.” – Scores for asset risk are produced for each location of interest.)
Claim 16
Regarding claim 16, Sun in view of Chang teaches claim 1, and further teaches:
wherein the displayed sub-area includes at least one property asset in form of a building schematically indicated by structural characteristics extracted from the image data by an image recognition algorithm. (Sun [0063] “The present invention also uses various data and utilizes any interested data with location information to serve various purposes, such as, locating and identifying features within an inundation area. Some examples of such data are: roads; bridges; infrastructures; mobile objects; demographic data; building footprints; building characteristics; time zones systems; and State Plane Coordinate System (SPCS) coverage, etc.” [0158] “For example, the present invention uses Google Earth (Pro edition or others) to manipulate the data and create 2D or 3D visualizations of water events, with geographic context such as buildings, trees, water bodies, etc. Operators and viewers directly interact with the software to “customize” the visual effects, such as tilting viewing angle, adding relevant data, or changing the viewing distance.” – The buildings are displayed based on the number of stories that are indicated in the image.)
Claim 17
Regarding claim 17, Sun in view of Chang teaches claim 1, and further teaches:
wherein geocoding is applied to the sub-areas and/or the property asset of interest identified in the sub area to provide geographic coordinates as geo location parameters of the sub area or the property asset. (Chang [0119]-[0120] “The First Impacting Water Surface Elevation for a structure can be determined by bringing in structure characteristics at that location, such as Lowest Structure Elevation (a.k.a. Lowest Floor Elevation or First Floor Elevation). The First Impacting Water Surface Elevation is the one that is equal to the Lowest Structure Elevation. The Threshold Events, such as First Impacting events, can be determined through modeling, or arbitrarily set. For example, an apartment owner on the 15th floor o fan apartment building can simply define his or her “first impacting flood event” as when water touches that floor, which is different from the building owner's determination, which might be when the water level reaches the first floor.” – The floor (geo height location) one is on is indicated and important to determine whether that floor is in danger of inundation.)
Claim 19
Regarding claim 19, Sun in view of Chang teaches claim 1, and further teaches:
further comprising at least a persistence storage having at least one data-structure for capturing technical parameters and/or user-specific parameters, wherein the technical parameters comprise image data of a plurality of geographic areas, geo location parameters for locations in the geographic areas and/or measurement-based risk relevant data for locations in the geographic areas, and wherein the user-specific parameters comprise property-specific information data about the property asset of interest. (Sun [0009] “Another objective of the present invention is to produce various information products and analytics of various forms. Some examples are: forecast; notification; visualization; fly-over video; base flood elevation; probability; insurance premium rating; virtual perimeter; and integrated analytical report with multiple model outputs on a single map.” [0131]-[0133] “The present invention transforms models of water events into various forms and formats such as: KML; KMZ; Shapefile; Geo database; MapInfo's “.tab”; Micro station's “.dgn”; Virtual Raster (*.vrt); Geo TIFF (*.tif); National Imagery Transmission Format (*.ntf); ERDAS Imagine Images (*.img). The present invention transforms water models into suitable formats, such as KML, by using various hardware and software, such as Google Earth products. The current preference is relying on a utility software, without which the task often becomes too tedious. (But an operator can choose to create KMLs by simply using just a common text editor software.) The present invention transforms models of water events into various data structures (data schemas) such as Federal Emergency Management Agency (FEMA) FIRM (Flood Insurance Rate Map) database. It performs such transformation using various combinations of software and hardware. The current preference is utilizing GIS (Geographic Information System) software packages. Examples of such packages include QGIS; ArcGIS by ESRI; Mapinfo; Micro Station; AutoCAD, etc.” – The reference contemplates persistent storage with data structures for capturing all of the recited technical and user-specific parameters.)
Claim 20
Regarding claim 20, Sun in view of Chang teaches claim 1, and further teaches:
wherein the processing circuitry is configured to implement a data receiving module, a display module and a user interface module, wherein the data receiving module is configured to receive image data of a geographic area for display by the display module, and the user interface module is configured to receive user input defining a boundary around the physical property asset at the location of interest for display by the display module. (Sun Claim 8 “ A System for producing and distributing information on water events in accordance to claim 1, further comprising: an interactive system for serving one or multiple groups of users; a display with user interface; a server; a transaction module; a recommendation module; and a platform for attracting and facilitating interactions among one or more groups of users.“ – user interface, data receiving, and display modules. [0009] “Another objective of the present invention is to produce various information products and analytics of various forms. Some examples are: forecast; notification; visualization; fly-over video; base flood elevation; probability; insurance premium rating; virtual perimeter; and integrated analytical report with multiple model outputs on a single map.” [0012] “Another objective of the present invention is to acquire indoor or outdoor elevations through acquisition, user input, analytical process, or barometric measurement.” [0193] “The present invention acquires said key information based on user inputs through a system such as a mobile app or a web browser. The preset invention acquires features with coordinates by querying an existing data source, by analyzing available information, or letting user input such information including digitization on screen as points, lines, or polygons.” – The user interface receives user input including boundaries/perimeters.)
Claim 21
Regarding claim 21, Sun in view of Chang teaches claim 1, and further teaches:
wherein the processing circuitry is configured to implement an image recognition module configured to recognize a property asset in the sub-area and display structural characteristics of the property asset. (Sun [0194] “The present invention determines elevation for both out side and inside of a structure. The present invention determines elevation OUTSIDE a structure through various means including traditional measurement in the field, through a mobile device equipped with a barometer, through analysis based on image analysis, querying various digital elevation datasets, querying and process LiDAR data, querying existing datasets, etc. The present invention determines elevation INSIDE a structure through various means including traditional surveying, querying and/or analyzing existing databases of building characteristics, image analysis, using a barometric device, or acquiring from a data source, etc. To determine elevations, especially elevations INSIDE a structure, the present invention prefers using a device, such as a mobile phone, on-site for obtaining elevation characteristics. This device is equipped with a sensor or sensors capable of measuring certain conditions of a location, such as barometric pressure, based on which elevation of the device is be directly or indirectly obtained. Currently preferred method for calibrating is to have at least two control points for measuring elevation inside a structure, one inside and one outside. The final elevation indoor is the result of the outside base elevation plus/minus the elevation difference between outside control point and the inside control point, then plus/minus the elevation difference between the indoor control point and the intended position for elevation acquisition. The current preference is to acquire elevation at one or more locations inside the structure.” – Image recognition is used to determine the floor and/or elevation within the building, and the floor/elevation is displayed.)
Claim 22
Regarding claim 22, Sun in view of Chang teaches claim 1, and further teaches:
wherein the processing circuitry is configured to implement an aggregation module configured to aggregate the image data of a geographic area, geo location parameters for locations in the sub-area and measurement-based risk relevant data for locations in the sub-area to generate the location digital twin for a property asset of interest in the sub-area. (Sun [0052]-[0062] and [0148]-[0167] – These include inputs that are used to generate the emulation/digital twin. These include image data of an area, geo location parameters for locations in the area, and measurement-based risk for particular locations.)
Claim 5: Sun, Chang, and Czer
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0165616 A1 to Sun et al. (Sun) in view of NPL: “Building GAN: Graph-Conditioned Architectural Volumetric Design Generation” to Chang et al. (Chang) and NPL: “Updating Digital Models of Existing Commercial Buildings using Deep Learning” by Czerniawski (Czer).
Claim 5
Regarding claim 5, Sun in view of Chang teaches claim 1, and further teaches:
further assigning fire protection rating values of partition walls. (Czer Page 68, Table 3-2 (Shown Below) – This assigns fire protection levels, including a Level 3 designation for an indoor partition wall.
PNG
media_image9.png
607
585
media_image9.png
Greyscale
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claims to modify the risk-related building determinations of Sun by the structural risk considerations of Czer because the person of ordinary skill in the art would be motivated by the aim of Sun to derive building architectures in simulation from image analysis, to look to Czer, which provides hazard risk assessment for different building structures. (Sun [0013] “Another objective of the present invention is to produce and distribute elevation related information, such as lowest adjacent grade, highest adjacent grade, by using geometric features including points, lines, and polygons defined by on-screen digitization.” [0018] “Another further objective of the present invention is to produce and distribute derivatives for various purposes, including flood determination; risk communication; and insurance rating.” [0063] “The present invention also uses various data and utilizes any interested data with location information to serve various purposes, such as, locating and identifying features within an inundation area. Some examples of such data are: roads; bridges; infrastructures; mobile objects; demographic data; building footprints; building characteristics; time zones systems; and State Plane Coordinate System (SPCS) coverage, etc.” [0194] “The present invention determines elevation for both out side and inside of a structure. The present invention determines elevation OUTSIDE a structure through various means including traditional measurement in the field, through a mobile device equipped with a barometer, through analysis based on image analysis, querying various digital elevation datasets, querying and process LiDAR data, querying existing datasets, etc. The present invention determines elevation INSIDE a structure through various means including traditional surveying, querying and/or analyzing existing databases of building characteristics, image analysis, using a barometric device, or acquiring from a data source, etc. ”; Czer Abstract “The vast scale of buildings poses many challenges to the flow of information. Automatically creating and updating building information models will require the adoption of computer vision systems capable of parsing a comprehensive set of building components all the while being resilient to imperfections inherent in large-scale data collection. Existing visual recognition methods have essentially been limited to recognizing floors, walls, ceilings, doors, and windows. To address this limitation, I show how a deep convolutional neural network can be trained to semantically segment RGB-D (i.e. color and depth) images into thirteen building component classes using a new annotated dataset called 3DFacilities. The dataset was designed using a common building taxonomy to ensure comprehensive semantic coverage and was collected from a diversity of buildings to ensure intra-class diversity. Transfer learning, class balancing, and prevention of overfitting are used to effectively overcome the dataset’s borderline adequate class representation. […] The contributions presented herein demonstrate how information systems can be made to contend with the immense spatial and temporal scale of buildings and how organizations must develop the requisite culture to successfully adopt them.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
NPL: “House-GAN: Relational Generative Adversarial Networks for Graph-Constrained House Layout Generation” by Nauta et al. (Teaches an earlier iteration of the methods of the Chang reference using different terminology from the Chang reference)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAY MICHAEL WHITE whose telephone number is (571) 272-7073. The examiner can normally be reached Mon-Fri 11:00-7:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.M.W./Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights