DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “image conversion device” in claims 1, 20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benesh et al (US 20190138667 A1) in view of Golparvar-Fard et al (US 20130155058 A1)
Regarding claim 1, Benesh discloses a construction compliance system ([0036] automated digital twin model of truth (DTMT) system, also known as a digital execution verification system), comprising:
an image conversion device configured to receive an image of a construction status of a construction project ([0041] DTMT system is operable to ingest (either periodically or continuously) reality data captured from the construction site, fabrication yard, manufacturing facility or any other location in the construction project supply chain.), and
to render a three-dimensional model of the construction status based on the image ([0038] Visualization of the digital twin overlaid on top of its corresponding CAD model can also be provided, as described in greater detail subsequently.),
a near edge processor configured to receive a three-dimensional model of a construction plan of the construction project ([0055] data points received (e.g., corresponding to plan data and/or reality data) can be tagged with additional metadata information in a metadata field such as the time of acquisition, geo localization data (i.e. longitude, latitude, and elevation), data accuracy, and audit data including information indicating the device that acquired the data, its accuracy, and the individual or data capture execution plan that requested the acquisition),
to compare the three-dimensional model of the construction status with the three-dimensional model of the construction plan to determine whether the construction status deviates from the construction plan ([0056] dense data capture can also facilitate the reporting of true construction progress and verify that the quality of execution is within tolerance), and
to provide an indication when the construction status deviates from the construction plan, wherein the near edge processor is located in a near edge of the construction compliance system that is remote from the construction project ([0067] This model would then be overlaid onto the Computer Aided Design (CAD)/Building Information Modelling (BIM) to identify any deviances and measure construction progress. This way, issues such as mismatches or construction inaccuracies may be addressed as they occur).
Golparvar-Fard discloses wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project ([0066] , the users may be allowed to select a set of corresponding control points from the as-built point cloud or the registered imagery and have those associated with the as-planned model, these points could be surveying control points or a set of points that represent the geospatial location of the site. In the case studies presented herein, these points are mostly chosen from corners of the foundation walls and columns as their detection and correspondence may be visually easier.)
Benesh and Golparvar-Fard are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify automated digital twin model of truth (DTMT) system of Benesh to include wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project as described by Golparvar-Fard.
The motivation for doing so would have been to generating as-built three-dimensional (3D) and four-dimensional (4D) (3D+time) point cloud models using low resolution images; generating an augmented reality environment by superimposing a building information model with the as-built point cloud model; and automated progress monitoring using the building information model integrated with a 3D or 4D point cloud model. (Golparvar-Fard, [0040]).
Therefore, it would have been obvious to combine Benesh and Golparvar-Fard to obtain the invention as specified in claim 1.
Regarding claim 2, Benesh discloses wherein the image conversion device includes a repository to store the three-dimensional model of the construction plan ([0060] , the interface may allow the user to identify objects within an image corresponding to components of the project (e.g. pipes, walls, etc.) and to map those objects to a corresponding counterpart within a CAD file).
Regarding claim 3, Benesh discloses wherein the image conversion device utilizes the three-dimensional model of the construction plan stored in the repository in rendering the three-dimensional model of the construction status ([0061] computer vision techniques can be applied to segment and classify objects within a point cloud gathered and link these objects to a corresponding CAD object in a CAD file., [0065] If the digital twin and the plan data do not match each other in the virtual world, then mismatches are automatically identified,).
Regarding claim 4, Benesh discloses a back end processor configured to receive the construction plan and to render the three-dimensional model of the construction plan based on the construction plan, wherein the back end processor is located in a datacenter of the construction compliance system ([0039] The DTMT system may alternately be hosted on a distributed or cloud-based environment located remotely at a data center (e.g. Google™ Cloud Services or Amazon™ Web Service))
Regarding claim 5, Benesh discloses wherein the back end processor further renders the three-dimensional model of the construction plan based on a regulation governing the construction plan ([0060] the interface may allow the user to identify objects within an image corresponding to components of the project (e.g. pipes, walls, etc.) and to map those objects to a corresponding counterpart within a CAD file.).
Regarding claim 6, Benesh discloses wherein the three-dimensional model of the construction plan includes a tolerance based on the regulation ([0060] Once this association is created, a tolerance threshold can be applied by the project manager or project controls team to identify all components which have been installed within tolerance, installed out of tolerance, or not installed).
Regarding claim 7, Benesh discloses wherein in the determination of whether the construction status deviates from the construction plan, the near edge processor is further configured to determine that the three-dimensional model of the construction status is outside of the tolerance ([0060] Once this association is created, a tolerance threshold can be applied by the project manager or project controls team to identify all components which have been installed within tolerance, installed out of tolerance, or not installed).
Regarding claim 8, Benesh discloses wherein in the determination of whether the construction status deviates from the construction plan, the near edge processor is further configured to determine that the three-dimensional model of the construction status violates a regulation governing the construction plan ([0060] if a re-installation is required to address the mismatch, the time required to conduct this re-installation should be considered and may affect the progress by causing a decrease in the percentage progress.).
Regarding claim 9, Benesh discloses wherein the indication identifies the regulation ([0065] digital twin can then be overlaid with the construction design as represented by the plan data (i.e., the plan model) and to digitally verify whether the real world construction matches the design)
Regarding claim 10, Benesh discloses wherein the image conversion device includes an image capture device configured to generate the image ([0046] 3D point cloud data acquired from a laser scanner [0047] camera photos acquired from a person, tripod, unmanned aerial vehicle (ie. drone) or unmanned ground vehicle. [0048] satellite imagery [0049] 360° images acquired from 360° cameras [0050] RFID tag data read by RFID readers).
Regarding claim 11, Benesh discloses a method ([0036] automated digital twin model of truth (DTMT) system, also known as a digital execution verification system), comprising:
providing, in a construction compliance system, an image conversion device ([0041] DTMT system is operable to ingest (either periodically or continuously) reality data captured from the construction site, fabrication yard, manufacturing facility or any other location in the construction project supply chain.),,
receiving, by the image conversion device, an image of a construction status of the construction project ([0056] a sparse data capture is carried out for the full facility (e.g. the entire construction site) to facilitate early stage planning and opportunity identification);
rendering, by the image conversion device, a three-dimensional model of the construction status based on the image ([0038] Visualization of the digital twin overlaid on top of its corresponding CAD model can also be provided, as described in greater detail subsequently.),;
providing, in the construction compliance system, a near edge processor, wherein the near edge processor is located in a near edge of the construction compliance system that is remote from the construction project ([0055] data points received (e.g., corresponding to plan data and/or reality data) can be tagged with additional metadata information in a metadata field such as the time of acquisition, geo localization data (i.e. longitude, latitude, and elevation), data accuracy, and audit data including information indicating the device that acquired the data, its accuracy, and the individual or data capture execution plan that requested the acquisition),;
receiving, by the near edge processor, a three-dimensional model of a construction plan of the construction project ([0059] point cloud data generated by a 3D scanner can be classified on the basis of matching the data point to an object in a design saved within a CAD file.);
comparing, by the near edge processor, the three-dimensional model of the construction status with the three-dimensional model of the construction plan to determine whether the construction status deviates from the construction plan ([0056] dense data capture can also facilitate the reporting of true construction progress and verify that the quality of execution is within tolerance), and; and
providing, by the near edge processor, an indication when the construction status deviates from the construction plan ([0067] This model would then be overlaid onto the Computer Aided Design (CAD)/Building Information Modelling (BIM) to identify any deviances and measure construction progress. This way, issues such as mismatches or construction inaccuracies may be addressed as they occur).
Golparvar-Fard discloses wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project ([0066] , the users may be allowed to select a set of corresponding control points from the as-built point cloud or the registered imagery and have those associated with the as-planned model, these points could be surveying control points or a set of points that represent the geospatial location of the site. In the case studies presented herein, these points are mostly chosen from corners of the foundation walls and columns as their detection and correspondence may be visually easier.)
Benesh and Golparvar-Fard are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify automated digital twin model of truth (DTMT) system of Benesh to include wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project as described by Golparvar-Fard.
The motivation for doing so would have been to generating as-built three-dimensional (3D) and four-dimensional (4D) (3D+time) point cloud models using low resolution images; generating an augmented reality environment by superimposing a building information model with the as-built point cloud model; and automated progress monitoring using the building information model integrated with a 3D or 4D point cloud model. (Golparvar-Fard, [0040]).
Therefore, it would have been obvious to combine Benesh and Golparvar-Fard to obtain the invention as specified in claim 11.
Regarding claim 12, Benesh discloses wherein the image conversion device includes a repository to store the three-dimensional model of the construction plan ([0060] , the interface may allow the user to identify objects within an image corresponding to components of the project (e.g. pipes, walls, etc.) and to map those objects to a corresponding counterpart within a CAD file).
Regarding claim 13, Benesh discloses wherein the image conversion device utilizes the three-dimensional model of the construction plan stored in the repository in rendering the three-dimensional model of the construction status ([0061] computer vision techniques can be applied to segment and classify objects within a point cloud gathered and link these objects to a corresponding CAD object in a CAD file., [0065] If the digital twin and the plan data do not match each other in the virtual world, then mismatches are automatically identified,).
Regarding claim 14, Benesh discloses providing, in the construction compliance system, a back end processor, wherein the back end processor is located in a datacenter of the construction compliance system; receiving, by the back end processor, the construction plan; and rendering, by the back end processor, the three-dimensional model of the construction plan based on the construction plan ([0039] The DTMT system may alternately be hosted on a distributed or cloud-based environment located remotely at a data center (e.g. Google™ Cloud Services or Amazon™ Web Service))
Regarding claim 15, Benesh discloses wherein the back end processor further renders the three-dimensional model of the construction plan based on a regulation governing the construction plan ([0060] the interface may allow the user to identify objects within an image corresponding to components of the project (e.g. pipes, walls, etc.) and to map those objects to a corresponding counterpart within a CAD file.).
Regarding claim 16, Benesh discloses wherein the three-dimensional model of the construction plan includes a tolerance based on the regulation ([0060] Once this association is created, a tolerance threshold can be applied by the project manager or project controls team to identify all components which have been installed within tolerance, installed out of tolerance, or not installed).
Regarding claim 17, Benesh discloses wherein in determining whether the construction status deviates from the construction plan, the method further comprises: determining, by the near edge processor, that the three-dimensional model of the construction status is outside of the tolerance ([0060] Once this association is created, a tolerance threshold can be applied by the project manager or project controls team to identify all components which have been installed within tolerance, installed out of tolerance, or not installed).
Regarding claim 18, Benesh discloses wherein in determining whether the construction status deviates from the construction plan, the method further comprises: determining, by the near edge processor, that the three-dimensional model of the construction status violates a regulation governing the construction plan ([0060] if a re-installation is required to address the mismatch, the time required to conduct this re-installation should be considered and may affect the progress by causing a decrease in the percentage progress.).
Regarding claim 19, Benesh discloses wherein the indication identifies the regulation ([0065] digital twin can then be overlaid with the construction design as represented by the plan data (i.e., the plan model) and to digitally verify whether the real world construction matches the design)
Regarding claim 20, Benesh discloses A construction compliance system ([0036] automated digital twin model of truth (DTMT) system, also known as a digital execution verification system), comprising:
an image conversion device including an image capture device configured to generate an image of a construction status of a construction project ([0041] DTMT system is operable to ingest (either periodically or continuously) reality data captured from the construction site, fabrication yard, manufacturing facility or any other location in the construction project supply chain.),
the image conversion device being configured to render a three-dimensional model of the construction status based on the image ([0038] Visualization of the digital twin overlaid on top of its corresponding CAD model can also be provided, as described in greater detail subsequently.),
a near edge processor configured to receive a three-dimensional model of a construction plan of the construction project ([0055] data points received (e.g., corresponding to plan data and/or reality data) can be tagged with additional metadata information in a metadata field such as the time of acquisition, geo localization data (i.e. longitude, latitude, and elevation), data accuracy, and audit data including information indicating the device that acquired the data, its accuracy, and the individual or data capture execution plan that requested the acquisition), and
to compare the three-dimensional model of the construction status with the three-dimensional model of the construction plan to determine whether the construction status deviates from the construction plan ([0056] dense data capture can also facilitate the reporting of true construction progress and verify that the quality of execution is within tolerance), and
to provide an indication when the construction status deviates from the construction plan, wherein the near edge processor is located in a near edge of the construction compliance system that is remote from the construction project ([0067] This model would then be overlaid onto the Computer Aided Design (CAD)/Building Information Modelling (BIM) to identify any deviances and measure construction progress. This way, issues such as mismatches or construction inaccuracies may be addressed as they occur).
a back end processor configured to receive the construction plan and to render the three-dimensional model of the construction plan based on the construction plan, wherein the back end processor is located in a datacenter of the construction compliance system ([0039] The DTMT system may alternately be hosted on a distributed or cloud-based environment located remotely at a data center (e.g. Google™ Cloud Services or Amazon™ Web Service)).
Golparvar-Fard discloses wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project ([0066] , the users may be allowed to select a set of corresponding control points from the as-built point cloud or the registered imagery and have those associated with the as-planned model, these points could be surveying control points or a set of points that represent the geospatial location of the site. In the case studies presented herein, these points are mostly chosen from corners of the foundation walls and columns as their detection and correspondence may be visually easier.)
Benesh and Golparvar-Fard are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify automated digital twin model of truth (DTMT) system of Benesh to include wherein the image conversion device is located in a far edge of the construction compliance system that is proximate to the construction project as described by Golparvar-Fard.
The motivation for doing so would have been to generating as-built three-dimensional (3D) and four-dimensional (4D) (3D+time) point cloud models using low resolution images; generating an augmented reality environment by superimposing a building information model with the as-built point cloud model; and automated progress monitoring using the building information model integrated with a 3D or 4D point cloud model. (Golparvar-Fard, [0040]).
Therefore, it would have been obvious to combine Benesh and Golparvar-Fard to obtain the invention as specified in claim 20.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9-5am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615