DETAILED ACTION
Claims 17-25 and 27-29 are currently presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/7/2026 has been entered.
Response to Arguments
Following Applicants amendments to the Drawings and Specification, the objections of the drawings are Withdrawn.
Following Applicants arguments and amendments, and in light of the 2019 Patent Eligibility guidance, the 101 rejection of the Claims is Withdrawn.
The claims are eligible under 101 as they now incorporate additional elements that cannot be done mentally, similar to McRO where the system uses a set of computer implemented rules to perform the steps of the claim.
Following Applicants arguments and amendments, the 103 rejection of the claims is Maintained.
Applicant’s Argument: Applicant’s arguments directed the 103 rejection are based on newly amended subject matter.
Examiner’s Response: All arguments are addressed in the 103 rejection of the claims below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 17-19, 23-24 and 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Ladha et al. USPPN 2018/0012125 in view of Chen et al. USPPN 2011/0218777.
Regarding claim 17, Ladha teaches determining a set of objects listed in a Building … for a building under construction for monitoring to determine status of the construction; ([0029], [0057], components or objects are listed so that they can be compared to what is seen for a status evaluation)
for each object in the set generating an object … comprising features characterizing the respective object; (Figures 1 and 2, [0039]-[0041], [0045], robots take sensor data and detect physical properties of the components in the building)
capturing an image of a region of the building using an image capture device (ICD); (Figures 1 and 2, [0026], [0039]-[0041], [0045], Lidar and other imaging systems are used to capture an image of the building)
processing the captured image to determine an image … comprising features characterizing the captured image; (Figures 1 and 2, [0026], [0039]-[0041], [0045], the image is process to determine features of the image)
selecting at least one detector from a plurality of machine-learning-based detectors stored in a pool of detectors, for processing the OFV and/or IFV based on features of the OFV and/or IFV, and according to a set of features for which the selected at least one detector is specialized, ([0056], a CNN is selected for object detection; [0073], A CNN is selected and trained to be specialized to recognize patterns; [0067] an RNN is selected for cost estimation (two NNs make a pool of NNs))
wherein each of the plurality of machine-learning-based detectors is trained to be specialized for processing the OFV and/or IFV for a different set of features of the OFV and/or IFV; ([0056], the NNs are trained prior to execution of their jobs)
processing the by the selected at least one detector O… and the I… to select the captured image for processing to detect presence of an image of the object in the captured image, and if present determine a location of the object in BIM coordinates as 3D BIM coordinates within the 3D environment of the model; and (Figure 2, [0056], [0064], components are found in the image; [0057], coordinate data is found for the object in 3D)
providing the status of the construction based on the detected presence and determined location. (Figure 2, [0064], [0080], construction status is reported by the system)
Ladha does not explicitly teach Building Information Model (BIM), wherein said BIM is a three-dimensional digital representation of the building under construction; object feature vector (OFV), image feature vector (IFV).
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
wherein said BIM is a three-dimensional digital representation of the building under construction; ([0003], [0005], [0068], A 3D BIM model is used for the building under construction)
object feature vector (OFV), ([0069], [0071]-[0074], an object is determined from vector images)
image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha with Chen as the references deal with classifying objects from an image, in order to implement a system that uses building information models as well as image feature vectors and object feature vectors. Chen would modify Ladha by using building information models as well as image feature vectors and object feature vectors. The benefit of doing so is the system can effectively and robustly retrieve similar objects from a drawing irrespective of their size or orientation. (Chen [0071])
Regarding claim 18, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches wherein the features of the O.. comprise metadata features based on metadata for the object in the B.., the 3D BIM coordinates of the object in the B.. 3D environment of the model, and/or features for characterizing the acquired image imaging the region of the building comprising the object. ([0044], features of the image that describe the building are shown)
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
object feature vector (OFV), ([0069], [0071]-[0074], an object is determined from vector images)
See motivation of claim 17
Regarding claim 19, the combination of Ladha and Chen teaches the limitation of claim 18. Ladha teaches wherein the metadata features comprise a name of the object, class of objects to which the object belongs, and/or an expected region of the building in which the object is expected to be located. ([0043], [0046]-[0057], the location of the objects is taken; [0060]-[0062] the location is compared to the expected location)
Regarding claim 23, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches wherein the I.. features comprise: a time and pose of the ICD at which the given image was captured, intrinsic features of the ICD, and/or image pixel coordinates for the object. ([0057], the coordinates intrinsic to the lidar point cloud are taken)
Chen teaches image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
See motivation of claim 17
Regarding claim 24, the combination of Ladha and Chen teaches the limitation of claim 23. Ladha teaches the I.. features comprise: 3D B.. coordinates withing the 3D environment of the model, for the object. ([0057], the coordinates intrinsic to the lidar point cloud are taken)
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
See motivation of claim 17
Regarding claim 27, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches providing a plurality of ICDs for capturing images of regions of the building; ([0045] Lidar and image sensor data is used)
providing a hub comprising a processor for processing images captured by the ICD; and (Figures 1 and 17, [0072]-[0075] a computer is used to process the image)
providing a communications network via which the ICDs communicate with the hub to transmit captured images to the hub. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
Regarding claim 28, the combination of Ladha and Chen teaches the limitation of claim 27. Ladha teaches providing the communications network with at least one transceiver operable to receive captured images from the ICD and forward the captured images to the hub. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
In regards to claim 29, it is the system embodiment of claim 17 with similar limitations to claim 17, and is such rejected using the same reasoning found in claim 17.
Ladha teaches a plurality of ICDs for capturing images of regions of a building under construction; ([0045] Lidar and image sensor data is used)
a hub comprising a BIM for the building; (Figures 1 and 17, [0072]-[0075] a computer is used to process the image)
a communications network via which the ICDs transmit images they capture to the hub; (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
wherein the hub comprises hardware and/or software for processing the captured images received from the plurality of ICDs in accordance with claim 17. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer which processes the images)
Claims 20, 22 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Ladha in view of Chen, and in further view of Zhang et al. USPPN 2019/0235083.
Regarding claim 20, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the OFV for characterizing the captured image comprise ranges for: distances at which the captured image is captured by the ICD and pose of the ICD.
Zhang teaches wherein the features of the OFV for characterizing the captured image comprise ranges for: distances at which the captured image is captured by the ICD and pose of the ICD. ([0101], [0106], [0148], the distance from the camera and other objects is captured; Figures 12 and 13b, [0077]-[0078], the pose of the image sensors is used)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha and Chen with Zhang as the references deal with capturing objects from an image, in order to implement a system that takes into account the pose and distance of the imaging device. Zhang would modify Ladha and Chen by taking into account the pose and distance of the imaging device. The benefit of doing so is the poses of the imaging devices can be arranged to maximize the field of view. (Zhang [0173])
Regarding claim 22, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the image in the OFV comprise a set of preferred poses for the ICD.
Zhang teaches wherein the features of the image in the OFV comprise a set of preferred poses for the ICD. ([0101], [0106], [0148], the distance from the camera and other objects is captured; Figures 12 and 13b, [0077]-[0078], the pose of the image sensors is used)
See motivation of claim 20
Regarding claim 25, the combination of Ladha and Chen teaches the limitation of claim 17. The combination of Ladha and Chen does not explicitly teach wherein the IFV features comprise brightness, contrast, and/or distance at which the captured image was captured.
Zhang teaches wherein the IFV features comprise brightness, contrast, and/or distance at which the captured image was captured. ([0101], [0106], [0148], the distance from the camera and other objects is captured)
See motivation of claim 20
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Ladha in view of Chen, and in further view of Jacobson et al. USPPN 2017/0315697.
Regarding claim 21, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the OFV for characterizing a captured image comprise ranges for: focus, brightness, and/or contrast characterizing the acquired image.
Jacobson teaches wherein the features of the OFV for characterizing a captured image comprise ranges for: focus, brightness, and/or contrast characterizing the acquired image. ([0453] the brightness range is set from light to dark)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha and Chen with Jacobson as the references deal with capturing objects from an image, in order to implement a system that has a brightness range. Jacobson would modify Ladha and Chen by taking into account the brightness of the image. The benefit of doing so is the system can determine if lights are on or off in the building as well as adjust the light attributes of the image. (Jacobson [0453])
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fard et al. “Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models”: Also uses image feature vectors and a BIM model to determine the construction status of a building.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL COCCHI whose telephone number is (469)295-9079. The examiner can normally be reached 7:15 am - 5:15 pm CT Monday - Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571-272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL EDWARD COCCHI/Primary Examiner, Art Unit 2188
Prosecution Insights