DETAILED ACTION
Claims 17-29 are currently presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application is acknowledged.
Information Disclosure Statement
The information disclosure statement (IDS) submitted has been considered by the Examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “46” has been used to designate both building site and second building site. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: 48. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: 56, 262B. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities: [0036] recites “in Fig.3 - Fig.5???” which is improper.
Appropriate correction is required.
Claim Objections
Claim 17 is objected to because of the following informalities: the claim recites “an image” multiple times when it is not the first recitation. Appropriate correction is required.
Claim 17 is objected to because of the following informalities: the claim recites “a region” when it is not the first recitation. Appropriate correction is required.
Claim 17 is objected to because of the following informalities: the claim recites “an image” when it is not the first recitation. Appropriate correction is required.
Claim 17 is objected to because of the following informalities: the claim recites “status” when it is not the first recitation. Appropriate correction is required.
Claim 17 is objected to because of the following informalities: the claim recites “determined location” when it is not the first recitation. Appropriate correction is required.
Claim 18 is objected to because of the following informalities: the claim recites “an acquired image” multiple times when it is not the first recitation. Appropriate correction is required.
Claim 18 is objected to because of the following informalities: the claim recites “a region” when it is not the first recitation. Appropriate correction is required.
Claim 19 is objected to because of the following informalities: the claim recites “a region” when it is not the first recitation. Appropriate correction is required.
Claim 21 is objected to because of the following informalities: the claim recites “a captured image” multiple times when it is not the first recitation. Appropriate correction is required.
Claim 29 is objected to because of the following informalities: the claim recites “a plurality of ICDs” when it is not the first recitation. Appropriate correction is required.
Claim 29 is objected to because of the following informalities: the claim recites “the ICDs” when each other recitation is “the plurality of ICDs”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 20 and 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “advantageous” in claims 20 and 21 is a relative term which renders the claim indefinite. The term “advantageous” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. As the “advantageous” range is not defined by the specification or the claims, the metes and bounds of the claim is rendered indefinite.
Examiner’s Note: For the purposes of examination “advantageous” will be interpreted as any value that results in a usable image.
All claims dependent on a 112 rejected base claim are rejected based on their dependency.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding claims 17-29, are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. abstract idea) without anything significantly more.
Step 1: Claims 17-28 are directed to a method, which is a process, which is a statutory category of invention. Claim 29 is directed to a system, which is a machine, which is a statutory category of invention. Therefore, claims 17-29 are directed to patent eligible categories of invention.
Step 2A, Prong 1: Claims 17 and 29 recite the abstract idea of determining the status of construction based on an image, constituting an abstract idea based on Mental Processes based on concepts performed in the human mind, or with the aid of pencil and paper. The limitation of "determining a set of objects listed in a Building Information Model (BIM) for a building under construction for monitoring to determine status of the construction;” covers mental processes including evaluating a dataset and judging what the status is. Additionally, the limitation of “for each object in the set generating an object feature vector (OFV) comprising features characterizing an image of a region of the building;” covers mental processes including evaluating an object and creating a vector that characterizes part of the building. Additionally, the limitation of “processing the captured image to determine an image feature vector (IFV) comprising features characterizing the captured image;” covers mental processes including evaluating an image and making a judgement on how to characterize it. Additionally, the limitation of “processing the OFV and the IFV to select the captured image for processing to detect presence of an image of the object in the captured image, and if present determine a location of the object in BIM coordinates; and” covers mental processes including evaluating an image and making a judgement on if an object is present and its coordinates. Additionally, the limitation of “providing status of the construction based on the detected presence and determined location.” covers mental processes including writing out the status based on evaluating the image to determine if an object is in the correct location. Thus, the claims recite the abstract idea of a mental process performed in the human mind, or with the aid of pencil and paper.
Dependent claims 18-28 further narrow the abstract ideas, identified in the independent claims.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application. In Claim 17 and 29 (as well as dependent claims, the additional element of “image capture device (ICD)”, as well as “hub comprising a processor” in claim 27 and similarly recited in 29, as well as “a communications network” in claims 27-29, as well as “at least one transceiver” in claim 28, merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)) The additional limitation of “capturing an image of a region of the building using an image capture device (ICD);” in claim 17, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to consulting and updating an activity log, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
Dependent claims 18-28 further narrow the abstract ideas, identified in the independent claims, and do not introduce further additional elements for consideration beyond those addressed above.
Step 2B: Claims 17 and 29 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. In Claim 17 and 29 (as well as dependent claims, the additional element of “image capture device (ICD)”, as well as “hub comprising a processor” in claim 27 and similarly recited in 29, as well as “a communications network” in claims 27-29, as well as “at least one transceiver” in claim 28, merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)) The additional limitation of “capturing an image of a region of the building using an image capture device (ICD);” in claim 17, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to amount to significantly more. This is akin to consulting and updating an activity log, which has been identified as extra solution activity. Therefore, the claim as a whole does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered alone or in combination, do not amount to significantly more than the judicial exception. As stated in Section I.B. of the December 16, 2014 101 Examination Guidelines, “[t]o be patent-eligible, a claim that is directed to a judicial exception must include additional features to ensure that the claim describes a process or product that applies the exception in a meaningful way, such that it is more than a drafting effort designed to monopolize the exception.”
The dependent claims include the same abstract ideas recited as recited in the independent claims, and merely incorporate additional details that narrow the abstract ideas and fail to add significantly more to the claims.
Dependent claim 18 is directed to further defining the object feature vector, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 19 is directed to further defining how the object is classified, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 20 is directed to further defining the object feature vector, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 21 is directed to further defining the object feature vector, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 22 is directed to further defining how the camera should be posed based on the features of an image, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 23 is directed to further defining how the camera should be posed based on the features of an image, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 24 is directed to further defining how the image has coordinates, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 25 is directed to further defining the features of the image, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Dependent claim 26 is directed to further defining how images are processed, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes.”
Accordingly, claims 17-29 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without anything significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 17-19, 23-24 and 26-29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ladha et al. USPPN 2018/0012125 in view of Chen et al. USPPN 2011/0218777.
Regarding claim 17, Ladha teaches determining a set of objects listed in a Building … for a building under construction for monitoring to determine status of the construction; ([0029], [0057], components or objects are listed so that they can be compared to what is seen for a status evaluation)
for each object in the set generating an object … comprising features characterizing an image of a region of the building; (Figures 1 and 2, [0039]-[0041], [0045], robots take sensor data and detect physical properties of the components in the building)
capturing an image of a region of the building using an image capture device (ICD); (Figures 1 and 2, [0026], [0039]-[0041], [0045], Lidar and other imaging systems are used to capture an image of the building)
processing the captured image to determine an image … comprising features characterizing the captured image; (Figures 1 and 2, [0026], [0039]-[0041], [0045], the image is process to determine features of the image)
processing the O… and the I… to select the captured image for processing to detect presence of an image of the object in the captured image, and if present determine a location of the object in BIM coordinates; and (Figure 2, [0056], [0064], components are found in the image; [0057], coordinate data is found for the object)
providing status of the construction based on the detected presence and determined location. (Figure 2, [0064], [0080], construction status is reported by the system)
Ladha does not explicitly teach Building Information Model (BIM), object feature vector (OFV), image feature vector (IFV).
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
object feature vector (OFV), ([0069], [0071]-[0074], an object is determined from vector images)
image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha with Chen as the references deal with classifying objects from an image, in order to implement a system that uses building information models as well as image feature vectors and object feature vectors. Chen would modify Ladha by using building information models as well as image feature vectors and object feature vectors. The benefit of doing so is the system can effectively and robustly retrieve similar objects from a drawing irrespective of their size or orientation. (Chen [0071])
Regarding claim 18, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches wherein the features of the O.. comprise metadata features based on metadata for the object in the B.., coordinates of the object in a B.. coordinate system, and/or features for characterizing an acquired image imaging a region of the building comprising the object. ([0044], features of the image that describe the building are shown)
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
object feature vector (OFV), ([0069], [0071]-[0074], an object is determined from vector images)
See motivation of claim 17
Regarding claim 19, the combination of Ladha and Chen teaches the limitation of claim 18. Ladha teaches wherein the metadata features comprise a name of the object, class of objects to which the object belongs, and/or a region of the building in which the object is expected to be located. ([0043], [0046]-[0057], the location of the objects is taken; [0060]-[0062] the location is compared to the expected location)
Regarding claim 23, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches wherein the I.. features comprise: a time and pose of the ICD at which the given image was captured, intrinsic features of the ICD, and/or image pixel coordinates for the object. ([0057], the coordinates intrinsic to the lidar point cloud are taken)
Chen teaches image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
See motivation of claim 17
Regarding claim 24, the combination of Ladha and Chen teaches the limitation of claim 23. Ladha teaches the I.. features comprise: B.. coordinates for the object. ([0057], the coordinates intrinsic to the lidar point cloud are taken)
Chen teaches Building Information Model (BIM), (Abstract, [0033]-[0034], the system extracts vector images and organizes them as a BIM)
image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
See motivation of claim 17
Regarding claim 26, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches providing a plurality of detectors, each specialized for processing the O.. and/or I.. for a different set of features of the O.. and/ or I..; and ([0042], [0045], multiple cad programs are processed, this includes Lidar and Image data)
selecting at least one detector of the plurality of detectors for processing the O.. and/or I.. responsive to features of the O.. and/or I.., and the set of features for which the detector is specialized. ([0073], A CNN is selected and trained to be specialized to recognize patterns)
Chen teaches object feature vector (OFV), ([0069], [0071]-[0074], an object is determined from vector images)
image feature vector (IFV) ([0069], [0071]-[0074], a vector image is taken)
See motivation of claim 17
Regarding claim 27, the combination of Ladha and Chen teaches the limitation of claim 17. Ladha teaches providing a plurality of ICDs for capturing images of regions of the building; ([0045] Lidar and image sensor data is used)
providing a hub comprising a processor for processing images captured by the ICD; and (Figures 1 and 17, [0072]-[0075] a computer is used to process the image)
providing a communications network via which the ICDs communicate with the hub to transmit captured images to the hub. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
Regarding claim 28, the combination of Ladha and Chen teaches the limitation of claim 27. Ladha teaches providing the communications network with at least one transceiver operable to receive captured images from the ICD and forward the captured images to the hub. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
In regards to claim 29, it is the system embodiment of claim 17 with similar limitations to claim 17, and is such rejected using the same reasoning found in claim 17.
Ladha teaches a plurality of ICDs for capturing images of regions of a building under construction; a plurality of ICDs for capturing images of regions of the building; ([0045] Lidar and image sensor data is used)
a hub comprising a BIM for the building; (Figures 1 and 17, [0072]-[0075] a computer is used to process the image)
a communications network via which the ICDs transmit images they capture to the hub; (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer)
wherein the hub comprises hardware and/or software for processing the captured images received from the ICDs in accordance with claim 17. (Figures 1 and 17, [0017], [0047] a robot is used to transmit the image to the computer which processes the images)
Claims 20, 22 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Ladha in view of Chen, and in further view of Zhang et al. USPPN 2019/0235083.
Regarding claim 20, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the OFV for characterizing the captured image comprise advantageous ranges for: distances at which the captured image is captured by the ICD and pose of the ICD.
Zhang teaches wherein the features of the OFV for characterizing the captured image comprise advantageous ranges for: distances at which the captured image is captured by the ICD and pose of the ICD. ([0101], [0106], [0148], the distance from the camera and other objects is captured; Figures 12 and 13b, [0077]-[0078], the pose of the image sensors is used)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha and Chen with Zhang as the references deal with capturing objects from an image, in order to implement a system that takes into account the pose and distance of the imaging device. Zhang would modify Ladha and Chen by taking into account the pose and distance of the imaging device. The benefit of doing so is the poses of the imaging devices can be arranged to maximize the field of view. (Zhang [0173])
Regarding claim 22, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the image in the OFV comprise a set of preferred poses for the ICD.
Zhang teaches wherein the features of the image in the OFV comprise a set of preferred poses for the ICD. ([0101], [0106], [0148], the distance from the camera and other objects is captured; Figures 12 and 13b, [0077]-[0078], the pose of the image sensors is used)
See motivation of claim 20
Regarding claim 25, the combination of Ladha and Chen teaches the limitation of claim 17. The combination of Ladha and Chen does not explicitly teach wherein the IFV features comprise brightness, contrast, and/or distance at which the captured image was captured.
Zhang teaches wherein the IFV features comprise brightness, contrast, and/or distance at which the captured image was captured. ([0101], [0106], [0148], the distance from the camera and other objects is captured)
See motivation of claim 20
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Ladha in view of Chen, and in further view of Jacobson et al. USPPN 2017/0315697.
Regarding claim 21, the combination of Ladha and Chen teaches the limitation of claim 18. The combination of Ladha and Chen does not explicitly teach wherein the features of the OFV for characterizing a captured image comprise advantageous ranges for: focus, brightness, and/or contrast characterizing the acquired image.
Jacobson teaches wherein the features of the OFV for characterizing a captured image comprise advantageous ranges for: focus, brightness, and/or contrast characterizing the acquired image. ([0453] the brightness range is set from light to dark)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ladha and Chen with Jacobson as the references deal with capturing objects from an image, in order to implement a system that has a brightness range. Jacobson would modify Ladha and Chen by taking into account the brightness of the image. The benefit of doing so is the system can determine if lights are on or off in the building as well as adjust the light attributes of the image. (Jacobson [0453])
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hoguet USPPN 2009/0160856: Also teaches the use of a BIM to classify objects in a building.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL COCCHI whose telephone number is (469)295-9079. The examiner can normally be reached 7:15 am - 5:15 pm CT Monday - Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571-272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL EDWARD COCCHI/Primary Examiner, Art Unit 2188