DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on August 6th, 2025 and October 6th, 2025 was reviewed and the listed references were noted.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “138” has been used to designate both first tower post 136 and second tower post 138. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: 136. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to because of the following informalities:
In Figure 4,
Camera 192 should read First Camera 192
Camera 193 should read Second Camera 193
Laser Emitter 194 should read First Laser Emitter 194
Laser Emitter 195 should read Second Laser Emitter 195
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In Paragraph [0040], add a parenthesis ( ) ) after noise
In Paragraph [0068], “the optical fiducials 170, 172” should read “the first and second optical fiducials 170, 172” for clarity
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 8, 9, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shi (CN 104976968) in view of Mai (CN 116182708 A).
Regarding Claim 1, Shi teaches “A method, comprising:” (Shi, Paragraph [0011], discloses: “the present invention provides a three-dimensional geometric measurement method based on LED tag tracking”); “translating a 3D scanner relative to a target object, the 3D scanner is arranged in an inspection cart” (Shi, Paragraph [0016], discloses: “the bracket is moved along the track to push the three dimensional scanner to a new position, and the third and fourth steps are repeated to complete the next partial measurement of the object to be tested”); “scanning, as the 3D scanner is translated, the target object using the 3D scanner to capture images of at least a slice of the target object, wherein optical fiducials integrated into the inspection cart are arranged in a field of view of the 3D scanner during the scanning” (Shi, Paragraph [0014], discloses: “a local measurement is started, the projector of the three-dimensional scanner projects structured light onto the surface of the object to be tested, and the two industrial cameras of the three-dimensional scanner capture partial images of the object to be tested in the respective fields of view, and The partial image data is transmitted to the server, and at the same time, the stereo tracker takes an image of the LED tag and transmits it to the server;”; Paragraph [0012], discloses: “a set of three-dimensional scanners with LED tags attached are mounted on a bracket that can move along a track beside the object to be tested, and the bracket is moved to one end of the track”); “stitching together the captured images of the target object obtained during the scanning to create a slice image, the captured images being stitched together according to position targets provided by the optical fiducials” (Shi, Paragraphs [0014]-[0020] discloses the following method:
PNG
media_image1.png
659
758
media_image1.png
Greyscale
); (Mai, Paragraph [0002]) that uses “AI to find and locate the defect position of the 2D color image, calculate the height difference on the point cloud data at the same position, and confirm the depth information of cracks and pits” (Mai, Paragraph [0114]). Therefore, it would be obvious to one of ordinary skill of the art before the effective filing date of the claimed invention to combine the image acquisition and stitching method from Shi and defect detection method found in Mai to obtain the same method that is described in Claim 1.
Regarding Claim 8, the combination of Shi and Mai discloses “The method of claim 1, wherein the slice of the target object is a first slice; and wherein the method further comprises:” (Shi, Paragraph [0015], discloses: “the server generates a partial three-dimensional image of the object to be tested according to partial image data of the two industrial cameras, and accurately positions the center point of the LED tag in the LED tag image to complete the a partial measurement of the object to be tested”); “a) moving the inspection cart so that the inspection cart aligns with a subsequent slice of the target object” (Mai, Paragraph [0075], discloses: “Move the track along the circumference of the storage tank to ensure that there is an overlapping area of width w between two adjacent scans of the crawler. If the crawler is out of the tracking camera's field of view, move the lifter accordingly.”); “b) translating, with the inspection cart aligned with the subsequent slice, the 3D scanner relative to the target object” (Shi, Paragraph [0016], discloses: “the bracket is moved along the track to push the three dimensional scanner to a new position”); “c) scanning, as the 3D scanner is translated with the inspection cart aligned with the subsequent slice, the target object using the 3D scanner to capture images of the subsequent slice of the target object, wherein the optical fiducials are arranged in the field of view of the 3D scanner during the scanning of the subsequent slice” (Mai, Paragraph [0085], discloses: “After obtaining the 2D image and point cloud information of an area, the crawler moves to the next area according to a fixed step, and obtains the 2D image and point cloud information of the next area in the same way.”; Shi, Paragraph [0015], discloses: “the server generates a partial three-dimensional image of the object to be tested according to partial image data of the two industrial cameras, and accurately positions the center point of the LED tag in the LED tag image to complete the a partial measurement of the object to be tested;”); “d) stitching together captured images of the subsequent slice of the target object to create a subsequent slice image, the captured images of the subsequent slice being stitched together according to the position targets provided by the optical fiducials” (Shi, Paragraphs [0014]-[0020], please refer to the above-described analysis for Claim 1); and “e) detecting one or more defects on the subsequent slice of the target object based on the subsequent slice image.” (Mai, Paragraph [0114], please refer the above-described analysis for Claim 1). Therefore, it would be obvious for one of ordinary skill of the art before the effective filing date of the claimed invention to use the known image acquisition and stitching techniques seen in the combination of Shi and Mai to improve the inspection system in the same way. By using the image acquisition and stitching techniques described in the combination of Shi and Mai, one of ordinary skill of the art could easily form slice images from different parts of the airplane to detect defects using various algorithms executed by the processors within the inspection cart. Thus, it would be obvious for one of ordinary skill in the art to use the combination of Shi and Mai to obtain the method described in Claim 8.
Regarding Claim 9, the combination of Shi and Mai discloses “The method of claim 8, further comprising:” (Shi, Paragraphs [0014]-[0020]; Mai, Paragraphs [0075], [0085], [0114]; please refer to the above-described analysis for Claim 8); “iterating a) through e) for a predetermined length of the target object; and stitching the slice image of the first slice and subsequent slice image of each one of the subsequent slices into a combined slice image.” (Shi, Paragraphs [0014]-[0020], disclosed in the above-described analysis for Claim 1, describe the steps to the 3D image acquisition and stitching process; Paragraph [0021] additionally discloses: “In the tenth step, steps 5 to 9 are repeated until the last partial measurement of the object to be tested ends to splicing a three-dimensional image of the entire surface of the object to be tested.”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the iteration of the image acquisition techniques seen in the combination of Shi and Mai to obtain the combined slice image described in Claim 9. By implementing these techniques within the inspection cart, the inspection cart is able to efficiently traverse and acquire the image data necessary to cover the length of one side of an entire airplane fuselage or other large-scale object. Therefore, it would be obvious for one of ordinary skill in the art to use the combination of Shi and Mai to obtain the same method described in Claim 9.
Regarding Claim 15, Shi discloses “An inspection system, comprising:” (Shi, Paragraph [0010], discloses: “The object of the present invention is to provide a three dimensional measurement method and system for large-scale objects based on LED tag tracking, which can conveniently, quickly and accurately perform visual measurement on large-scale objects, and at the same time eliminate the adverse effect of natural light intensity changes on measurement accuracy.”); (Shi, Paragraph [0012], discloses: “In the first step, a set of three dimensional scanners with LED tags attached are mounted on a bracket that can move along a track beside the object to be tested, and the bracket is moved to one end of the track, the three dimensional scanner Mainly composed of two industrial cameras and one projector”). Shi does not explicitly disclose “an inspection cart, comprising:” or “a tower supporting a track”. However, in an analogous field of endeavor, Mai discloses a “storage tank three-dimensional information reconstruction and defect detection system, which includes a movable crawler track and a lifter capable of circular motion along the storage tank, the shape of the movable crawler track is consistent with the outer contour of the storage tank or The inner contours are similar, and the movable crawler track is equipped with a crawler that can move up and down along the movable crawler track.” (Mai, Paragraph [0009]). Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the two cameras, projector, and optical fiducials (LED Tags) seen in Shi with the crawler tower robot supporting a track seen in Mai to obtain the same system described in Claim 15. By combining the elements seen in both of these references, the inspection cart will be able to fully achieve the necessary methods to inspect the fuselage and other large-scale objects. Thus, it would be obvious for one of ordinary skill in the art to combine both Shi and Mai to obtain the system of Claim 15.
Claim 20 recites an apparatus (inspection cart) with features corresponding to the elements of the system recited in Claim 15. Therefore, the recited features of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Shi and Mai references, presented in rejection of Claim 15, apply to this claim.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Coffman (US 2023/0186563).
Regarding Claim 2, the combination of Shi and Mai discloses “The method of claim 1, further comprising:” (Shi, Paragraphs [0011], [0012], and [0014]-[0020]; Mai, Paragraphs [0002] and [0114]; please refer to the above-described analysis for Claim 1). The combination of Shi and Mai is not relied on to disclose “projecting, by the 3D scanner, one or more defect indicators onto the target object to indicate respective ones of the one or more defects detected on the target object”. However, in an analogous field of endeavor, Coffman discloses a method that includes: “receiving images acquired from a number of viewpoints of different sections of the vehicle, and performing photogrammetry on the images to extract a profile of the vehicle. The method includes creating a wireframe mesh or point cloud from the profile, and generating the 3D model of the vehicle. The images are processed to determine areas on a surface of the vehicle in which a defect is detected, and markers are appended onto respective areas of the 3D model that correspond to the areas on the surface of the vehicle such that the defect is appended onto the 3D model. And the method includes generating a display of the 3D model of the vehicle including the markers that indicate the areas on the surface of the vehicle in which the defect is detected.” (Coffman, Abstract). Although the defect indicators described in Coffman are not projected onto the vehicle directly, a 3D scanner was used to obtain the 3D model of the vehicle to allow the defects to be indicated and projected onto the 3D representation for an end user to observe in real-time, which is closely related to what is being done in Claim 2. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the image acquisition and stitching methods described in the combination of Shi and Mai with the method of projecting defect indicator onto a target object seen in Coffman to yield predictable results for Claim 2.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Coffman, and further in view of Georgeson 1 (US 9,448,758).
Regarding Claim 3, the combination of Shi, Mai, and Coffman discloses “The method of Claim 2” (Shi, Paragraphs [0011], [0016], [0012], and [0014]-[0020]; Mai, Paragraphs [0002] and [0114]; Coffman, Abstract; please refer to the above described analysis for Claim 3). The combination of Shi, Mai, and Coffman is not relied on to disclose “wherein at least one defect indicator of the one or more defect indicators is projected onto the target object outlining a defect of the one or more defects detected on the target object”. However, in an analogous field of endeavor, Georgeson 1 discloses the following: “system for providing location specific maintenance history utilize a handheld device incorporating a camera, a projector and a communications interface. A microcontroller is connected to the communications interface and interconnected to the camera to receive image information from a region on an object within the FOV of the camera. The microcontroller is further interconnected to the projector to transmit repair history information for projection onto the object. A remote computer system incorporates a CAD database and a maintenance database with repair history and a mating communications interface. Software modules present on one or both of the microcontroller and remote computer system are employed for identifying markers on a structure present in the image information. The software modules then determine an absolute reference position for the handheld device to determine coordinates in the CAD database corresponding to the FOV and repair history information from the maintenance database is retrieved corresponding to determined coordinates. The repair history information is communicated to the microcontroller for transmission to the projector for projection onto the surface of the object at the defined coordinate location.” (Georgeson 1, Col 2, Lines 3-25). Georgeson 1 also discloses a pictorial view of a user projecting and outlining a previous defect onto the fuselage in Figure 4, where “information regarding a repair is projected showing the location and extent of the repair 42 and a data block 44 regarding the maintenance performed as a puncture repair” (Georgeson 1, Col 5, lines 5-8; Figure 4).
PNG
media_image2.png
448
665
media_image2.png
Greyscale
Therefore, it would be obvious to combine the method of indicating defects seen in the combination of Shi, Mai, and Coffman with the outlining of the defects shown in Georgeson 1 to enable a user of the inspection cart to adequately assess the defects of the large-scale object. By implementing the outlining methods described in Georgeson 1, the inspection cart can intuitively inform a user of specific defects that may be present within the large-scale object they are inspecting at the present moment. Additionally, the combination of Shi, Mai, and Coffman offer the ability to pinpoint where exactly the defect is on within the slice image that has been acquired. Thus, it would be obvious to combine Shi, Mai, Coffman, and Georgeson 1 to obtain the method described in Claim 3.
Claims 4 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Brajovic (US 12,409,952).
Regarding Claim 4, the combination of Shi and Mai discloses “The method of claim 1” (Shi, Paragraphs [0011], [0012], and [0014]-[0020]; Mai, Paragraphs [0002] and [0114]; please refer to the above-described analysis for Claim 1); “wherein the 3D scanner is translated along a linear path” (Shi, Paragraph [0071] and Figure 1 (see below), discloses: “the present invention provides a three-dimensional geometric measurement system based on LED tag tracking, comprising a high performance server 40 , a set of stereo tracker 30 and a set of three-dimensional scanners with high brightness LED tags 23 . 20. Wherein, the three dimensional scanner 20 is mounted on a bracket 12 that can move along the track 11 on the side of the object 10 to be tested, and mainly has two industrial cameras 22 capable of capturing a partial image of the object 10 to be tested and one capable of A projector 21 that projects structured light onto the surface of the object 10 to be tested is composed.”)
PNG
media_image3.png
668
739
media_image3.png
Greyscale
100 including a vehicle 102 configured to autonomously navigate and inspect an object 110” (Brajovic, Col 2, lines 57-59), whereas “The inspection route 114 can include instructions to capture images of certain features of the object 110 in the 3D (CAD) model from certain directions. For example, FIG. 5 depicts a non-limiting aspect wherein the object 110 is an aircraft fuselage to be inspected and the inspection route 114 instructs the vehicle 102 to capture an image of a specific location L on the fuselage from a predetermined distance D (as a non-limiting example, 1.3 to 2.3 m) at a predetermined angle θ relative to the aircraft. The specific location L on the aircraft 110 can be specified relative to the 3D CAD model of the aircraft 110” (Brajovic, Col 5, lines 10-22, Figure 5 (see below)).
PNG
media_image4.png
460
516
media_image4.png
Greyscale
Accordingly, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the 3D scanner along a linear track seen in the combination of Mai and Shi with the angular position of the vehicle seen in Brajovic to effectively traverse through the entire slice of an object. By designing the 3D scanner to be movable along a track as shown in the combination of Mai and Shi, the cart is able to effectively translate to get appropriate slice images. This design choice combined with the 3D scanner of the cart being at an angle with respect to the vertical direction seen in Brajovic allow the inspection cart to cover the whole vertical area of a respective slice of the target object (i.e., fuselage or another large-scale object). Therefore, it would be obvious for one of ordinary skill in the art to combine the methods seen in Shi, Mai, and Brajovic to obtain the same method described in Claim 4.
Regarding Claim 12, the combination of Shi and Mai teach “The method of claim 8” (Shi, Paragraphs [0014]-[0020]; Mai, Paragraphs [0085] and [0114]; please refer to the above-described analysis for Claim 8); 100 including a vehicle 102 configured to autonomously navigate and inspect an object 110” (Brajovic, Col 2, lines 57-59), whereas “The inspection route 114 can include instructions to capture images of certain features of the object 110 in the 3D (CAD) model from certain directions. For example, FIG. 5 depicts a non-limiting aspect wherein the object 110 is an aircraft fuselage to be inspected and the inspection route 114 instructs the vehicle 102 to capture an image of a specific location L on the fuselage from a predetermined distance D (as a non-limiting example, 1.3 to 2.3 m) at a predetermined angle θ relative to the aircraft. The specific location L on the aircraft 110 can be specified relative to the 3D CAD model of the aircraft 110” (Brajovic, Col 5, lines 10-22, Figure 5 (see above from the analysis for Claim 4)). Therefore, it would be obvious for one of ordinary skill of the art before the effective filing date of the claimed invention to combine the image acquisition methods of the first slice seen in the combination of Shi and Mai with the autonomous movement of a vehicle found in Brajovic to enable the inspection cart to autonomously move around the fuselage. By allowing the inspection cart to be an autonomous vehicle, one of ordinary skill in the art could effectively allow the inspection cart to perform its inspection of the outer surface of the aircraft without need for human supervision all the time. Therefore, it would be obvious for one of ordinary skill in the art to combine Mai, Shi, and Brajovic to achieve the same method described in Claim 12.
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Brajovic, and further in view of Clark (US 6,600,999).
Regarding Claim 5, the combination of Shi and Mai discloses “The method of claim 4, wherein the optical fiducials include” (Shi, Paragraphs [0014]-[0020] and [0071] as well as Figure 1; Mai, Paragraphs [0002] and [0114]; Brajovic, Col 2, lines 57-59, Col 5, lines 10-22, and Figure 5; please refer to the above-described analysis for Claim 4). The combination of Shi and Mai is not relied on to disclose “a first fiducial rail and a second fiducial rail that are spaced from one another and arranged parallel to the linear path.”. However, in an analogous field of endeavor, Clark discloses in Figure 11 of their invention that a “detector system 104 includes a detector carriage 110, which carries a magnetic induction sensor system 130 and, optionally, an ultrasonic sensor system 160. FIGS. 11-13 illustrate a detector carriage 110 according to an embodiment of the invention. The detector carriage 110 includes a frame 111 having a left side frame rail 112 and a right side frame rail 115. The left side frame rail 112 is formed from a left outside channel 113 and a left inside channel 114 joined by a forward end plate 118 and a rearward end plate 119.” (Clark, Col 6, Lines 66-67 and Col 7, lines 1-13)
PNG
media_image5.png
561
796
media_image5.png
Greyscale
Therefore, it would be obvious for one of ordinary skill in the art to combine the optical fiducials seen on the inspection cart in the combination of Shi, Mai, and Brajovic with the left side frame rail and right side frame rail seen in Clark to achieve the same inspection cart described in Claim 5.
Regarding Claim 6, the combination of Shi, Mai, Brajovic and Clark disclose “The method of claim 5” (Shi, Paragraphs [0011], [0012], [0014]-[0020], and [0071] as well as Figure 1; Mai, Paragraphs [0002] and [0114]; Brajovic, Col 2, lines 57-59, Col 5, lines 10-22, and Figure 5; Clark, Col 6, Lines 66-67, Col 7, Lines 1-8, and Figure 11; please refer to the above-described analysis for Claim 5); wherein “the position targets are arranged on first fingers that extend from the first fiducial rail and on second fingers that extend from the second fiducial rail, the first fingers are spaced from one another along the first fiducial rail and the second fingers are spaced from one another along the second fiducial rail” (Clark, Col 7, Lines 13-19 and Figure 11 (see above) discloses: “The channels 115 and 116 are also spaced slightly apart and configured for suspension of sensing equipment from attachment brackets bolted thereto. A clevis 126 is attached to the upper side of each frame rail 112, 115 and extends upward therefrom. The devises 126 are positioned near the center of the rail frames 112 and are configured for attachment of a tow bar for towing of the detector carriage 110”; Col 8, lines 66-67, Col 9, Lines 1-10, discloses: “With reference to FIGS. 11-13, the detector system 104 includes a magnetic induction sensor system 130 that is attached to the detector carriage 110. The magnetic inductor sensor system 130 includes a left magnetic induction sensor set 131 for inspection of one rail (left rail) of a track and a right magnetic induction sensor set 132 for inspection of the other rail (right rail). Each induction sensor set 131, 132 includes a pair of brush assemblies 140 and an induction sensor unit (ISU) 150. The brush assemblies 140 are used to saturate the railhead with current, thus establishing a magnetic field around the rail. The ISU 150 is used to detect irregularities in the magnetic field caused by defects within the rail.”). Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to perform a simple substitution of the magnetic and ultrasonic sensors placed along the left and right rails of the detector system seen in the combination of Shi, Mai, Brajovic, and Clark with the position targets seen on the first and second fiducial rails to achieve the same inspection cart described in Claim 6. By doing this, one of ordinary skill in the art can effectively align the optical fiducials needed as the 3D scanner acquires a slice image for the fuselage. Thus, it would be obvious to swap the sensors in the detector system seen in the combination of Shi, Mai, Brajovic, and Clark with the position targets on the first and second fiducial rails to achieve the same method described in Claim 6.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Zylka (US 6379043 B1).
Regarding Claim 7, the combination of Shi and Mai disclose “The method of claim 1, further comprising:” (Shi, Paragraphs [0011], [0012], [0014]-[0020]; Mai, Paragraph [0114]). The combination of Shi and Mai is not relied on to disclose “masking the optical fiducials in the slice image so that the optical fiducials do not appear on the slice image.” However, in an analogous field of endeavor, Zylka discloses that for X-ray imaging, “it has been found that from a construction point of view it is advantageous to mount calibration phantoms, provided with reference markers, directly on the patient or on the operating table. The position of the reference markers can be localized by the position measuring device. In the different orientations of the X-ray examination apparatus, it may occur that some reference markers are masked, so that they cannot be localized by the position measuring device. In that case it is advantageous to take recourse to calibration members or phantoms whose reference markers can be localized by the position measuring device.” (Col 3, lines 15-25). Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the known technique of masking optical fiducials (such as reference markers) seen in Zylka with the image acquisition methods seen in the combination of Shi and Mai to improve the capabilities of the inspection cart. By implementing this method into the slice image acquisition of the inspection cart, one of ordinary skill in the art can be able to see the full slice image without having to see any obstructions that could interfere with the thorough examination of the airplane fuselage. Thus, it would be obvious to combine the Shi, Mai, and Zylka references to achieve the method seen in Claim 7.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Troy (US 2021/0090269).
Regarding Claim 10, the combination of Shi and Mai discloses “The method of claim 9, wherein” (Shi, Paragraphs [0014]-[0021]; Mai, Paragraphs [0075], [0085], [0114]; please refer to the above-described analysis for Claim 9); and “the combined slice image” (Shi, Paragraphs [0014]-[0020], disclosed in the above-described analysis for Claim 1, describe the steps to the 3D image acquisition and stitching process; Paragraph [0021] additionally discloses: “In the tenth step, steps 5 to 9 are repeated until the last partial measurement of the object to be tested ends to splicing a three-dimensional image of the entire surface of the object to be tested.”) Figure 8 of a “flowchart identifying steps of a method 130 for computing a current absolute location of a 1-D NDI scanner, defined in a coordinate system of the target object, in accordance with one embodiment. During set-up of the system, the 1-D NDI scanner (including a 1-D scanner array and associated circuitry) is placed on the surface of the target object at a known physical location (also referred to herein as initial “absolute location”) that is defined in the coordinate system of the target object (step 132). The 1-D NDI scanner is then translated across the surface of the target object in an X direction at a known speed from a first X position to second, third and fourth X positions in succession (step 134). Successive sets of sensor data are acquired at a known capture rate as the scanner translates in the X direction (step 136). The successive sets of sensor data are converted to respective scan strips of scan image data (step 138). An image processor then constructs a pair of scan images by aggregating scan strips. More specifically, a first scan image is constructed from a first sequence of scan strips converted from sensor data acquired during movement of the 1-D sensor array from the first X position to the third X position (step 140); and a second scan image is constructed from a second sequence of scan strips converted from sensor data acquired during movement of the 1-D sensor array from the second X position to the fourth X position (step 142). The image processor (more specifically, the image processing and feature point comparison module 24) then finds feature points in the first and second scan images (step 144) and determines which feature points found in step 144 are common feature points in the first and second scan images (step 146).” (Troy, Paragraph [0080], Figure 8 (see below)).
PNG
media_image6.png
700
450
media_image6.png
Greyscale
Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method for obtaining first and second scan images based on different positions on the target object seen in Troy with the combined slice image acquisition methods seen in the combination of Shi and Mai to achieve the same method described in Claim 10. Doing this would allow the inspection cart to cover the entire perimeter of the fuselage or other large-scale object without falling short of the desired outcome of observing defects in real time. This also ensures that repairmen and engineers can work simultaneously to fix defects once the cart performs its scan of the entire fuselage. Therefore, it would be obvious to combine the Shi, Mai, and Troy references to perform the method described in Claim 10.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Troy, and further in view of Coffman.
Regarding Claim 11, the combination of Shi, Mai, and Troy disclose “The method of claim 10, further comprising” (Shi, Paragraphs [0014]-[0021]; Mai, Paragraphs [0075], [0085], [0114]; Troy, Paragraph [0080], Figure 8 (see above); please see the above-described analysis for Claim 10); The combination of Shi, Mai, and Troy does not explicitly disclose “creating a digital twin of the target object based at least in part on the first combined slice image and the second combined slice image.” However, in an analogous field of endeavor, Coffman discloses “A method is provided for generating a three-dimensional (3D) visually representative model of a vehicle. The method includes receiving images acquired from a number of viewpoints of different sections of the vehicle, and performing photogrammetry on the images to extract a profile of the vehicle. The method includes creating a wireframe mesh or point cloud from the profile, and generating the 3D model of the vehicle. The images are processed to determine areas on a surface of the vehicle in which a defect is detected” (Coffman, Abstract). Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method of creating a digital twin seen in Coffman with the method for acquiring multiple combined slice images seen in the combination of Shi, Mai, and Troy to achieve the same method described in Claim 11.
Claims 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Brügge (Application of 3D‑scanning for structural and geometric assessment of aerospace structures).
Regarding both Claims 13 and 14, the combination of Shi and Mai discloses “The method of claim 1” (Shi, Paragraphs [0011], [0012], [0014]-[0020]; Mai, Paragraph [0114]). The combination of Shi and Mai is not relied on to disclose for both claims “wherein the target object is a fuselage”, “wherein the fuselage is scanned so that the captured images of the slice extend from a midline of an underbelly of the fuselage to a window line of the fuselage.” and “wherein the fuselage is scanned so that the captured images of the slice extend from a midline of an underbelly of the fuselage to a midline of a top of the fuselage.”. However, in an analogous field of endeavor, Brugge discloses of different applications of using 3D scanning technology to assess “the geometric characterization of aerodynamic surfaces of UAVs and propellers. The extracted geometric characteristics, such as wing positions, airfoils and aerodynamic angles, proved suitable for usage in low-fidelity aerodynamic simulation.” (Brügge, Pg. 2, Col. 2). In one application for a scan of the surface of an aircraft, “A hand-held laser 3D-scanner of type CREAFORM HandySCAN 700 [17] in combination with an optical coordinate measurement system (photogrammetric triangulation) of type CREAFORM MaxSHOT 3D [18] is utilized for the collection of surface data. The system utilizes a static coordinate system generated by circular target points that are applied either on the rigid object or surrounding surfaces. These points are captured by the coordinate measurement system with a photo camera from different angles. The 3D-scanner employs these points to determine its orientation in space and generates a surface point cloud with the help of reflecting laser beams. At least six points have to be in the field of view and in range (approx. 300 mm) of the 3D-scanner at a time for it to be able to orientate itself in space.” (Brügge, Pg. 3, Col. 2). Figure 2 discloses the scan environment for large objects.
PNG
media_image7.png
473
742
media_image7.png
Greyscale
To perform a scan, “The scanning process starts with the creation of a reference system where the targets are registered separately. The global reference targets are photographed using the MaxSHOT 3D for big objects as a first step. Standard scanning targets are captured directly by the HandySCAN 700. A specific part of the object is chosen as the origin of the coordinate system and the scan aligns accordingly. The scanning of the object can be started by scanning in a star-like fashion from the center to the outer edges of the object after the coordinate measurement system is resolved successfully. Challenges arise for the transition from the upper to the lower surface of thin surfaces such as wings or propellers via the leading or trailing edge. The placement of small tetrahedrons fitted with scanning targets on the upper side of a wing helps this transition. The scanner provides the ability to merge multiple scan partitions from the same object if sufficient overlay of targets or surface geometry is present. This will, however introduce a further source of error as the merging process will lead to uncertainties at the edges.” (Brugge, pg. 4, Col 1-2). Later on in the article, Brugge discloses the following in Page 5, Columns 1 and 2 and Figure 4:
PNG
media_image8.png
615
461
media_image8.png
Greyscale
PNG
media_image9.png
287
460
media_image9.png
Greyscale
As shown in Figures 2 and 4, both the MaxSHOT 3D and HandyMan700 have been able to successfully take enough slice images of the midline of an underbelly to the window line of the fuselage as well as the midline of an underbelly to the top of the fuselage in order to create the full 3D image of the airplane to be able to analyze it further. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the slice image acquisition methods seen in the combination of Shi and Mai with the method for capturing images from the midline of an underbelly of a fuselage to the top of the fuselage and midline of an underbelly of a fuselage to the window of the fuselage to achieve the same methods described in Claims 13 and 14.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Pollard (WO 2016209276 A1).
Regarding Claim 16, the combination of Mai and Shi discloses “The inspection system of claim 15, further comprising” (Shi, Paragraphs [0010] and [0012]; Mai, Paragraph [0009]; please see above-described analysis for Claim 15); (Shi, Paragraph [0014], discloses “In the third step, a local measurement is started, the projector of the three-dimensional scanner projects structured light onto the surface of the object to be tested, and the two industrial cameras of the three-dimensional scanner capture partial images of the object to be tested in the respective fields of view”; Shi, Paragraph [0016], discloses: “In the fifth step, the bracket is moved along the track to push the three dimensional scanner to a new position, and the third and fourth steps are repeated to complete the next partial measurement of the object to be tested”); “stitching together the captured images of the slice of the target object to create a slice image, the captured images being stitched together according to position targets provided by the optical fiducials” (Shi, Paragraphs [0014]-[0020], discloses the following:
PNG
media_image1.png
659
758
media_image1.png
Greyscale
); and detecting one or more defects on the target object based on the slice image (Mai, Paragraph [0114], discloses: “Use Al to find and locate the defect position of the 2D color image, calculate the height difference on the point cloud data at the same position, and confirm the depth information of cracks and pits”). However, the combination of Shi and Mai does not explicitly disclose “one or more processors and one or more non-transitory memory devices storing a program, which, when executed by any combination of the one or more processors, causes the one or more processors to perform an operation, the operation comprising:”. However, in an analogous field of endeavor, Pollard discloses in the following figure that “Figure 4 is a flow diagram of the example method of generating a 3D representation of the object by a computer system which is to process the scanning data. For example, it may be implemented by machine readable instructions stored on a non-transitory storage medium and executed by a processor of the computing system 1.” (Pollard, Paragraphs [0027] and Figure 4)
PNG
media_image10.png
577
470
media_image10.png
Greyscale
Therefore, it would be obvious for one of ordinary skill in the art to combine the processors and memory executing a program for 3D image scanning as shown in Pollard with the slice image acquisition methods found in the combination of Shi and Mai to achieve the same method described in Claim 16.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Pollard, and further in view of Georgeson 1.
Regarding Claim 17, the combination of Shi, Mai, and Pollard disclose “The inspection system of claim 16, wherein the operation further comprises:” (Shi, Paragraphs [0010], [0012], [0014]-[0020]; Mai, Paragraph [0114]; Pollard, Paragraphs [0027] and Figure 4; please see the above-described analysis for Claim 16). The combination of Shi, Mai, and Pollard is not relied on to disclose “causing the projector of the 3D scanner to project one or more defect indicators onto the target object to indicate respective ones of the one or more defects detected on the target object.”. However, in an analogous field of endeavor, Georgeson 1 discloses the following: “system for providing location specific maintenance history utilize a handheld device incorporating a camera, a projector and a communications interface. A microcontroller is connected to the communications interface and interconnected to the camera to receive image information from a region on an object within the FOV of the camera. The microcontroller is further interconnected to the projector to transmit repair history information for projection onto the object. A remote computer system incorporates a CAD database and a maintenance database with repair history and a mating communications interface. Software modules present on one or both of the microcontroller and remote computer system are employed for identifying markers on a structure present in the image information. The software modules then determine an absolute reference position for the handheld device to determine coordinates in the CAD database corresponding to the FOV and repair history information from the maintenance database is retrieved corresponding to determined coordinates. The repair history information is communicated to the microcontroller for transmission to the projector for projection onto the surface of the object at the defined coordinate location.” (Georgeson 1, Col 2, Lines 3-25). Georgeson 1 also discloses a pictorial view of a user projecting and outlining a previous defect onto the fuselage in Figure 4, where “information regarding a repair is projected showing the location and extent of the repair 42 and a data block 44 regarding the maintenance performed as a puncture repair” (Georgeson 1, Col 5, lines 5-8; Figure 4).
PNG
media_image2.png
448
665
media_image2.png
Greyscale
Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the projection of defects onto a large-scale object as seen in Georgeson 1 with the slice image acquisition methods within a stored program executed by one or more processors seen in the combination of Shi, Mai, and Pollard to allow users of the cart to effectively inspect defects on the airplane fuselage. By using the projector and camera on the cart to project defects as seen in Georgeson 1, the cart can intuitively alert and guide repair personnel towards specific issues easily so the necessary steps can be made to fix them. By knowing the exact location where the defect is located, engineers and maintenance technicians can also save time by using the inspection cart to quickly assess the damage. Therefore, it would be obvious for one of ordinary skill in the art to combine the Shi, Mai, Pollard, and Georgeson 1 references to achieve the method described in Claim 17.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Clark.
Regarding Claim 18, the combination of Shi and Mai discloses “The inspection system of Claim 15” (Shi, Paragraphs [0010], [0012], [0014]-[0020]; Mai, Paragraph [0114]; please see the above-described analysis for Claim 15); “wherein the track is a linear track and the optical fiducials include” (Shi, Figure 1 and Paragraph [0080] disclose “a set of three-dimensional scanners with LED tags attached are mounted on a bracket that can move along a track beside the object to be tested, and the bracket is moved to one end of the track” , where the track as shown in the figure traverses a linear path.
PNG
media_image3.png
668
739
media_image3.png
Greyscale
Figure 11: “The detector system 104 includes a detector carriage 110, which carries a magnetic induction sensor system 130 and, optionally, an ultrasonic sensor system 160. FIGS. 11-13 illustrate a detector carriage 110 according to an embodiment of the invention. The detector carriage 110 includes a frame 111 having a left side frame rail 112 and a right side frame rail 115. The left side frame rail 112 is formed from a left outside channel 113 and a left inside channel 114 joined by a forward end plate 118 and a rearward end plate 119.” (Clark, Col 6,
PNG
media_image5.png
561
796
media_image5.png
Greyscale
lines 66-67; Col 7, lines 1-8; Figure 5 (see below))
As this detector system is meant to inspect rails on a railroad, the track the system goes along is linear in nature. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the left and right rails arranged parallel to each other seen in Clark with the optical fiducials (LED tags) from the inspection system seen from the combination of Shi and Mai to allow the optical fiducials to have separate areas to guide the image acquisition as the cart is used for inspecting fuselage, as described in Claim 18.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Shi in view of Mai and Georgeson 2 (US 20150329221 A1).
Regarding Claim 19, the combination of Shi and Mai disclose “The inspection system of claim 15” (Shi, Paragraphs [0010], [0012], [0014]-[0020]; Mai, Paragraph [0114]; please see the above-described analysis for Claim 15). The combination of Shi and Mai is not relied on to disclose “wherein the target object is a fuselage, and wherein the track is a curved track that extends above and below the fuselage”. However, in an analogous field of endeavor, Georgeson 2 discloses “Systems and methods for high-speed non-destructive inspection of a half- or full-barrel-shaped workpiece, such as a barrel-shaped section of an aircraft fuselage. Such workpieces can be scanned externally using a mobile (e.g., translating) arch gantry system comprising a translatable arch frame disposed outside the fuselage section, a carriage that can travel along a curved track carried by the arch frame, a radially inward-extending telescopic arm having a proximal end fixedly coupled to the carriage, and an NDI sensor unit coupled to a distal end of the telescoping arm” (Georgeson 2, Abstract). Figures 12 and 13 best describe the curved tracks extending above and below the fuselage, whereas they “are diagrams representing respective end views of portions of an internal scanning system for scanning stringers of a full-barrel fuselage section 112 supported by a support base 113. This system has a single-arm, double-end-effector configuration of the type shown in FIG. 10 and a linear motion platform 78. The linear motion platform 78 comprises rollers 132 that ride on a pair of linear rails 130a and 130b. In this case the linear rails 130a and 130b are mounted to and supported by a platform lift 120 instead of on the floor.” (Georgeson 2, Paragraph [0101], Figures 12 & 13).
PNG
media_image11.png
534
427
media_image11.png
Greyscale
PNG
media_image12.png
526
406
media_image12.png
Greyscale
Figure 1 describes another configuration of the curved track inspection system, whereas it “represents an isometric view of portions of an external scanning system for ultrasonic inspection of the OML of a half-barrel fuselage section 12 in accordance with one embodiment. (The same external scanning system can be adapted to inspect respective halves of the OML of a full-barrel fuselage section.) Means for supporting the half-barrel fuselage section 12 are not shown in FIG. 1.” (Georgeson 2, Paragraph [0041])
PNG
media_image13.png
431
555
media_image13.png
Greyscale
Accordingly, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the curved tracks seen in Georgeson 2 with the inspection system described from the combination of Shi and Mai to ensure different embodiments of the cart can investigate the entire curvature of the surface of the airplane fuselage. Doing this will ensure that an inspection cart not only covers the sides of the plane during a scan, but also the top and bottom sides of the cylindrical structure of the body. Thus, it would be obvious to one of ordinary skill in the art to combine the Shi, Mai, and Georgeson 2 references to obtain the inspection system seen in Claim 19.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Negro (EP 3561764) teaches of a method to perform stitching of 2D images generated in a windowed conveyor system at a high speed.
Jovancevic (3D Point Cloud Analysis for Detection and Characterization of Defects on Airplane Exterior Surface) teaches a novel automatic vision-based inspection system that detects defects on an airplane exterior surface.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SORIE I KOROMA JR whose telephone number is (571)272-9259. The examiner can normally be reached Monday - Friday 8AM-6:00PM; Alternate Fridays Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about
filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SORIE I KOROMA JR/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights