DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election of Invention II (claim 4) in the reply filed on 03/23/2026 is acknowledged. Applicants' election is noted with traverse. Although Applicant points to the common AFP inspection environment and the fact that the disclosed algorithms may operate within an integrated in-process inspection pipeline, the groups as set forth to independent and distinct technical features such that the search is diverse for each group ( Invention I is directed to classical image-processing techniques, namely thresholding and morphology algorithms for detecting gap and overlap defects. Invention II is directed to a height/ profile-based machine-learning model trained to detect splice, missing tow, twisted tow, wrinkled tow, and folded tow defects. Invention III is directed to a luminance-based model for detecting marked splice and backer tape. Invention IV is directed to calibration of the detection algorithms based on comparison with a standardized workpiece and/or calibration window. Invention V is directed to an AI feedback model using detected defects and processing parameters to provide feedback to the AFP manufacturing process. )
Thus, while the restricted inventions may be disclosed as usable together in the same AFP inspection system, the groups are directed to different underlying concepts and different technical approaches: classical threshold/ morphology image processing, height/ profile machine-learning defect recognition, luminance/ texture-based detection, metrology/ calibration, and AI-based manufacturing feedback/ control. A search directed to one group would not necessarily be expected to identify the most relevant prior art for the other groups. For example, a search for thresholding and morphology gap/ overlap detection would not necessarily locate the most relevant prior art for height-profile machine-learning detection of tow defects; likewise, a search for luminance-based marked-splice/ backer-tape detection would not necessarily locate the most relevant prior art for calibration of defect-detection algorithms or AI feedback based on process parameters.
Therefore, the search for the generic claims would not reasonably encompass the specific subject matter of each dependent subcombination, and examination of all groups together would impose a serious search and examination burden. For at least these reasons, and upon reconsideration of Applicant’s traversal, the restriction requirement is still deemed proper and is therefore made FINAL.
Accordingly, examination will proceed on the elected invention only. The application has pending claims 1-20 (non-elected claims 3, 5, 13, and 16 are withdrawn from further consideration).
Drawings
The drawings are objected to because Fig. 7 contains text and labels that are blurred and not sufficiently legible. In particular, the words appearing in Fig. 7 cannot be clearly read from the submitted drawing, such that the figure does not clearly show the subject matter intended to be illustrated. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
Each of claims 2, 15, and 17 recites the claim languages: claim 2 recites "detecting at least one of a missing tow, a foreign object debris, a twisted tow, a folded tow, a wrinkled tow, a marked splice, an unmarked splice and a backer tape defect", claim 15 recites "detect at least one of a missing tow, a foreign object debris, a twisted tow, a folded tow, a wrinkled tow, a marked splice, an unmarked splice, a backer tape defect, an overlap defect, and a gap defect", claim 17 recites "present at least one of a location of the one or more defects on the AFP workpiece, a type of the one or more defects on the AFP workpiece, and an identification of a cumulative defect", respectively. The Superguide Corp. v. DirecTV Enterprises, Inc., 69 USPQ2d 1865 (Fed. Cir. 2004) decision regarding the claim interpretation of “at least one of x, y, and z.” at pages 15-16 set forth the rationale for determining that the term “and” is conjunctive (i.e. at least one of x, at least one of y, and at least one of z). Therefore the plain meaning of the current claim language “… at least one of …, and …” in light of the specification is interpreted to be:
Claim 2: "detecting at least one of a missing tow, at least one of a foreign object debris, at least one of a twisted tow, at least one of a folded tow, at least one of a wrinkled tow, at least one of a marked splice, at least one of an unmarked splice and at least one of a backer tape defect".
Claim 15: "detect at least one of a missing tow, at least one of a foreign object debris, at least one of a twisted tow, at least one of a folded tow, at least one of a wrinkled tow, at least one of a marked splice, at least one of an unmarked splice, at least one of a backer tape defect, at least one of an overlap defect, and at least one of a gap defect".
Claim 17: "present at least one of a location of the one or more defects on the AFP workpiece, at least one of a type of the one or more defects on the AFP workpiece, and at least one of an identification of a cumulative defect"
Also see MPEP 2111.01 on the subject of Plain Meaning given to claim terms.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4 and 8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 4, claim 4 recites: “wherein executing the series of detection algorithms on the grayscale image comprises training a height machine learning model to detect a splice, a missing tow, a twisted tow, a wrinkled tow, and a folded tow.”
The metes and bounds of claim 4 are unclear because the claim recites that executing the series of detection algorithms on the grayscale image comprises training a height machine learning model. It is unclear whether the claim requires detection to train the height machine learning model as part of the in-process defect-detection operation performed on the acquired grayscale image, or whether the claim requires executing/ using an already-trained height machine learning model to detect defects in the grayscale image.
Training a machine-learning model and applying a trained machine-learning model for inference are different operations. Therefore, the claim language does not clearly define whether the claimed method requires model training, model inference, or both. Accordingly, the scope of claim 4 is unclear.
For clarity, applicant may wish to amend claim 4 to recite, for example, “-- executing a trained height machine learning model --” or “ -- executing the series of algorithms on the grayscale image comprises training a height machine learning model--” the listed defects, if such amendment is supported by the originally filed disclosure. For the purpose of examination, the Examiner interprets claim 4 as follows: “ – executing the series of algorithms on the grayscale image comprises training a height machine learning model –"
Regarding claim 8, claim 8 recites the limitation "correlating each profile of the plurality of profiles with a respective AFP robot position of the plurality of AFP robot positions based on the plurality of profile timestamps and the plurality of AFP robot timestamps".
Claim 8 recites “the plurality of AFP robot positions” without proper antecedent basis. Claim 8 depends from claim 7. Claim 7 introduces a plurality of profiles and a plurality of profile timestamps, but claim 7 does not introduce “a plurality of AFP robot positions” The plurality of AFP robot positions appears to be introduced in claim 6, but claim 7 does not depend from claim 6. Therefore, it is unclear what antecedent subject matter provides basis for “the plurality of AFP robot positions” in claim 8. Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4 and 12 are rejected under 35 U.S.C. §102(a)(1) as being anticipated by Sacco (Sacco et al, “Machine Learning Methods for Rapid Inspection of Automated Fiber Placement Manufactured Composite Structures.” Scholar Commons, 2019).
Regarding claim 1, Sacco teaches a method of in-process inspection comprising:
acquiring a grayscale image of an automated fiber placement (AFP) workpiece;
( [Sec. 3.1], [Sec. 3.2.1], [Fig. 3.3]: describing Keyence LJ-V profilometers mounted on a KUKA robot scanning AFP scans and produces height profiles that are converted to grayscale images of the AFP surface. )
executing a series of detection algorithms on the grayscale image to identify a plurality of characteristics in the grayscale image indicative of one or more defects in the AFP workpiece; and
( [Sec. 3.1–3.2], [Sec. 3.3.1]: Sacco discloses processing the grayscale image wherein the software application processes profilometry data for identification and characterization of AFP defects. Sacco further discloses that the AFP defect detection software uses fully convolutional networks to process ACSIS scan images and segment scans into respective defect categories [corresponding to the characteristics]. Sacco also discloses that the initial output of the defect detection network is an array of pixel values corresponding to classes, and that a marching-squares algorithm places a bounding polygon around defect pixel collections, identifies classes, extracts boundary points, and calculates centroids as defect characteristics. )
detecting the one or more defects in the AFP workpiece based on the identified plurality of characteristics.
( [Sec. 3.3.1], [Sec. 4.2.2], [Figs. 4.8–4.9]: Sacco discloses that defects are identified in scan images and exact size and shape characteristics are extracted for export. Sacco further discloses that the defect detection software segments scans into respective defect categories and produces a prediction map highlighting collections of pixels corresponding to AFP defects [corresponding to the detection]. Sacco further discloses that the test article was scanned with ACSIS, images were analyzed using the Defect Detection Network, hand-placed defects were identified, characteristics were extracted, and common defects such as twist were properly identified by the system. )
Regarding claim 4, Sacco teaches the method of claim 1, wherein executing the series of detection algorithms on the grayscale image comprises training a height machine learning model to detect a splice, a missing tow, a twisted tow, a wrinkled tow, and a folded tow.
( [Page 40-42, Sec. 3.3.4 & Table 3.3]: Sacco further teaches training the system using scan images from live training environments, wherein the machine-learning algorithms are taught to identify defects in the scans. Sacco’s Table 3.3 identifies the trained defect categories including Twist, Splice, Missing Tow, Wrinkle, and Fold. )
Regarding claim 12, Sacco teaches the method of claim 1, further comprising measuring the plurality of characteristics in the grayscale image to determine whether at least one parameter of the one or more defects in the AFP workpiece is outside of a predetermined, acceptable tolerance range for a defect measurement within the AFP workpiece.
( [Abstract], [Sec. 3.4.2 & Fig. 3.14], [Sec. 4.1.2], [Sec. 5.3-5.4]: Sacco teaches that defects are identified in scan images and exact size and shape characteristics are extracted for export. Sacco further teaches that the defect detection network outputs pixel values corresponding to defect classes, and that the Marching Squares algorithm places a bounding polygon around each defect-pixel collection, identifies classes, extracts boundary points, and calculates centroids as logged defect characteristics. Sacco further teaches checking that each preprogrammed defect is correctly labeled and that its total area is properly bounded. Sacco also teaches the quantifiable data about part quality can be extracted an assessed that exact quantification of defect production and part quality includes extracting exact size and shape characteristics of an individual defect, and that rapid analysis from such defect information can indicate whether a defect is significant enough to repair or can be left on the ply. Sacco further teaches that the operator can control defect sizes and characterization fidelity, including refining how large a defect needs to be before being logged. )
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. §103 as being unpatentable over Sacco in view of Harik (Harik et al, US Automated Fiber Placement Defect Identity Cards: Cause, Anticipation, Existence, Significance, and Progression, 21 May 2018), further in view of Juarez (Juarez et al, Automated Fiber Placement Defect Identity Cards: Cause, Anticipation, Existence, Significance, and Progression, 21 May 2018).
Regarding claim 2, Sacco teaches the method of claim 1, wherein executing the series of detection algorithms on the grayscale image comprises detecting at least one of a missing tow, a foreign object debris, a twisted tow, a folded tow, a wrinkled tow,
( [Sec. 3.3.1], [Table 3.3], [Sec. 4.2.2, Fig. 4.6–4.9]: Sacco teaches that the AFP defect detection software uses fully convolutional networks to process ACSIS scan images and segment the scans into respective defect categories, resulting in a prediction map that highlights collections of pixels corresponding to AFP defects. Sacco’s Table 3.3 expressly identifies defect categories including Twist, Missing Tow, Wrinkle, FOD [foreign object debris], and Fold, which respectively correspond to a twisted tow, a missing tow, a wrinkled tow, foreign object debris, and a folded tow. Sacco further discloses test-article defects including Twist, Full Wrinkle, Fold, and gap/overlap defects, and states that the system identified hand-placed defects and extracted their characteristics. )
However, Sacco lists “Splice” but does not distinguish marked splice from unmarked splice where Harik discloses: a marked splice, an unmarked splice
( [Sec. 3.12 “Splice”]: Harik teaches that a splice occurs when two tows are joined end-to-end by overlapping and tacking them together, resulting in a portion of the spool that is thicker than the rest and usually marked by white dashes for detection. Harik further teaches that splices are difficult to detect visually if not marked, and that the thickness increase over the splice allows detection with a detection system. )
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to configure Sacco’s AFP defect-detection model to separately detect marked and unmarked splice defects, as taught by Harik, because Sacco already detects splice defects in an AFP inspection system and Harik explains that splices may be marked for detection or may be unmarked but still detectable by their thickness increase. Such modification would have predictably expanded Sacco’s splice-detection category into known AFP splice subtypes, improving defect classification specificity and inspection usefulness without changing Sacco’s underlying profilometry, ML defect-detection framework.
However, Sacco [as modified by Harik] lists FOD but still fails to disclose where Juarez teaches:
and a backer tape defect
( [Sec. 1], [Sec. 7], [Fig. 7], [Table 1]: Juarez teaches that foreign object debris (FOD) in AFP is often small pieces of manufacturing consumables, such as backing paper, accidentally embedded during construction. Juarez further teaches performance-assessment tests using typical AFP FOD materials including poly backing paper, which Juarez identifies as the material adhered to the tow before it is unwound from the AFP spool during layup. Juarez further teaches in-situ FOD defects using poly backing paper attached to individual tows before layup, and Table 1 reports measured results for Backing paper FOD on substrate and Backing paper FOD on Tow. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further configure Sacco [as modified by Harik]’s AFP defect-detection model to detect backing/ backer material defects, as taught by Juarez, because Sacco already detects AFP foreign object debris (FOD), and Juarez identifies backing paper/ poly backing paper as a typical AFP FOD material that may be embedded on the substrate or tow during layup. Such modification would have predictably expanded Sacco’s known FOD-detection category to include a known AFP manufacturing-consumable defect, thereby improving defect coverage and inspection reliability without changing Sacco’s underlying profilometry/ ML defect-detection framework.
Claims 6–11, 14 and 18–20 are rejected under 35 U.S.C. §103 as being unpatentable over Sacco in view of Schuster (Schuster et al, “Inline Quality Control for Thermoplastic Automated Fibre Placement.” Procedia Manufacturing, vol. 51, 1 Jan. 2020, pp. 505–511).
Regarding claim 6, Sacco teaches the method of claim 1, a profilometer setup, however Sacco does not teach where Schuster teaches: further comprising acquiring a plurality of AFP robot positions during a timeframe of AFP operation, wherein each AFP robot position of the plurality of AFP robot positions has a corresponding time stamp during the timeframe of AFP operation.
( [Sec. 3.2 “Profile Generation and Triggering”], [Sec. 4.2 “Positional Synchronization”], [Sec. 4.3 “Data Storage”]: Schuster teaches an inline measurement system for automated fiber placement, where the camera has a hardware-triggering interface for profile acquisition, and where profiles are triggered by robot movement. Schuster further teaches using KUKA’s “Fast Send Driver” which allows path-synchronous triggering and sends a UDP packet with the actual robot position whenever the trigger signal is activated. Schuster further teaches that incoming positional data is stored in memory, and that before storing the data, the asynchronous data sources for profiles and trigger positions are fused so that the robot position for every profile is associated with the profile. Schuster also teaches that each acquired profile is stored with encoder count and time stamp, allowing later determination of the real-world position where the profile was acquired. Therefore, Schuster teaches acquiring a plurality of AFP robot positions during AFP operation, with corresponding timing information for the robot-position/profile acquisition. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sacco’s AFP profilometry-based inspection method to acquire AFP robot positions with corresponding timestamps, as taught by Schuster, in order to synchronize each acquired inspection profile with the robot position at which the profile was captured. Such modification would have predictably improved defect localization, inspection-data traceability, and correlation of detected defects with AFP machine state. Sacco already recognizes correlating profilometer data with machine or programming data, and Schuster provides a known implementation using path-synchronous triggering and timestamped robot-position, profile association.
Regarding claim 7, Sacco teaches the method of claim 1, wherein acquiring the grayscale image of the AFP workpiece comprises, however Sacco does NOT teach a corresponding timestamp where Schuster teaches: capturing a plurality of profiles with a profilometer during a timeframe of AFP operation, wherein each profile of the plurality of profiles has a corresponding time stamp during the timeframe of AFP operation.
( [Sec. 3.1-3.2], [Sec. 4.1-4.2]: Schuster teaches a laser-line/ camera measurement system in which the camera grabs images and calculates height profiles from laser-line displacement. Schuster further teaches that the camera includes a hardware-triggering interface for profile acquisition, and that whenever a trigger is received, the camera grabs an image, calculates the height profile, and stores the profile with encoder count and time stamp. Schuster further teaches that after a predefined number of profiles has been acquired, the profile chunk is transferred to a buffer and read out. Schuster also teaches that the profiles are triggered by robot movement using path-synchronous triggering. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sacco’s profilometer-based AFP inspection method to capture and store each acquired profile with a corresponding timestamp, as taught by Schuster, in order to synchronize the acquired profile data with the AFP operation timeframe. Such modification would have predictably improved traceability of the inspection data, enabled accurate correlation of each captured profile with where and when it was acquired, and improved defect localization during AFP inspection.
Regarding claim 8, Sacco [as modified by Schuster] teaches the method of claim 7, further comprising correlating each profile of the plurality of profiles with a respective AFP robot position of the plurality of AFP robot positions based on the corresponding time stamp during the timeframe of AFP operation.
( [Sec. 3.2], [Sec. 4.2-4.3]: Schuster teaches that whenever a trigger is received, the camera grabs an image, calculates the height profile, and stores the profile in camera memory together with encoder count and time stamp, allowing later determination of the real-world position where the profile was acquired. Schuster further teaches that profiles are triggered by robot movement using the robot’s built-in path-synchronous triggering, and that KUKA’s “Fast Send Driver” sends a UDP packet with the actual robot position whenever the trigger signal is activated. Schuster further teaches that before storing the data, the asynchronous data sources for profiles and trigger positions are fused such that the robot position for every profile is associated with the profile. )
Regarding claim 9, Sacco [as modified by Schuster] teaches the method of claim 7, further comprising grouping the plurality of profiles to create a batch image of the AFP workpiece.
( Sacco, in [Sec. 3.2.1, Page. 31–32] & [Sec. 5.5.2, Page. 76–78], teaches that the profilometer scans contain height information in relation to the surface of the AFP part and that the height profiles are collapsed into a greyscale image that can then be processed. Sacco further teaches that the profilometer originally takes a single height profile and later stitches many of these profiles together to create a single image. )
Regarding claim 10, Sacco [as modified by Schuster] teaches the method of claim 9, further comprising processing the batch image to include a region of interest on the batch image, wherein the series of detection algorithms are performed on the region of interest.
( [Sec. 3.2 “Profile Generation and Triggering”], [Sec. 4.1 “Profile Acquisition”], [Sec. 7 “Inline Capability”]: Schuster teaches that profile chunks are stored in memory and stitched to a single image after acquisition of the actual track is finished. Schuster further teaches evaluating the resulting profile image by considering only a narrow region of interest (ROI) around the assumed tape ends, and states that a ±2.5 mm zone corresponding to 128 pixels was sufficient and stabilizing for the algorithm. Schuster further teaches that curve-fitting is then used to determine gaps, overlaps, and bumps. )
Regarding claim 11, Sacco [as modified by Schuster] teaches the method of claim 7, further comprising:
capturing a plurality of batch images of the AFP workpiece; and
( [Sec. 3.2.1], [Sec. 4.1.1-4.1.2], [Sec. 5.1]: Sacco teaches that scans are taken per ply and across each course in the ply, and that the profilometer scans contain height information that is collapsed into greyscale images. Sacco further teaches that a cylinder was scanned using all four profilometers with an offset to account for missing data patches between scan areas, and that the inspection procedure verifies receipt of scan images, where a series of images appears showing the height profile of the test part translated into a greyscale image. Sacco further summarizes that profilometry scans were taken in a ply-by-ply manner through the ACSIS platform and compressed into greyscale images for analysis. )
compiling the plurality of characteristics in the grayscale image across the plurality of batch images with a grouping algorithm into a main defect list.
( [Sec. 3.4.1-3.4.2; Fig. 3.13-3.14]; [Sec. 5.1]: Sacco teaches storing defect information locally and transferring it to an AFP Defect Server as a JSON file. Sacco further teaches that JSON files are used to store defect information in a tree-like data structure, and that the initial output of the defect detection network is an array of pixel values corresponding to classes. Sacco teaches using the Marching Squares algorithm to place bounding polygons around defect-pixel collections, separate polygons into respective cases, identify classes, extract boundary points, and calculate centroids as logged defect characteristics. Sacco further teaches that each of these characteristics is transferred into a JSON file and uploaded to the AFP Defect Server. Sacco summarizes that the bounding polygon and defect type are collected and used to create a JSON file listing all defects found across a given part, organized so that defects are placed in the context of the overall part and ply number. )
Regarding claim 14, with deficiencies of Sacco noted in square brackets [ ], Sacco teaches an in-process inspection system for automated fiber placement (AFP) manufacturing, the in-process inspection system integrated with an AFP machine configured to deposit composite material tows onto an AFP workpiece, the in-process inspection system comprising:
at least one profilometer coupled to [ an AFP head of ] the AFP machine, the at least one profilometer configured to collect profile data associated with the AFP workpiece by scanning the composite material tows [ during operation of the AFP machine ];
( [Sec. 3.2.1], [Sec. 4.1.1]: Sacco teaches that the Ingersoll Machine Tools Automated Composite Structure Inspection System (ACSIS) added a profilometry-based scanning system integrated with a KUKA KR120 robotic arm, and that the profilometry platform includes a four-laser array of Keyence LJ-7080 blue-light profilometers. Sacco further teaches that, during scanning, the profilometer is actuated across the AFP part in the orientation of the fiber angle, scans are taken per ply and across each course in the ply, and the profilometer is optimized to scan tows. Sacco further teaches that the profilometer scans contain height information in relation to the surface of the AFP part. Sacco also teaches a cylinder produced with an Ingersoll Machine Tools Lynx AFP machine and that ACSIS was programmed to run scans on a section of the cylinder using all four profilometers. )
an automated inspection module comprising a computer having one or more processors and a non-transitory computer readable storage medium, the computer communicatively coupled to the AFP machine and to the at least one profilometer, said computer configured to:
( [Sec. 3.4], [Sec. 3.5.4], [Sec. 4.1.1-4.1.2]: Sacco teaches a multi-computer hardware execution system for executing inspection ML algorithms and analysis tools, where defect information is stored locally on the inspection computer and transferred to a server as a JSON file. Sacco also teaches USC AFP Inspection software executing on an analysis computer, automatically loading bitmap images logged by ACSIS and analyzing them for defects using the image-processing network. Sacco further teaches ACSIS computer 2 connected to the Keyence LJ-7080 profilometer and forwarding scan data to ACSIS analysis software and USC defect-detection software. )
convert the profile data into a grayscale image of the AFP workpiece;
( [Sec. 3.2.1, Fig. 3.3], [Sec. 5.1]: Sacco teaches that the profilometer scans contain height information in relation to the surface of the AFP part, and that the height profiles are collapsed into a greyscale image that can then be processed. Sacco also summarizes that profilometry scans were taken through the ACSIS platform and compressed into greyscale images for analysis. )
identify a plurality of characteristics in the grayscale image indicative of one or more defects in the AFP workpiece; and
( [Sec. 3.1], [Sec. 3.3.1], [Sec. 3.4.2], [Fig. 3.14]: Sacco teaches that the software application processes profilometry data for the identification and characterization of AFP defects. Sacco further teaches that the AFP defect detection software uses fully convolutional networks to process ACSIS scan images and segment scans into respective defect categories, resulting in a prediction map that highlights collections of pixels corresponding to AFP defects. Sacco further teaches that the defect detection network outputs pixel values corresponding to defect classes, and that the Marching Squares algorithm places bounding polygons around each defect pixel collection, identifies classes, extracts boundary points, and calculates centroids, thereby identifying characteristics in the grayscale image indicative of AFP defects. )
[ detect the one or more defects in the AFP workpiece based on the identified characteristics during operation of the AFP machine. ]
As noted above in square brackets, Sacco’s profilometer is on the ACSIS/ KUKA inspection platform, not clearly coupled to the AFP head performing layup; and Sacco expressly says its ACSIS setup is not ideal for an on-line system, and the mandrel must be rotated away from the AFP machine after each ply. However, Schuster teaches:
at least one profilometer coupled to an AFP head of the AFP machine, the at least one profilometer configured to collect profile data associated with the AFP workpiece by scanning the composite material tows during operation of the AFP machine;
( [Abstract], [Sec. 3.1-3.2 & Fig. 3], [Sec. 4.2]: Schuster teaches an inline measurement system for automated fiber placement based on a laser-line/camera system for inline data acquisition. Schuster further teaches a robot-cell setup including an AFP head, heat source, laser line source, and camera, where the laser line source/ camera measurement system is arranged with the AFP head for scanning during the AFP process. Schuster teaches that the camera grabs images and calculates height profiles based on laser-line displacement. Schuster further teaches that profile acquisition is triggered by robot movement using path-synchronous triggering, and that the KUKA Fast Send Driver sends the actual robot position whenever the trigger signal is activated. )
detect the one or more defects in the AFP workpiece based on the identified characteristics during operation of the AFP machine.
( [Abstract], [Sec. 2.2], [Sec. 6.2], [Sec. 7], [Fig. 10]: Schuster teaches an inline measurement system for AFP that performs data acquisition, storage, and evaluation during the AFP process. Schuster further teaches that gaps, overlaps, and missing tows can be detected inline by evaluating single profiles, and that the acquired profiles are treated as a depthmap suitable for evaluation with standard computer-vision or machine-learning algorithms. Schuster further teaches considering a region of interest around the assumed tape ends and using curve-fitting to determine gaps, overlaps, and bumps. Schuster also discloses inline evaluation results in which a gap, an overlap, and a tow border were detected. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sacco’s AFP profilometry-based inspection system by coupling the profilometer system to the AFP head and performing profile acquisition and defect detection during AFP machine operation, as taught by Schuster, in order to provide inline inspection without requiring separate post-ply scanning. Such modification would have predictably improved inspection efficiency, reduced AFP machine downtime, and enabled earlier detection/ localization of defects during the manufacturing process. Sacco already teaches profilometer-based grayscale-image defect detection for AFP workpieces, and Schuster provides a known inline AFP implementation using head-associated profile measurement and real-time/ inline defect evaluation.
Regarding claim 18, Sacco [as modified by Schuster] teaches the system of claim 14, wherein said computer is further configured to present the plurality of characteristics using one or more of a defect grid, an image overlay, and a 3D viewport.
( Sacco, [Sec. 3.4.2 & Fig. 3.14], [Sec. 3.5.1], [Figs. 3.15–3.16]: Sacco teaches that, after defect pixels are identified, the Marching Squares algorithm places a bounding polygon around each defect-pixel collection, identifies classes, extracts boundary points, and calculates centroids as logged defect characteristics [characteristics using one or more of a defect grid]. Sacco further teaches that an image with the bounding polygons overlaid is produced and saved locally, and that the operator user interface includes functions for displaying color codes, viewing a segmentation map, returning analysis of the part to the display, and toggling course images. Sacco therefore teaches presenting defect characteristics using at least an image overlay/ segmentation-map display. Schuster, in [Sec. 5], further teaches an inline AFP inspection system based on laser-line height-profile measurement, where collected profiles are stored, stitched, evaluated, and may be treated as a depthmap for standard computer-vision or machine-learning evaluation. Schuster therefore supports presenting the profile/defect information in a 3D height-profile/depth-based viewport. )
Regarding claims 19-20, the rationale provided in the rejection of claims 9-10 is incorporated herein. In addition, Sacco [as modified by Schuster] teaches multi-computer hardware execution system is implemented [Sacco, Sec. 3.4 “Data Transfer and Communication”; Schuster, Sec. 4 “Acquisition Software”]. Accordingly, the method for in-process inspection of claims 9-10, corresponds to the system of claims 19-20, and performs the steps disclosed herein. Therefore, the claims are all rejected.
Claim 15 is rejected under 35 U.S.C. §103 as being unpatentable over Sacco [as modified by Schuster] in view of Harik (Harik et al, US Automated Fiber Placement Defect Identity Cards: Cause, Anticipation, Existence, Significance, and Progression, 21 May 2018), further in view of Juarez (Juarez et al, Automated Fiber Placement Defect Identity Cards: Cause, Anticipation, Existence, Significance, and Progression, 21 May 2018).
Regarding claim 15, Sacco [as modified by Schuster] teaches the system of claim 14, wherein said computer is further configured to detect at least one of a missing tow, a foreign object debris, a twisted tow, a folded tow, a wrinkled tow,
( [Sec. 3.3.1], [Table 3.3], [Sec. 4.2.2, Fig. 4.6–4.9]: Sacco teaches that the AFP defect detection software uses fully convolutional networks to process ACSIS scan images and segment the scans into respective defect categories, resulting in a prediction map that highlights collections of pixels corresponding to AFP defects. Sacco’s Table 3.3 expressly identifies defect categories including Twist, Missing Tow, Wrinkle, FOD [foreign object debris], and Fold, which respectively correspond to a twisted tow, a missing tow, a wrinkled tow, foreign object debris, and a folded tow. Sacco further discloses test-article defects including Twist, Full Wrinkle, Fold, and gap/ overlap defects, and states that the system identified hand-placed defects and extracted their characteristics. )
However, Sacco [as modified by Schuster] lists “Splice” but does not distinguish marked splice from unmarked splice where Harik discloses: a marked splice, an unmarked splice
( [Sec. 3.12 “Splice”]: Harik teaches that a splice occurs when two tows are joined end-to-end by overlapping and tacking them together, resulting in a portion of the spool that is thicker than the rest and usually marked by white dashes for detection. Harik further teaches that splices are difficult to detect visually if not marked, and that the thickness increase over the splice allows detection with a detection system. )
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to configure Sacco [as modified by Schuster]’s AFP defect-detection model to separately detect marked and unmarked splice defects, as taught by Harik, because Sacco already detects splice defects in an AFP inspection system and Harik explains that splices may be marked for detection or may be unmarked but still detectable by their thickness increase. Such modification would have predictably expanded Sacco’s splice-detection category into known AFP splice subtypes, improving defect classification specificity and inspection usefulness without changing Sacco’s underlying profilometry, ML defect-detection framework.
However, Sacco [as modified by Schuster and Harik] lists FOD but still fails to disclose where Juarez teaches: and a backer tape defect
( [Sec. 1], [Sec. 7], [Fig. 7], [Table 1]: Juarez teaches that foreign object debris (FOD) in AFP is often small pieces of manufacturing consumables, such as backing paper, accidentally embedded during construction. Juarez further teaches performance-assessment tests using typical AFP FOD materials including poly backing paper, which Juarez identifies as the material adhered to the tow before it is unwound from the AFP spool during layup. Juarez further teaches in-situ FOD defects using poly backing paper attached to individual tows before layup, and Table 1 reports measured results for Backing paper FOD on substrate and Backing paper FOD on Tow. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further configure Sacco [as modified by Schuster and Harik]’s AFP defect-detection model to detect backing/ backer material defects, as taught by Juarez, because Sacco already detects AFP foreign object debris (FOD), and Juarez identifies backing paper/ poly backing paper as a typical AFP FOD material that may be embedded on the substrate or tow during layup. Such modification would have predictably expanded Sacco’s known FOD-detection category to include a known AFP manufacturing-consumable defect, thereby improving defect coverage and inspection reliability without changing Sacco’s underlying profilometry/ ML defect-detection framework.
Claim 17 is rejected under 35 U.S.C. §103 as being unpatentable over Sacco [as modified by Schuster] in view of Engelbart (Engelbart et al, US 2009/0148030 A1, 2009).
Regarding claim 17, Sacco [as modified by Schuster] teaches the system of claim 14, wherein said computer is further configured to present at least one of a location of the one or more defects on the AFP workpiece, a type of the one or more defects on the AFP workpiece,
( [Sec. 3.4.2, Fig. 3.13-3.14], [Sec. 4.1.2], [Sec. 5.1]: Sacco teaches that the defect detection network outputs pixel values corresponding to defect classes, and that the Marching Squares algorithm places bounding polygons around defect-pixel collections, separates the polygons into respective cases, identifies classes, extracts boundary points, and calculates centroids as logged defect characteristics. Sacco further teaches that the location of the data in each JSON file gives an indication of the course where the defect is located, and Fig. 3.14 is titled “Defect Characteristics from Pixel Data to Position and Classes.” Sacco further teaches that the defect log indicates both the proper defect class and approximate location for each known defect on the test part, and that the bounding polygon and defect type are collected and used to create a JSON file listing all defects found across a given part. )
Sacco [as modified by Schuster] teaches presenting a location and type of defects, but does not clearly teach presenting an identification of a cumulative defect where Engelbart teaches:
wherein said computer is further configured to present at least one of a location of the one or more defects on the AFP workpiece, a type of the one or more defects on the AFP workpiece, and an identification of a cumulative defect.
( [0014-0015], [Fig. 9-10]: Engelbart teaches a processor, memory unit, error file, image-processing reference library, lookup table, user interface, and display screen for identifying and presenting foreign-object information. Engelbart teaches that the error file notes the x-y position of the foreign object, thereby presenting a defect location. Engelbart further teaches using dimensional attributes with an image-processing reference library and lookup table to assign a type or category to the detected foreign object. Engelbart further teaches a user interface/ display screen and FOD counter that provides a running total by category and cumulative total for FOD, thereby presenting an identification of a cumulative defect.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sacco [as modified by Schuster]’s AFP defect-presentation system to include Engelbart’s cumulative defect identification, because Sacco already presents defect location and defect type, and Engelbart teaches presenting cumulative defect totals by category and overall count to assist inspection review. Such modification would have predictably improved operator awareness of defect accumulation across the AFP workpiece, enabled more efficient review of overall part quality, and provided useful summary information for repair or acceptance decisions without changing Sacco’s underlying defect-detection framework.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights