DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s Amendments filed on June 16, 2026, has been entered and made of record.
Currently pending Claim(s) 1, 3-10, and 12-18
Independent Claim(s) 1 and 10
Amended Claim(s) 1, 3, 5-6, 10, 12, and 14-15
Canceled Claim(s) 2 and 11
Response to Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on June 16, 2026.
In view of amendments filed on June 16, 2026, to the claims, the Applicant has amended the independent claims 1 and 10 to include new limitations which further describe functions of the signal data preprocessor. The Applicant has also amended dependent claims 5 and 14 to include new limitations describing how preprocessing and candidate group determination occur on A-scan data before transforming candidate groups to B-scan or C-scan images for subsequent defect analysis of those images. The other amended claims 3, 6, 12, and 15 are amended to update claim dependencies or to add further clarification to limitations; these changes to claims 3, 6, 12, and 15 do not add any new matter.
In view of Applicant Arguments/Remarks filed June 16, 2026, with respect to the claims, the Applicant first argued (Remarks pages 9-11) that claims 1 and 10, as amended, are patentable over the cited prior art of Townsend (US 2018/0315180 A1) and Al-Hashmy (US 2022/0018811 A1). Specifically, the Applicant argued that the cited prior art merely discloses identifying blank locations of data as potential defect locations, but these references fail to teach the amended limitations that the signal data preprocessor is configured to detect two peaks, each having an amplitude greater than a threshold value from the noise-processed signal data, determine a clustering region by setting the detected two peaks as start point and end point, and divide the noise-processed signal data included in the clustering region into a plurality of clusters having a certain size.
The Examiner respectfully disagrees. Townsend in Fig. 5(a)-5(h) teaches identifying object regions within the ultrasonic signal data. Fig. 5(e) shows an example where pixels below the amplitude threshold are black (representing scanned objects or echoes/noise), and pixels above the amplitude threshold are white (representing the peak amplitude areas of empty space) [0076]. In Fig. 5(f), an object layer is determined between areas of empty space (peak amplitude) on the vertical axis. The Examiner argues that an object layer directly corresponds to a clustering region of claim 1. In Figs. 5(g)-5(h), an object layer is further divided into areas of interest based on the location of discontinuities or other abnormalities in that layer where a defect is likely to occur. Each discontinuity is a range of coordinates. Thus, an object layer is divided into subgroups which correspond to clusters.
Therefore, preprocessing ultrasonic signal data by selecting an area between two amplitude peaks and dividing that area into subregions is not novel over Townsend. Furthermore, dividing an area into clusters having a certain size does not provide novelty, since claim 1 does not provide any unique or specific criteria for determining the cluster size.
Next, the Applicant argued (Remarks pages 11-15) that Townsend fails to teach the newly amended limitations of claim 5. Specifically, the Applicant argued that Townsend does not teach that the preprocessed signal data is A-scan data, and image data generated for the defect candidate groups is 2D B-scan or 3D C-scan image data. The Examiner finds this argument to be persuasive. Townsend teaches performing operations on ultrasonic image data or ultrasonic signal matrix data, but Townsend does not specifically label the ultrasonic data as A-scan, B-scan, or C-scan data. Thus, Townsend does not teach or motivate identifying defect candidate groups in A-scan data and generating 2D B-scan or 3D C-scan images of the defect candidate groups for subsequent defect determination. Furthermore, as the Applicant argued on pages 13-15, Townsend teaches operating on 2D images or signal matrix data, so Townsend does not teach operating on 1D A-scan data.
Lastly, the Applicant argued (Remarks pages 15-16) that the other cited prior art also fails to make up for the deficiencies of Townsend regarding claim 5, and the Examiner finds this argument to be persuasive. Al-Hashmy teaches using machine learning to identify defects in ultrasonic data. Although Al-Hashmy’s methods can be performed on A-scan, B-scan, or C-scan data [0039], Al-Hashmy does not teach or motivate identifying areas of interest in A-scan data, converting the regions of interest to B-scan or C-scan image data, and then identifying defects in each image. Similarly, Konishi (US 2020/0096454 A1) also fails to teach this limitation.
Thus, the Examiner respectfully disagrees with the Applicant’s arguments regarding the independent claims 1 and 10, and the Examiner maintains the rejections under 35 U.S.C. § 103 applied to claims 1 and 10. Regarding claims 5 and 14, the Examiner finds the Applicant’s arguments to be persuasive, so claims 5 and 14 are objected to but indicated as potentially allowable in the body of rejection below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
ultrasonic flaw-detection device in claim 1;
defect candidate group selection unit in claims 1 and 3;
image data generator in claims 1 and 5-6;
defect determination unit in claims 1, 7, and 9;
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
These limitations are interpreted broadly as hardware and software modules as explained on pages 8-9 of the Specification. These modules may utilize generic computer components such as memory and at least one processor for executing instructions, and a non-transitory computer readable medium, such as a hard disk, may be included for storage of software and recorded data.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 7-10, 12, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Townsend (US 2018/0315180 Al) in view of Al-Hashmy et al. (US 2022/0018811 Al), hereafter Al-Hashmy.
Regarding claim 1, Townsend teaches an ultrasonic flaw-detection system comprising:
an ultrasonic flaw-detection device configured to transmit an ultrasonic wave to a detection target, collect an ultrasonic echo wave reflected from the detection target, and then generate a signal data ([0070] "Ultrasonic NDT scanning is performed by an array of probes 91, which emit sound waves that propagate inside the pipe structure 90 and receive the echoes resulting from interactions with its front and back walls (FIG. 3). The amplitude of the echoes is measured along two dimensions, length x which is measured in a direction parallel to the length of the pipe 90, and time t. The resulting data may be represented as a numerical matrix with echo amplitudes, where each row i corresponds to a given propagation time t, and each column j corresponds to a given horizontal position xj.");
a signal data preprocessor configured to preprocess the signal data ([0048-0051] and Fig. 2( a-d) show preprocessing of signal data to remove noise and irrelevant signals. [0048] states that the noise removal can be performed on raw signal data before converting the data to image form.);
a defect candidate group selection unit configured to select a defect candidate group based on the preprocessed signal data and generate defect candidate signal data which is the preprocessed signal data of the defect candidate group (Townsend teaches clustering areas in the signal matrix. In Fig. 2(e), resulting clusters are shown, and cluster 1 is selected for examination since it is the most relevant area to check for defects. [0060] "The set of objects remaining after the filtering are then sorted into clusters according to a predefined criterion." [0067] "For example, the area below a back wall in an NDT scan is not necessary for defect detection and can therefore be discarded so that defect detection can be focused on the relevant area, as shown in FIG. 2(i). Similar processes may be applied to any analogous scenario in which layers of interest define boundaries for unwanted data." Additionally, Townshend teaches identifying areas, such as gaps, in the signal data within cluster 1, and these areas are expected to contain a defect and are selected for subsequent examination for defects. [0042] "To assist in the detection of a defect in the portion of interest identified using steps 51 to S4, in step S5 a location of a gap in the data forming the cluster identified as the portion of interest is identified as a site of a potential defect. For example, gaps in the back wall of an item undergoing testing may be identified as ranges in the axis parallel to the wall's bounding box (usually x-axis) which do not contain any part of any back wall object." [0045] "Gap identifier 5 is configured to identify a location of a gap in the data forming the cluster as the portion of interest as a site of a potential defect." [0063] "To assist in the detection of a defect in the layer of interest, a location of a gap in the data forming the cluster identified as the layer of interest is identified as a site of a potential defect. As shown by the dashed lines in FIG. 2(g) the position of missing data, i.e. discontinuities or gaps in the layer of interest, are noted.");
an image data generator configured to generate image data based on the defect candidate signal data included in the defect candidate group (Townshend teaches selecting data gaps from the matrix signal data as defect candidate locations. This process can be applied to an ultrasonic image or signal matrix, but after identification of defect candidate locations, the locations are visualized as an image for an operator to inspect defects. [0065-0066] “Missing data, i.e. discontinuities in a layer, may be presented by drawing any shape that encloses the corresponding range in list LC, for example by drawing vertical lines either side of the gap or by drawing a circle with the centre of the range as its centroid and the length of the range as its diameter. Gap boundaries may be drawn in any thickness, color, and may or may not be filled. The annotated image may be displayed on screen, saved to file, printed or presented by any other means or combination of means.” [0082-0083] “In this case the user also wishes to identify gaps in the back wall. Every range of values in the x-axis (the layer axis) which contains no pixels from any objects in the cluster corresponding to the layer of interest, as shown in FIG. 5(h), is added to a list L1=[G1,0, G1,1, G1,2]=[[50, 60], [300, 305], [610, 630]]. The location of the back wall, and its gaps, are presented to the user, as shown in FIG. 5(i). A box is drawn around the back wall, and vertical lines are drawn on either side of each gap.”);
wherein the signal data preprocessor is configured to:
generate noise-processed signal data by removing noise from the signal data ([0067] “For the back wall detection problem, or any analogous scenario, noise or any other unwanted data may be identified using the locations of any layers of interest. For example, the area below a back wall in an NDT scan is not necessary for defect detection, and can therefore be discarded so that defect detection can be focused on the relevant area, as shown in FIG. 2(i). Similar processes may be applied to any analogous scenario in which layers of interest define boundaries for unwanted data.” Additionally, Al-Hashmy teaches noise removal in detail in [0040] and is obvious to combine with Townsend, as argued below.),
detect two peaks, each having an amplitude greater than a threshold value from the noise-processed signal data; determine clustering region by setting the detected two peaks as start point and end point (See Figs. 5(a)-5(f). Fig. 5(e) shows an example where pixels below the amplitude threshold are black (representing scanned objects or echoes/noise), and pixels above the amplitude threshold are white (representing the peak amplitude areas of empty space) [0076]. In Fig. 5(f), an object layer is determined as an object between areas of empty space (peak amplitude). An object layer corresponds to a clustering region.); and
divide the noise-processed signal data included in the clustering region into a plurality of clusters having a certain size (In Figs. 5(g)-5(h), an object layer (clustering region) is further divided based on the location of discontinuities. Each discontinuity is a range of coordinates which is denoted as an area of interest within the object layer which is most likely to contain a defect.).
As shown above, Townshend teaches the process of selecting candidate defect locations from ultrasonic signal data and displaying an image of the locations to an operator who can view the images and identify defects. [0044] “After the portion of interest is identified, the portion of interest in the image may be visually analyzed by an operator to identify one or more defects in the item under test.” Thus, Townshend fails to teach utilization of machine learning (as required by the defect determination unit) for determining defects in each defect candidate group. More specifically, Townshend fails to teach a defect determination unit configured to determine whether there is a defect in the defect candidate group based on the image data.
However, Al-Hashmy teaches a defect determination unit configured to determine whether there is a defect in the defect candidate group based on the image data (Figs. 3-4 show the Aberration Detection System (ADS) taught by Al-Hashmy. The ADS receives an ultrasonic image of damaged objects, such as a wind turbine blade, and utilizes machine learning to detect and classify defects in the images. [0008] “…receiving raw ultrasound scan image data of a test section comprising the material having internal defects or voids; sending an image rendering signal to cause a computer resource asset to display an ultrasound scan image based on the raw ultrasound scan image data; and receiving a label corresponding to the ultrasound scan image, the label including an aberration type, an aberration location or an aberration dimension of each aberration on the test section, wherein the aberration type comprises a harmful or potentially harmful aberration.”).
Townshend and Al-Hashmy are analogous in the art, because both teach methods of utilizing ultrasound waves for analyzing objects for defects. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the final step of defect detection. As shown above, Townsend teaches the process of detecting defect candidate groups from ultrasonic signal and outputting an ultrasonic image of a group for an operator to examine, but utilizing a machine learning technique to analyze the image would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.”).
Regarding claim 3, Townshend and Al-Hashmy teach the ultrasonic flaw-detection system of claim 1. Townshend further teaches wherein the defect candidate group selection unit is configured to: determine whether a defect is included in the noise-processed signal data belonging to each cluster based on a deep learning algorithm that uses each of the plurality of clusters as an input, and select the cluster determined to include a defect as the defect candidate group (In Fig. 2(e), clusters are shown, and cluster 1 is selected for examination since it is the most relevant area to check for defects. Townshend also teaches selecting gaps within a cluster from the matrix signal data as defect candidate locations. This process can be applied to an ultrasonic image or a signal matrix, but after identification of defect candidate locations, the locations are visualized as an image for an operator to inspect defects. [0065-0066] “Missing data, i.e. discontinuities in a layer, may be presented by drawing any shape that encloses the corresponding range in list LC, for example by drawing vertical lines either side of the gap or by drawing a circle with the centre of the range as its centroid and the length of the range as its diameter. Gap boundaries may be drawn in any thickness, color, and may or may not be filled. The annotated image may be displayed on screen, saved to file, printed or presented by any other means or combination of means.” [0082-0083] “In this case the user also wishes to identify gaps in the back wall. Every range of values in the x-axis (the layer axis) which contains no pixels from any objects in the cluster corresponding to the layer of interest, as shown in FIG. 5(h), is added to a list L1=[G1,0, G1,1, G1,2]=[[50, 60], [300, 305], [610, 630]]. The location of the back wall, and its gaps, are presented to the user, as shown in FIG. 5(i). A box is drawn around the back wall, and vertical lines are drawn on either side of each gap.” Also see Fig. 5(a)-5(h) which show noise-processing the signal before dividing the data into layers (clusters) during preprocessing and determining areas likely to obtain defects in the layers.).
Regarding claim 7, Townsend and Al-Hashmy teach the ultrasonic flaw-detection system of claim 1. Al-Hashmy further teaches wherein the defect determination unit is configured to determine whether each of the defect candidate groups has a defect based on a deep learning algorithm using the image data as an input ([0060] “The ADS system 100 can include at least one machine learning platform. The ADS system 100 includes a bus 105, a processor 110 and a storage 120. The ADS system 100 can include a network interface 130, an input-output (TO) interface 140, a driver unit 150, an aberration detection and evaluation (ADE) stack 160…” [0061] “The ADE stack 160 can include a feature extraction unit 162, a classification unit 164, an aberration predictor 166, and a labeler unit 168. The ADE stack 160 can include a machine learning (ML) platform, including, for example, one or more feedforward or feedback neural networks. The ML platform can include, for example, an artificial neural network (ANN), a convolutional neural network (CNN), a deep convolutional neural network (DCNN), a recurrent convolutional neural network (RCNN), a Mask-RCNN,…”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by using machine learning for automating the step of defect detection. Townsend teaches automatically identifying defect candidate defect group in signal matrix data and generating ultrasonic images for human inspection, but utilizing a machine learning technique to analyze the image would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.”).
Regarding claim 8, Townsend and Al-Hashmy teach the ultrasonic flaw-detection system of claim 7. Al-Hashmy further teaches wherein the deep learning algorithm is a you only look once (YOLO) algorithm or a Faster R-CNN algorithm ([0061] “The ADE stack 160 can include a machine learning (ML) platform, including, for example, one or more feedforward or feedback neural networks. The ML platform can include, for example, an artificial neural network (ANN), a convolutional neural network (CNN), a deep convolutional neural network (DCNN), a recurrent convolutional neural network (RCNN), a Mask-RCNN…”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the step of defect detection (See the rationale applied to claim 7 above). Additionally, the choice of a YOLO, R-CNN, or any other machine learning algorithm would be obvious to one of ordinary skill applying machine learning to analyze defects in ultrasonic images. As shown in 0061 of Al-Hashmy, many different machine learning options could be employed for this task.
Regarding claim 9, Townsend and Al-Hashmy teach the ultrasonic flaw-detection system of claim 7. Al-Hashmy further teaches wherein the defect determination unit is configured to output whether there is a defect for each of the defect candidate groups and output, when there is a defect, a bounding box that surrounds the corresponding defect ([0079] “The aberration predictor 166 can be arranged to receive the resultant image cells and predict aberrations that might exist in the asset 10, including, for example, on an outer surface, in a wall portion, or an inner surface of the asset 10. The aberration predictor 166 can generate a confidence score for each image cell that indicates the likelihood that a bounding box includes an aberration. The aberration predictor 166 can interact with the classification unit 164 and perform bounding box classification, refinement and scoring based on the aberrations in the image represented by the UT image data.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the step of defect detection (See the rationale applied to claim 7 above). Additionally, the choice of utilizing a bounding box would have been obvious to one of ordinary skill implementing a CNN for detecting a defect area in an image. Townsend taught annotating images to visually show areas with defects [0064-0067].
Regarding claim 10, Townsend teaches an ultrasonic flaw-detection method by the ultrasonic flaw-detection system, the method comprising:
transmitting an ultrasonic wave to a detection target, collecting an ultrasonic echo wave reflected from the detection target, and then generating a signal data ([0070] "Ultrasonic NDT scanning is performed by an array of probes 91, which emit sound waves that propagate inside the pipe structure 90 and receive the echoes resulting from interactions with its front and back walls (FIG. 3). The amplitude of the echoes is measured along two dimensions, length x which is measured in a direction parallel to the length of the pipe 90, and time t. The resulting data may be represented as a numerical matrix with echo amplitudes, where each row i corresponds to a given propagation time t, and each column j corresponds to a given horizontal position xj.");
preprocessing the signal data ([0048-0051] and Fig. 2( a-d) show preprocessing of signal data to remove noise and irrelevant signals. [0048] states that the noise removal can be performed on raw signal data before converting the data to image form.);
selecting a defect candidate group based on the preprocessed signal data and generate defect candidate signal data which is the preprocessed signal data of the defect candidate group (Townsend teaches clustering areas in the signal matrix. In Fig. 2(e), resulting clusters are shown, and cluster 1 is selected for examination since it is the most relevant area to check for defects. [0060] "The set of objects remaining after the filtering are then sorted into clusters according to a predefined criterion." [0067] "For example, the area below a back wall in an NDT scan is not necessary for defect detection and can therefore be discarded so that defect detection can be focused on the relevant area, as shown in FIG. 2(i). Similar processes may be applied to any analogous scenario in which layers of interest define boundaries for unwanted data." Additionally, Townshend teaches identifying areas, such as gaps, in the signal data within cluster 1, and these areas are expected to contain a defect and are selected for subsequent examination for defects. [0042] "To assist in the detection of a defect in the portion of interest identified using steps 51 to S4, in step S5 a location of a gap in the data forming the cluster identified as the portion of interest is identified as a site of a potential defect. For example, gaps in the back wall of an item undergoing testing may be identified as ranges in the axis parallel to the wall's bounding box (usually x-axis) which do not contain any part of any back wall object." [0045] "Gap identifier 5 is configured to identify a location of a gap in the data forming the cluster as the portion of interest as a site of a potential defect." [0063] "To assist in the detection of a defect in the layer of interest, a location of a gap in the data forming the cluster identified as the layer of interest is identified as a site of a potential defect. As shown by the dashed lines in FIG. 2(g) the position of missing data, i.e. discontinuities or gaps in the layer of interest, are noted.");
generating image data based on the defect candidate signal data included in the defect candidate group (Townshend teaches selecting data gaps from the matrix signal data as defect candidate locations. This process can be applied to an ultrasonic image or signal matrix, but after identification of defect candidate locations, the locations are visualized as an image for an operator to inspect defects. [0065-0066] “Missing data, i.e. discontinuities in a layer, may be presented by drawing any shape that encloses the corresponding range in list LC, for example by drawing vertical lines either side of the gap or by drawing a circle with the centre of the range as its centroid and the length of the range as its diameter. Gap boundaries may be drawn in any thickness, color, and may or may not be filled. The annotated image may be displayed on screen, saved to file, printed or presented by any other means or combination of means.” [0082-0083] “In this case the user also wishes to identify gaps in the back wall. Every range of values in the x-axis (the layer axis) which contains no pixels from any objects in the cluster corresponding to the layer of interest, as shown in FIG. 5(h), is added to a list L1=[G1,0, G1,1, G1,2]=[[50, 60], [300, 305], [610, 630]]. The location of the back wall, and its gaps, are presented to the user, as shown in FIG. 5(i). A box is drawn around the back wall, and vertical lines are drawn on either side of each gap.”); and
wherein the preprocessing the signal data comprises:
generate noise-processed signal data by removing noise from the signal data ([0067] “For the back wall detection problem, or any analogous scenario, noise or any other unwanted data may be identified using the locations of any layers of interest. For example, the area below a back wall in an NDT scan is not necessary for defect detection, and can therefore be discarded so that defect detection can be focused on the relevant area, as shown in FIG. 2(i). Similar processes may be applied to any analogous scenario in which layers of interest define boundaries for unwanted data.” Additionally, Al-Hashmy teaches noise removal in detail in [0040] and is obvious to combine with Townsend, as argued below.);
detecting two peaks, each having an amplitude greater than a threshold value from the noise-processed signal data; determining clustering region by setting the detected two peaks as start point and end point (See Figs. 5(a)-5(f). Fig. 5(e) shows an example where pixels below the amplitude threshold are black (representing scanned objects or echoes/noise), and pixels above the amplitude threshold are white (representing the peak amplitude areas of empty space) [0076]. In Fig. 5(f), an object layer is determined as an object between areas of empty space (peak amplitude). An object layer corresponds to a clustering region.); and
dividing the noise-processed signal data included in the clustering region into a plurality of clusters having a certain size (In Figs. 5(g)-5(h), an object layer (clustering region) is further divided based on the location of discontinuities. Each discontinuity is a range of coordinates which is denoted as an area of interest within the object layer which is most likely to contain a defect.).
As shown above, Townshend teaches the process of selecting candidate defect locations from ultrasonic signal data and displaying an image of the locations to an operator who can view the images and identify defects. [0044] “After the portion of interest is identified, the portion of interest in the image may be visually analyzed by an operator to identify one or more defects in the item under test.” Thus, Townshend fails to teach utilization of machine learning (as required by the defect determination unit) for determining defects in each defect candidate group. Specifically, Townshend fails to teach determining whether there is a defect in the defect candidate group based on the image data.
However, Al-Hashmy teaches determining whether there is a defect in the defect candidate group based on the image data (Figs. 3-4 show the Aberration Detection System (ADS) taught by Al-Hashmy. The ADS receives an ultrasonic image of damaged objects, such as a wind turbine blade, and utilizes machine learning to detect and classify defects in the images. [0008] “…receiving raw ultrasound scan image data of a test section comprising the material having internal defects or voids; sending an image rendering signal to cause a computer resource asset to display an ultrasound scan image based on the raw ultrasound scan image data; and receiving a label corresponding to the ultrasound scan image, the label including an aberration type, an aberration location or an aberration dimension of each aberration on the test section, wherein the aberration type comprises a harmful or potentially harmful aberration.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the final step of defect detection. As shown above, Townsend teaches the process of detecting defect candidate groups from ultrasonic signal and outputting an ultrasonic image of a group for an operator to examine, but utilizing a machine learning technique to analyze the image would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.”).
Regarding claim 12, Townsend and Al-Hashmy teach the ultrasonic flaw-detection method of claim 10. Townsend further teaches wherein the selecting a defect candidate group comprises determining whether a defect is included in the noise-processed signal data belonging to each cluster based on a deep learning algorithm that uses each of the plurality of clusters as an input (In Fig. 2(e), resulting clusters are shown, and cluster 1 is selected for examination since it is the most relevant area to check for defects. Townshend also teaches selecting gaps within a cluster from the matrix signal data as defect candidate locations. This process can be applied to an ultrasonic image or signal matrix, but after identification of defect candidate locations, the locations are visualized as an image for an operator to inspect defects. [0065-0066] “Missing data, i.e. discontinuities in a layer, may be presented by drawing any shape that encloses the corresponding range in list LC, for example by drawing vertical lines either side of the gap or by drawing a circle with the centre of the range as its centroid and the length of the range as its diameter. Gap boundaries may be drawn in any thickness, color, and may or may not be filled. The annotated image may be displayed on screen, saved to file, printed or presented by any other means or combination of means.” [0082-0083] “In this case the user also wishes to identify gaps in the back wall. Every range of values in the x-axis (the layer axis) which contains no pixels from any objects in the cluster corresponding to the layer of interest, as shown in FIG. 5(h), is added to a list L1=[G1,0, G1,1, G1,2]=[[50, 60], [300, 305], [610, 630]]. The location of the back wall, and its gaps, are presented to the user, as shown in FIG. 5(i). A box is drawn around the back wall, and vertical lines are drawn on either side of each gap.”).
Regarding claim 16, Townsend and Al-Hashmy teach the ultrasonic flaw-detection method of claim 10. Al-Hashmy further teaches wherein the determining whether there is a defect in the defect candidate group based on the image data comprises determining whether there is a defect in each of the defect candidate groups based on a deep learning algorithm using the image data as an input ([0060] “The ADS system 100 can include at least one machine learning platform. The ADS system 100 includes a bus 105, a processor 110 and a storage 120. The ADS system 100 can include a network interface 130, an input-output (TO) interface 140, a driver unit 150, an aberration detection and evaluation (ADE) stack 160…” [0061] “The ADE stack 160 can include a feature extraction unit 162, a classification unit 164, an aberration predictor 166, and a labeler unit 168. The ADE stack 160 can include a machine learning (ML) platform, including, for example, one or more feedforward or feedback neural networks. The ML platform can include, for example, an artificial neural network (ANN), a convolutional neural network (CNN), a deep convolutional neural network (DCNN), a recurrent convolutional neural network (RCNN), a Mask-RCNN,…”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by using machine learning for automating the step of defect detection. Townsend teaches automatically identifying defect candidate defect group in signal matrix data and generating ultrasonic images for human inspection, but utilizing a machine learning technique to analyze the image would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.”).
Regarding claim 17, Townsend and Al-Hashmy teach the ultrasonic flaw-detection method of claim 16. Al-Hashmy further teaches wherein the deep learning algorithm is a you only look once (YOLO) algorithm or a Faster R-CNN algorithm ([0061] “The ADE stack 160 can include a machine learning (ML) platform, including, for example, one or more feedforward or feedback neural networks. The ML platform can include, for example, an artificial neural network (ANN), a convolutional neural network (CNN), a deep convolutional neural network (DCNN), a recurrent convolutional neural network (RCNN), a Mask-RCNN…”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the step of defect detection (See the rationale applied to claim 16 above). Additionally, the choice of a YOLO, R-CNN, or any other machine learning algorithm would be obvious to one of ordinary skill applying machine learning to analyze defects in ultrasonic images. As shown in 0061 of Al-Hashmy, many different machine learning options could be employed for this task.
Regarding claim 18, Townsend and Al-Hashmy teach the ultrasonic flaw-detection method of claim 16. Al-Hashmy further teaches wherein the determining whether there is a defect in the defect candidate group based on the image data comprises outputting whether there is a defect in each of the defect candidate groups and outputting, when there is a defect, a bounding box that surrounds the corresponding defect ([0079] “The aberration predictor 166 can be arranged to receive the resultant image cells and predict aberrations that might exist in the asset 10, including, for example, on an outer surface, in a wall portion, or an inner surface of the asset 10. The aberration predictor 166 can generate a confidence score for each image cell that indicates the likelihood that a bounding box includes an aberration. The aberration predictor 166 can interact with the classification unit 164 and perform bounding box classification, refinement and scoring based on the aberrations in the image represented by the UT image data.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by automating the step of defect detection (See the rationale applied to claim 16 above). Additionally, the choice of utilizing a bounding box would have been obvious to one of ordinary skill implementing a CNN for detecting a defect area in an image. Townsend taught annotating images to visually show areas with defects [0064-0067].
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Townsend (US 2018/0315180 A1) and Al-Hashmy (US 2022/0018811 A1), and further in view of Park et al. (System Invariant Method for Ultrasonic Flaw Classification in Weldments Using Residual Neural Network. Appl. Sci. 2022, 12, 1477.), hereafter Park.
Regarding claim 4, Townsend and Al-Hashmy teach the ultrasonic flaw-detection system of claim 3. Townsend teaches that portions of interest where defects are likely present in the ultrasonic signal matrix data are determined, and Al-Hashmy teaches that many different machine learning models are applicable for finding defects in ultrasonic images in [0061]. However, neither Townsend or Al-Hashmy specifically teach wherein the deep learning algorithm is a variational auto encoder (VAE) or a residual neural network (ResNet) for finding defect candidate locations in signal data.
However, Park teaches wherein the deep learning algorithm is a variational auto encoder (VAE) or a residual neural network (ResNet) (See all of section 4 and 5.3-5.4 for discussions of the ResNet architecture and performance. In this study, a ResNet was applied to receive ultrasonic signal data and classify the echo into a defect group to identify if a defect is present. The defect groups included: crack, lack of fusion, slag inclusion, porosity, and incomplete penetration.).
Townsend, Al-Hashmy, and Park are analogous in the art to the claimed invention, because all teach methods of analyzing ultrasonic signal and/or images for determining the presence of defects in an object. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by utilizing a ResNet for automating the process of defect detection. Townsend teaches automatically identifying defect candidate defect group in signal matrix data, but utilizing a machine learning technique specifically to analyze the signal data would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.” Regarding the use of a ResNet specifically, Park motivates the use of a ResNet over traditional CNN architectures (used by Al-Hashmy) since deep ResNets can offer better performance ([Park Section 4.2] “Conventional CNNs feature a weak spot in significantly deep networks, as their performance degrades after a certain depth. However, significantly deep networks are required to train with large amounts of data. ResNets solve this problem through their distinct network architecture as previously described, obtaining the best results by training 152 layers of ResNet using the CIFAR-10 dataset.”).
Regarding claim 13, Townsend and Al-Hashmy teach the ultrasonic flaw-detection method of claim 12. Townsend teaches that portions of interest where defects are likely present in the ultrasonic signal matrix data are automatically determined, and Al-Hashmy teaches that many different machine learning models are applicable for finding defects in ultrasonic images in 0061. However, neither Townsend or Al-Hashmy specifically teach wherein the deep learning algorithm is a variational auto encoder (VAE) or a residual neural network (ResNet) for finding defect candidate locations in signal data.
However, Park teaches wherein the deep learning algorithm is a variational auto encoder (VAE) or a residual neural network (ResNet) (See all of section 4 and 5.3-5.4 for discussions of the ResNet architecture and performance. In this study, a ResNet was applied to receive ultrasonic signal data and classify the echo into a defect group to identify if a defect is present. The defect groups included: crack, lack of fusion, slag inclusion, porosity, and incomplete penetration.).
Townsend, Al-Hashmy, and Park are analogous in the art to the claimed invention, because all teach methods of analyzing ultrasonic signal and/or images for determining the presence of defects in an object. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Townsend’s invention by utilizing a ResNet for automating the process of defect detection. Townsend teaches automatically identifying defect candidate defect group in signal matrix data, but utilizing a machine learning technique specifically to analyze the signal data would save additional time and reduce overall effort, which is a purpose of Townsend’s invention ([Townsend 0012] “By automating the initial part of the inspection process in this way, quality control engineers/technicians can complete their inspections in more efficient ways, taking less inspection time per test object and hence reducing the overall human effort and costs.”). Additionally, Al-Hashmy shares this motivation by identifying the need for a cost-effective technology solution for defect detection ([Al-Hashmy 0004] “Since both metallic and non-metallic assets are commonly used in a variety of industries, there exists a great unfulfilled need for a cost-effective and reliable technology solution for inspecting, detecting, monitoring, analyzing or assessing aberrations in either or both metallic or nonmetallic assets.” Regarding the use of a ResNet specifically, Park motivates the use of a ResNet over traditional CNN architectures (used by Al-Hashmy) since deep ResNets can offer better performance ([Park Section 4.2] “Conventional CNNs feature a weak spot in significantly deep networks, as their performance degrades after a certain depth. However, significantly deep networks are required to train with large amounts of data. ResNets solve this problem through their distinct network architecture as previously described, obtaining the best results by training 152 layers of ResNet using the CIFAR-10 dataset.”).
Allowable Subject Matter
Claims 5-6 and 14-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 5, the closest prior art of record, Townsend (US 2018/0315180 Al) and Al-Hashmy (US 2022/0018811 Al), teaches the ultrasonic flaw-detection system of claim 1. However, neither Townsend or Al-Hashmy teach wherein the preprocessed signal data generated by the signal data preprocessor is pre-processed A-scan data, which is one-dimensional ultrasonic signal data representing magnitude of the signal data corresponding to each inspection location within the detection target, the pre-processed A-scan data being data before being transformed into two-dimensional B-scan data or three-dimensional C-scan data, wherein the image data generated by the image data generator is two-dimensional B-Scan image data or three-dimensional C-Scan image data based on the defect candidate signal data included in the defect candidate group, wherein the defect determination unit is configured to determine whether there is a defect in the defect candidate group based on the two-dimensional B-Scan image data or the three-dimensional C-Scan image data.
Townsend teaches performing operations on ultrasonic image data or ultrasonic signal matrix data, but Townsend does not specifically label this data as A-scan, B-scan, or C-scan data. Thus, Townsend does not teach or motivate identifying candidate defect locations in A-scan data and generating images of only the candidate defect locations in 2D B-scan or 3D C-scan images for subsequent defect determination.
Al-Hashmy teaches using machine learning to identify defects in ultrasonic data. Although Al-Hashmy’s methods can be performed on A-scan, B-scan, or C-scan data [0039], Al-Hashmy does not teach or motivate identifying candidate defect locations in A-scan data before converting only the candidate defect locations to 2D B-scan or 3D C-scan images for subsequent defect determination.
Furthermore, the other prior art fails to teach this specific architecture of acquiring A-scan data, identifying candidate defect locations in the A-scan data, generating 2D B-scan or 3D C-scan images for only the candidate defect locations, and subsequently applying machine learning to identify defects in the images. Other prior art generally operates on one specific type of scan data throughout the entire process. For example, Medak (Automated Defect Detection From Ultrasonic Images Using Deep Learning in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 68(10), 3126-3134; from the IDS dated December 19, 2024) teaches utilizing EfficientDet to analyze ultrasonic images for identifying defects in B-scan or C-scan images, and Medak teaches that the images are obtained from A-scan data [Fig. 1]. However, Medak does not teach identifying candidate defect locations from A-scan data and converting only those locations to B-scan or C-scan images for subsequent machine learning analysis of defects. Similarly, Chen (Deep Learning for the Detection and Recognition of Rail Defects in Ultrasound B-Scan Images. Transportation Research Record: Journal of the Transportation Research Board, 2675(11), 888-901.) teaches using a YOLOv3 classifier for detecting and classifying defects in B-scan images, but Chen does not mention a preceding step of analyzing A-scan data for candidate defect locations to B-scan images of those locations specifically. As an additional example, Gou (CN 110988140 B) utilizes a R-CNN for identifying fatigue cracks in B-scan and C-scan images. Gou specifically teaches obtaining A-scan data and determining B-scan and C-scan images to be used with the R-CNN for defect detection, but Gou does not teach identifying candidate defect locations from the A-scan data first.
Regarding claim 6, Townshend and Al-Hashmy teach wherein the image data generator is configured to generate the image data on an area in which the defect candidate group is included in the detection target, based on the signal data included in the defect candidate group. Townsend teaches generating images of areas of interest, such as discontinuities, for subsequent defect determination [0065]. Furthermore, Al-Hashmy teaches analyzing images of areas of interest to determine defects, and Al-Hashmy specifies that the images can include B-scan or C-scan data [0039]. However, Townshend and Al-Hashmy do not teach the ultrasonic flaw-detection system of claim 5. Thus, claim 6 is objected to due to its dependence on claim 5.
Claims 14 and 15 are objected to for the same reasons as indicated above for claims 5 and 6, respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Posilović et al. (Generative adversarial network with object detector discriminator for enhanced defect detection on ultrasonic B-scans, Neurocomputing, 459, 361-369.) teaches methods of generating B-scan data images with defects for training machine learning models, such as YOLOv3 and SSD, which can detect and classify defects in ultrasonic B-scans.
Chen et al. (Deep Learning for the Detection and Recognition of Rail Defects in Ultrasound B-Scan Images. Transportation Research Record: Journal of the Transportation Research Board, 2675(11), 888-901.) teaches utilizing a YOLOv3 machine learning model for detecting and classifying defects in B-scan ultrasound images.
Gou et al. (CN 110988140 B) utilizes a R-CNN for identifying fatigue cracks in B-scan and C-scan images.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, JENNIFER MEHMOOD, can be reached at (571)272-2976. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/JENNIFER MEHMOOD/ Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights