DETAILED ACTION
This action is in response to communications filed 09/30/2025. Claims 1-20 have been amended. Claim 21 has been cancelled. No new claims have been added. Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Abstract
Applicant has amended the abstract in response to various informalities.
The abstract has been sufficiently amended to address issues cited in the previous office action. Accordingly, the objection to the Abstract has been withdrawn
Specification
Applicant has amended the specification in response to the Examiner’s recommendations regarding subjectivity introduced when using terms such as “we” in the disclosure.
The specification has been amended appropriately and therefore the objections to the specification have been withdrawn.
Claim objections
Applicant has amended the claims in response to the objections set forth in the previous action.
Examiner confirms that the objections are sufficiently overcome by the amendments and accordingly the objections have been withdrawn.
Response to Arguments
Rejections under 35 U.S.C. § 112(a)
Applicant has amended the claim 13 and cancelled claim 21 in response to the previous rejection under 35 U.S.C. § 112. Applicant further argues that the specification contains a clear written description of the capability of the disclosed system to “monitor crack growth and predict remaining useful life” of the monitored structure as required by claim 13. Applicant particularly notes that the disclosed subject matter has the potential to use “the AE signals to monitor crack growth and predict remaining useful life” and provides citations to the specification noting that quantification of the crack length is important for scheduling the maintenance of the structure in which the crack growth is happening and that prior methods lacked an early warning capability. Applicant asserts that the disclosed subject matter provides a solution and further asserts that remaining useful life is predictable as understood by one having ordinary skill in the art.
Applicant arguments have been considered and are persuasive. Though the claim defines the invention in functional language specifying the desired result, the specification does sufficiently describe how the function is performed and the result is achieved.
Accordingly, the rejection of claim 13 has been withdrawn.
Rejections under 35 U.S.C. § 112(b)
Applicant has amended the claims in response to the previously set forth rejections under 35 U.S.C. § 112(b).
Examiner affirms that the amendments to the claims sufficiently overcome the prior rejections. Accordingly, rejections to the claims under 35 U.S.C. § 112(b) have been withdrawn.
Rejections under 35 U.S.C. § 101
Applicant has amended the independent claims in response to the rejections under 35 U.S.C. § 101. Applicant argues that the independent claim 1 is directed to a specific, inventive application of a machine-learned CNN model for a structural health monitoring application. Applicant agrees that in analyzing the claims under the 2019 PEG, under Step 2A Prong 1, “amended claim 1 could be construed as a judicial exception (specifically, an abstract idea), such that analysis should proceed to Step 2A Prong 2.” Applicant argues that under Step 2A, prong 2, the abstract idea is integrated into a practical application because claim 1 recites a specialized computing system as a machine-learned CNN. Applicant further argues that performing a transformation on AE data to generate a fixed-dimension visual feature map involves transforming time-series AE data into a 2D image-like format for use in the CNN and asserts that the data preparation provides a practical application for the judicial exception. Applicant lastly argues that the output of the CNN model is used to determine the crack length in real-time and asserts that the estimate is enabled by the processing of the CNN, which can then forth be used for predicting remaining useful life. Applicant asserts the claimed combination of all elements provides an inventive technical solution that improves the technical function of real-time structural health monitoring.
Applicant’s arguments have been fully considered; however, Examiner disagrees. The use of a CNN is the recitation of a general purpose computing component that is used as a tool to make a prediction (Mere Instructions to Apply an Exception (MPEP 2106.05(f))). Applicant’s admission that a CNN model is “known for image recognition and processing” further demonstrates that the CNN is functioning in its expected and ordinary capacity and is only limited in the claim by the particular input data used which relates the use of the judicial exception to a particular technological environment. There are no additional elements which demonstrate inventiveness in the operation of the CNN.
The transformation of time-series AE data into a 2D image-like format for use in the CNN is the recitation of an additional judicial exception- an abstract ideal of mental process. For example, raw time-series data from sensors may be represented as numeric values with corresponding timestamps wherein a filter or transformation can be applied to such data to produce an alternative representation of the data. This task can be performed using the human mind and pen and paper as assistive physical aids, wherein a human can take the raw data, apply transformation rules to the data, and generate an output which can be represented as a visual feature map and draw such a map on the paper. Accordingly, the inclusion of this limitation introduces additional judicial exceptions and does not integrate the other exception into a practical application.
Using the output of the CNN model to determine crack length in real-time and subsequently using the crack length to predict remaining useful life does not integrate the judicial exception(s) into a practical application because the limitation is an additional recitation of a process which can be practically performed in the human mind. For example, predicting remaining useful life based on data is an evaluation of data so as to inform a judgement of the status of the structure being monitored.
The assertion that the claimed invention improves the technical function of real-time structural health monitoring does not appear to be reflected in the claims. The claims appear to recite the improvement of a series of tasks which can be practically performed in the human mind, using pen and paper or computers as assistive physical aids. Per MPEP 2106.05(a)(II), “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology” and that additional elements must provide the inventive concept. The computing components relied upon to carry out the functions of the claimed invention are recited at a high level of generality and functioning in their normal capacity have been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)). The input and output of the CNN are understood to be the sending and receiving of data over a network, which is understood as a well-understood, routine, and conventional computer function when claimed in a merely generic manner. The recitation of an outcome of a judicial exception without providing particular details as to how the outcome is accomplished is further noted as Mere Instructions to Apply an Exception (MPEP 2106.05(f)). There do not appear to be any additional elements which described an inventive concept such that they would integrate the judicial exceptions into a practical application.
Applicant further argues that the particular transformation of AE signal data to fixed-dimension visual feature map provides for a non-conventional data processing technique for structural health monitoring that would amount to significantly more than any recited judicial exceptions.
Examiner disagrees because the data transformation step itself is a process which can be performed in the human mind or using pen and paper as assistive aids.
For the reasons stated in this response, in conjunction with the updated rejection provided in this action, the claims 1-20 remain rejected under 35 U.S.C. § 101.
Rejections under 35 U.S.C. § 103
Applicant's arguments filed 09/30/2025 with regard to the rejections under 35 U.S.C. § 103 have been fully considered but they are not persuasive.
In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). The combination of references previously presented would have made the claimed invention apparent at the time of filing largely because claimed matter relies on the teachings of the reference Giurgiutiu as disclosed in the rejection set forth. Imparting a neural network architecture and other claimed features were not unbeknownst to a person having skill in the art, as presented by the referenced matter previously set forth in the rejection.
Applicant argues that the independent claims, as amended, are not fully disclosed by the prior art of record. Applicant has amended the claims such that the scope of the claims required further search and consideration. Accordingly, a new grounds of rejection has been set forth to incorporate the teachings of Khan as stated in this action, wherein such reference particularly discloses the utilization of a CNN to make predictions on vibrational data embedded in spectrograms.
Although a new grounds of rejection has been applied, the examiner has determined a response necessary for the portion of the remarks wherein the references applied in the prior rejection of record are still being relied upon in the new grounds of rejection to teach or suggest the subject matter being challenged in the applicant’s arguments.
For the limitation for real-time structural health monitoring of a metallic sheet structure, Giurgiutiu discloses such limitations fully (See at least (Giurgiutiu, ¶88); (Giurgiutiu, ¶99); (Giurgiutiu, ¶244); (Giurgiutiu, ¶100); (Giurgiutiu, ¶123)). For the limitation performing a transformation on the AE signal data to generate a fixed-dimension visual feature map;, Khan is relied upon to disclose the claimed feature (See at least (Khan, Page 593, Col 1, ¶1); (Khan, Page 590, Col 1, ¶1) ). For the limitation determining a characteristic dominant frequency of a crack-length-dependent standing wave pattern, applicant argues that Giurgiutiu does not sufficiently disclose the claimed matter because the relied upon reference points to a plurality of dominant frequency peaks and allegedly there is nothing to isolate a characteristic dominant frequency. Examiner disagrees. The claim requires at least one dominant frequency be recognized for utilization but does not impose particular limitations that require only one frequency be utilized or isolated. The dominant peaks in the frequency spectrum are characteristic of the behavior and so accordingly a characteristic frequency, under broadest reasonable interpretation, could be read to encompass plural dominant frequency peaks. For the limitation as an output of the machine-learned CNN model, quantitatively determining the crack length of the crack generating the AE signal data; Giurgiutiu is relied upon to disclose the determination of crack length (See at least (Giurgiutiu, Figure 37); (Giurgiutiu, ¶91)). Khan is relied upon to disclose the utilization of a CNN model to output predictions (See at least (Khan, Page 590, col 1, ¶1); (Khan, Page 590, Col 2, ¶3)).
Given that the new grounds of rejection, necessitated by amendment, covers the totality of the claimed invention in the instant claims, for the reasons stated in this response, in conjunction with the updated rejection of this action, claims 1-20 remain rejected under 35 U.S.C. § 103.
Claim Objections
Claim 20 is objected to because of the unnecessary duplicate recitation of “machine-learned convolutional neural network (CNN) model” which is introduced in lines 10-11 and reintroduced in lines 14-15.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 20 include the recitation of the phrase “structure potentially having a fatigue crack” renders the claim indefinite. The word “potentially” does not clearly define the scope of the claim because it is unclear whether the limitation(s) following the phrase are part of the claimed invention.
Dependent claims 2-19 incorporate the deficiency of claim 1 and are therefore rejected under the same rationale.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The following section follows the 2019 Patent Eligibility Guidance (PEG) for analyzing subject matter eligibility:
Step 1 - Statutory Category:
Step 1 of the PEG analysis entails considering whether the claimed subject matter falls within the four statutory categories of patentable subject matter identified by 35 U.S.C. 101 (process, machine, manufacture, or composition of matter).
Step 2A Prong 1 - Judicial exception:
In Step 2A Prong 1, examiners evaluate whether the claim recites a judicial exception (an abstract idea, law of nature, or a natural phenomenon).
Step 2a Prong 2 - Integration into a practical application:
If claims recite a judicial exception, the claim requires further analysis in Step 2A Prong 2. In Step 2A Prong 2, examiners evaluate whether the claim as a whole integrates the exception into a practical application.
Step 2B - Significantly More:
If the additional elements identified in Step 2A Prong 2 do not integrate the exception into a practical application, then the claim is directed to the recited judicial exception and requires further analysis under Step 2B- Significantly More.
As noted in the MPEP 2106.05(II): The identification of the additional element(s) in the claim from Step 2A Prong 2, as well as the conclusions from Step 2A Prong 2 on the considerations discussed in MPEP 2106.05(a) -(c), (e), (f), and (h) are to be carried over. Claim limitations identified as Insignificant Extra-Solution Activities are further evaluated to determine if the elements are beyond what is well -understood, routine, and conventional (WURC) activity, as dictated by MPEP 2106.05(II).
Independent Claims:
Claim 1:
Step 1: Claim 1 and its dependent claims 2-19 are directed to a computing system which falls within one of the four statutory categories of a machine.
Step 2A Prong 1: Claim 1 recites a judicial exception, noted in bold:
and to quantitatively estimate crack geometric features of the structure in real-time; and The claim limitation can be reasonably read to entail making a quantitative estimation of crack geometric features of a structure. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
performing a transformation on the AE signal data to generate a fixed- dimension visual feature map; The claim limitation can be reasonably read to entail transforming AE signal data so as to create a visual feature map. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. For example, a human being can take corresponding data values from the signal and transform them so as to create a visual feature map by drawing such a feature map using pen and paper and make the transformation either completely in the human mind or using assistive aids. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
determining a characteristic dominant frequency of a crack-length-dependent standing wave pattern resulting from AE energy generated at one crack tip and traveling to the other crack tip of a crack formed in the monitored structure; The claim limitation can be reasonably read to entail evaluating a standing wave pattern so as to make a judgement of the characteristic dominant frequency. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
quantitatively determining the crack length of the crack generating the AE signal data; and. The claim limitation can be reasonably read to entail evaluating the signal data so as to make a quantitative judgement on the crack length associated with the signal. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
using the quantitative crack length determination to monitor crack growth and predict remaining useful life of the monitored structure. The claim limitation can be reasonably read to entail observing the quantified crack length so as to inform predictive judgements regarding remaining useful life of the structure. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
Therefore, the claim recites a judicial exception.
Step 2A Prong 2: Additional elements were identified and are noted in italics.
one or more processors; and- This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
one or more non-transitory computer-readable media that collectively store:- This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
a machine-learned convolutional neural network (CNN) model configured to - This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
receive a time- frequency representation of high-frequency Acoustic Emission (AE) signal data sensed from a structure - This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering and Field of Use and Technological Environment (MPEP 2106.05(h)) for linking the use of the judicial exception to the particular technological environment of acoustic emissions data on a structure
instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising: - This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
inputting the fixed-dimension visual feature map – This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering
into the machine- learned CNN model;- This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
as an output of the machine-learned CNN model - This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) as mere data outputting
The courts have found that merely including instructions to implement an abstract idea on a computer or merely using a computer as a tool to perform an abstract idea (Mere Instructions to Apply an Exception (MPEP 2106.05(f))); adding insignificant extra- solution activity to the judicial exception (Insignificant Extra Solution Activity (MPEP 2106.05(g))); and generally linking the use of a judicial exception to a particular technological environment or field of use (Field of Use and Technological Environment (MPEP 2106.05(h))) does not integrate the judicial exception into a practical application.
When viewed independently and within the claim as a whole, the additional element does not appear to integrate the judicial exception into a practical application.
Step 2B: As discussed in Step 2A Prong 2, additional elements were identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) which must be further evaluated to determine if they are beyond WURC activities. Additional elements identified otherwise and conclusions from Step 2A Prong 2 are carried over for evaluating if the claim, as a whole, amounts to an inventive concept that is significantly more than the judicial exception:
receive a time- frequency representation of high-frequency Acoustic Emission (AE) signal data sensed from a structure – This limitation has been identified as the insignificant extra solution activity of mere data gathering. Under broadest reasonable interpretation, the limitation includes receiving and transmitting data over a network. This computer functionality has been recognized by the courts as well-understood, routine, and conventional activity when claimed in a merely generic manner.
inputting the fixed-dimension visual feature map – This limitation has been identified as the insignificant extra solution activity of mere data gathering. Under broadest reasonable interpretation, the limitation includes receiving and transmitting data over a network. This computer functionality has been recognized by the courts as well-understood, routine, and conventional activity when claimed in a merely generic manner.
as an output of the machine-learned CNN model – This limitation has been identified as the insignificant extra solution activity of mere data gathering. Under broadest reasonable interpretation, the limitation includes receiving and transmitting data over a network. This computer functionality has been recognized by the courts as well-understood, routine, and conventional activity when claimed in a merely generic manner.
The courts have found that simply appending insignificant extra solution activities that are well-understood, routine, and conventional activities to the judicial exception does not qualify the limitations as “significantly more” than the recited judicial exception. The remaining additional elements were identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) and Field of Use and Technological Environment (MPEP 2106.05(h)), as stated previously. The courts have found that merely using generic computer or computer components as a tool to perform a mental process and generally linking the use of a judicial exception to a particular technological environment does not qualify the limitations as “significantly more” than the recited judicial exception.
With the additional elements viewed independently and as part of the ordered combination, the claim as a whole does not appear to amount to significantly more than the recited judicial exception because the claim is using generic computing components recited at a high level of generality and functioning in their normal capacity in conjunction with well-understood, routine, and conventional activity to enable the performance of a task that can practically be performed within the human mind or using pen and paper as an assistive physical aid. Therefore, the claim does not include additional elements, alone or in combination that are sufficient to amount to significantly more than the recited judicial exception.
Conclusion: Based on this rationale, the claim has been deemed to be ineligible subject matter under 35 U.S.C. 101.
Claim 20:
Step 1: Claim 20 is directed to a method which falls within one of the four statutory categories of a process.
Step 2A Prong 1: Claim 20 recites a judicial exception, noted in bold:
performing a transformation on the AE signal data to generate a fixed-dimension visual feature map; The claim limitation can be reasonably read to entail transforming AE signal data so as to create a visual feature map. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. For example, a human being can take corresponding data values from the signal and transform them so as to create a visual feature map by drawing such a feature map using pen and paper and make the transformation either completely in the human mind or using assistive aids. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
to quantitatively estimate crack geometric features of the structure in real-time; The claim limitation can be reasonably read to entail making a quantitative estimation of crack geometric features of a structure. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
quantitatively determining, … the crack length of the crack generating the AE signal data; and The claim limitation can be reasonably read to entail evaluating the signal data so as to make a quantitative judgement on the crack length associated with the signal. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
using the quantitative crack length determination to monitor crack growth and predict remaining useful life of the monitored structure. The claim limitation can be reasonably read to entail observing the quantified crack length so as to inform predictive judgements regarding remaining useful life of the structure. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim limitation includes the recitation of the judicial exception of abstract ideas of a mental process.
Therefore, the claim recites a judicial exception.
Step 2A Prong 2: Additional elements were identified and are noted in italics.
obtaining, …, detected Acoustic Emission (AE) signal data from sensors used with an associated structure to be monitored;- This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering. The limitation has been further identified as Field of Use and Technological Environment (MPEP 2106.05(h))
… by a computing system comprising one or more computing devices, …;- This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
inputting, …, the fixed-dimension visual feature map into a machine-learned convolutional neural network (CNN) model configured to receive AE signal data sensed from a structure and - This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering. The limitation has further been identified as Field of Use and Technological Environment (MPEP 2106.05(h))
…by the computing system…- This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
receiving, …, as an output of the machine-learned convolutional neural network (CNN) model, a characteristic dominant frequency of a crack-length-dependent standing wave pattern resulting from AE energy generated at one crack tip and traveling to the other crack tip of a crack formed in the monitored structure;- This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data outputting. The limitation has further been identified as Field of Use and Technological Environment (MPEP 2106.05(h)).
by the computing system This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f))
as an output of the machine-learned CNN model This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data outputting
The courts have found that merely including instructions to implement an abstract idea on a computer or merely using a computer as a tool to perform an abstract idea (Mere Instructions to Apply an Exception (MPEP 2106.05(f))); adding insignificant extra- solution activity to the judicial exception (Insignificant Extra Solution Activity (MPEP 2106.05(g))); and generally linking the use of a judicial exception to a particular technological environment or field of use (Field of Use and Technological Environment (MPEP 2106.05(h))) does not integrate the judicial exception into a practical application.
When viewed independently and within the claim as a whole, the additional element does not appear to integrate the judicial exception into a practical application.
Step 2B: As discussed in Step 2A Prong 2, additional elements were identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) which must be further evaluated to determine if they are beyond WURC activities. Additional elements identified otherwise and conclusions from Step 2A Prong 2 are carried over for evaluating if the claim, as a whole, amounts to an inventive concept that is significantly more than the judicial exception:
obtaining, …, detected Acoustic Emission (AE) signal data from sensors used with an associated structure to be monitored;- This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering, as stated previously. Under broadest reasonable interpretation, this claim limitation encompasses transmitting and receiving data over a network. Transmitting and receiving data over a network have been recognized by the courts as computer functions that are well-understood, routine, and conventional activities when claimed in a generic manner.
inputting, …, the fixed-dimension visual feature map into a machine-learned convolutional neural network (CNN) model configured to receive AE signal data sensed from a structure and - This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data gathering, as stated above. Under broadest reasonable interpretation, this claim limitation encompasses transmitting and receiving data over a network. Transmitting and receiving data over a network have been recognized by the courts as computer functions that are well-understood, routine, and conventional activities when claimed in a generic manner.
receiving, …, as an output of the machine-learned convolutional neural network (CNN) model, a characteristic dominant frequency of a crack-length-dependent standing wave pattern resulting from AE energy generated at one crack tip and traveling to the other crack tip of a crack formed in the monitored structure;- This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data outputting, as noted previously. Under broadest reasonable interpretation, this claim limitation encompasses transmitting and receiving data over a network. Transmitting and receiving data over a network have been recognized by the courts as computer functions that are well-understood, routine, and conventional activities when claimed in a generic manner.
as an output of the machine-learned CNN model This limitation has been identified as Insignificant Extra Solution Activity (MPEP 2106.05(g)) of mere data outputting. Under broadest reasonable interpretation, this claim limitation encompasses transmitting and receiving data over a network. Transmitting and receiving data over a network have been recognized by the courts as computer functions that are well-understood, routine, and conventional activities when claimed in a generic manner.
The courts have found that simply appending insignificant extra solution activities that are well-understood, routine, and conventional activities to the judicial exception does not qualify the limitations as “significantly more” than the recited judicial exception. The remaining additional elements were identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) and Field of Use and Technological Environment (MPEP 2106.05(h)), as stated previously. The courts have found that merely using generic computer or computer components as a tool to perform a mental process and generally linking the use of a judicial exception to a particular technological environment does not qualify the limitations as “significantly more” than the recited judicial exception.
With the additional elements viewed independently and as part of the ordered combination, the claim as a whole does not appear to amount to significantly more than the recited judicial exception because the claim is using generic computing components recited at a high level of generality and functioning in their normal capacity in conjunction with well-understood, routine, and conventional activity to enable the performance of a task that can practically be performed within the human mind or using pen and paper as an assistive physical aid. Therefore, the claim does not include additional elements, alone or in combination that are sufficient to amount to significantly more than the recited judicial exception.
Conclusion: Based on this rationale, the claim has been deemed to be ineligible subject matter under 35 U.S.C. 101.
Dependent Claims:
Examiner notes limitations identified as judicial exceptions are indicated in italicized bold and limitations identified as additional elements are indicated using italics.
Claim 2
Step 1: Regarding dependent claim 2, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 2 additionally recites the limitation the determining operations include detecting peaks in a detected frequency spectrum that shift as crack length changes., which can reasonably be read to entail observing and evaluating AE waveforms to make an assessment of the presence of peaks in a particular frequency spectrum over time. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 2 additionally recites the limitation wherein the one or more processors are further configured so that. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 3
Step 1: Regarding dependent claim 3, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 3 additionally recites the limitation learns to predict crack length directly from individual AE signal data signals, for estimating in real-time the crack length information from high-frequency AE signal waveforms during fatigue crack growth., which can reasonably be read to entail observing and evaluating AE waveforms to make an assessment and judgment on the data in order to determine a length value. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process
Step 2A Prong 2: Claim 3 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 4
Step 1: Regarding dependent claim 4, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 4 additionally recites the limitation estimates fatigue crack length in sheet metal structures using the crack length information contained in the high- frequency AE signal signatures., which can reasonably be read to entail observing AE signal signatures for information and evaluating the information to estimate crack length. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 4 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 5
Step 1: Regarding dependent claim 5, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 5 additionally recites the limitation uses physics-based modeling to generate synthetic datasets for training Al algorithms., which can reasonably be read to entail using mathematical calculations to generate a dataset, since physic-based modeling is understood to leverage established mathematical equations to simulate and predict behavior of systems. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. Additionally, this task can be performed within the human mind or using a pen and paper as an assistive physical aid and therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 5 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application.
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 6
Step 1: Regarding dependent claim 6, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 6 additionally recites the limitation comprises finite element modeling (FEM) simulation conducted to identify the correlation between the AE signal and crack length during a fatigue crack growth event., which can reasonably be read to entail using finite element methods to produce numeric values and subsequently observe and evaluate the numeric values in order to determine how AE signals are correlated to crack length. Finite element methods are mathematical-based modeling methods that utilize mathematical calculations to derive outcomes. The correlation between numeric data sets is additionally the recitation of a mathematical relationship. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. A human is capable of performing finite element modeling methods using pen and paper as an assistive physical aid and a human is further capable of observing and evaluating data to make a judgement as to how data is correlated. The claim limitation includes the utilization of a simulation to perform part of the abstract idea, wherein a simulation is understood to be a generic computing component recited at a high level of generality. The courts do not distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. Using a computer as a tool to perform a mental process still equates to the recitation of a mental process. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process
Step 2A Prong 2: Claim 6 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 7
Step 1: Regarding dependent claim 7, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 7 additionally recites the limitation comprises fatigue crack growth source modeling due to a crack growth event modeled using the dipole moment excitation concept., which can reasonably be read to entail utilizing physics-based mathematical calculations to mathematically model the behavior of the source which generates fatigue crack growth. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. Additionally, applying physics concepts to mathematical equations to derive numeric outputs can practically be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the additional recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 7 additionally recites the limitation , wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application.
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 8
Step 1: Regarding dependent claim 8, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 8 does not recite any additional judicial exceptions
Step 2A Prong 2: Claim 8 additionally recites the limitation wherein the fatigue crack growth event comprises self-equilibrating dipole forces acting at one of the opposing crack tips. This limitation has been identified as Field of Use and Technological Environment (MPEP 2106.05(h)) because the limitation is generally linking the judicial exception(s) to a particular environment. The courts have ruled generally linking an abstract idea to a particular technological environment or field of use does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application.
Step 2B: The courts have found that limitations that amount to generally linking the use of a judicial exception to a particular technological environment or field of use are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 9
Step 1: Regarding dependent claim 9, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 9 additionally recites the limitation so that the surface strain (Exx and Eyy) captured by a PWAS sensor is extracted from FEM simulation so that the … model learns wavefield patterns due to fatigue crack growth., which can reasonably be read to entail performing a dipole force calculation and further evaluating the strain result of the calculation to associate patterns of wavefields that correspond to fatigue events and crack growth. The inclusion of force calculations is the explicit recitation of a mathematical calculation and therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. A human being is capable of evaluating numerical data and observing patterns in data to infer relationships. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. The claim includes the utilization of a machine-learned AI-enabled technology neural network architecture model to perform the recited mental process. Using generic computing components recited at a high level of generality to perform a mental process still amounts to the recitation of a mental process. Therefore, this claim includes the additional recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 9 additionally recites the limitations wherein the one or more processors are further configured and machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 10
Step 1: Regarding dependent claim 10, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 10 additionally recites the uses finite element modeling using the moment tensor concept for achieving prediction of how crack length values affect the high-frequency content of AE signals. which can reasonably be read to entail utilizing the numerical representation of a physical phenomenon to evaluate how crack length values affect frequency data of AE signals. The moment tensor concept is understood to leverage a mathematical function and therefore this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. Additionally, the numeric values derived from utilizing the moment tensor concept can be observed and evaluated in order to enable the prediction of how crack length values and high frequency content is related. The relationship between the length and the content of the signals is additionally the recitation of a mathematical relationship. In addition to the claim including the recitation of mathematical concepts, the claim further includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 10 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application.
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 11
Step 1: Regarding dependent claim 11, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 11 additionally recites the limitation uses adaptation of the three-dimensional (3D) moment-tensor concept from geophysics to enable the prediction of AE signals in thin-plates using guided-wave theory., which can reasonably be read to entail utilizing the numerical representation of a physical phenomenon for the moment tensor in conjunction with mathematical concepts of guided wave theory to enable a prediction of AE signals. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept. The prediction can furthermore be performed practically within the human mind or using a pen and paper as an assistive physical aid, as a human is capable of evaluating mathematical equations and inferring information to enable predictions. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process
Step 2A Prong 2: Claim 11 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 12
Step 1: Regarding dependent claim 12, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 12 additionally recites the limitation determines a proportional relation between the crack length and peaks in the frequency spectrum of the AE signal., which can reasonably be read to entail observing the peaks of a frequency spectrum with regard to crack length to make a judgment as to how the elements are related. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Because the peaks in the frequency spectrum can be quantified and the crack length is additionally a numeric value, determining the proportional relationship between the elements is furthermore the recitation of a mathematical relationship. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept.
Step 2A Prong 2: Claim 12 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 13
Step 1: Regarding dependent claim 13, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 13 additionally recites the limitation uses AE signals to monitor crack growth and predict remaining useful life of the monitored structure., which can reasonably be read to entail evaluating crack growth and making a judgement as to the remaining useful life of the structure based on the evaluation. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 13 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 14
Step 1: Regarding dependent claim 14, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 14 does not recite any additional judicial exceptions.
Step 2A Prong 2: Claim 14 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model is tuned so that predictive AE models achieve concurrence with experimentally observed AE signals. This limitation has been identified as Field of Use and Technological Environment (MPEP 2106.05(h)) because the limitation merely describes the particular technological environment that the abstract idea is performed in. The courts have ruled generally linking the judicial exception to a particular technological environment does not integrate the judicial exception into a practical application. Claim 14 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned Al-enabled technology neural network architecture model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to generally linking the use of the judicial exception to a particular technological environment or field of use and that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 15
Step 1: Regarding dependent claim 15, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 15 additionally recites the limitation makes selection of representative AE signal features in time domain and frequency domain to enable tuning of the predictive AE models. which can reasonably be read to entail evaluating AE signal features in time and frequency domains to determine and choose which representative features best enable tuning of predictive models. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 15 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 16
Step 1: Regarding dependent claim 16, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 16 additionally recites the limitation sifts through experimental AE signals datasets to identify dominant trends correlated with crack length information., which can reasonably be read to entail observing AE signals datasets to infer patterns within the data that correspond to crack length information. This task can be performed within the human mind or using a pen and paper as an assistive physical aid. Therefore, this claim includes the recitation of the judicial exception of abstract ideas of a mental process.
Step 2A Prong 2: Claim 16 additionally recites the limitation wherein the one or more processors are further configured so that the machine-learned CNN model. . This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 17
Step 1: Regarding dependent claim 17, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 17 does not recite any additional judicial exceptions.
Step 2A Prong 2: Claim 17 additionally recites the limitation wherein the machine-learned CNN model comprises an AlexNet convolutional neural network (CNN). This limitation has been identified as Field of Use and Technological Environment (MPEP 2106.05(h)) because the limitation further describes the technological environment in which the abstract ideas are executed. The courts have ruled generally linking the judicial exception to a particular technological environment or field of use does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to generally linking the recited abstract ideas into a particular technological environment or field of use are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 18
Step 1: Regarding dependent claim 18, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 18 additionally recites the limitation so that a Choi-Williams transform of the acoustic emission signals is cropped and augmented to fit 227 x 227-pixel criteria before being entered into an input layer of the AlexNet convolutional neural network architecture, which can reasonably be read to entail applying the Choi-Williams distribution, which is understood to be mathematical technique, to data. Organizing information and manipulating information through mathematical correlations is the recitation of a mathematical relationship. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept.
Step 2A Prong 2: Claim 18 additionally recites the limitation wherein the one or more processors are further configured. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception.
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim 19
Step 1: Regarding dependent claim 19, the judicial exception of independent claim 1 is further incorporated. The claim falls within the corresponding statutory category as stated previously.
Step 2A Prong 1: Claim 19 additionally recites the limitation for training its neural connections by backpropagating error and adjusting connection weights following standard steepest gradient descent., wherein backpropagation and steepest gradient descent are understood to be mathematical calculations. Therefore, this claim includes the recitation of the judicial exception of abstract ideas as a mathematical concept
Step 2A Prong 2: Claim 19 additionally recites the limitation wherein the machine-learned CNN model comprises neural network architecture following a standard multilayer perception model. This limitation has been identified as Mere Instructions to Apply an Exception (MPEP 2106.05(f)) because the claim is invoking computers or other machinery merely as a tool to perform an existing process. The courts have ruled merely using a computer as a tool to perform the abstract idea does not integrate the judicial exception into a practical application. With the additional element viewed in conjunction with the other limitations, the claim as a whole does not appear to integrate the judicial exception into a practical application
Step 2B: The courts have found that limitations that amount to mere instructions to implement an abstract idea on a computer are not enough to qualify the claim as significantly more than the abstract idea. Therefore, the claim does not include additional elements, alone or in the ordered combination that are sufficient to amount to significantly more than the recited judicial exception
This claim is not eligible subject matter under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-9, 12-13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Giurgiutiu et al. (US Patent Publication No. US 2020/0408720 A1), hereinafter referred to as Giurgiutiu in view of Khan et al (Khan, A., Ko, D., Lim, S., and Kim, H., “Structural vibration-based classification and prediction of delamination in smart composite laminates using deep learning neural network”, 2019, Composites Part B 161, pp 586-594), hereinafter referred to as Khan.
Regarding claim 1, Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) A computing system for real-time structural health monitoring of a metallic sheet structure potentially having a fatigue crack extending between opposing crack tips, comprising: A methodology is described as being carried out using a system comprising a personal computer connected to other equipment ((Giurgiutiu, ¶88) " Acoustic emission testing can be performed in the field with portable instruments or in a stationary laboratory setting. Typically, systems contain a sensor, preamplifier, filter, and amplifier, along with measurement, display, and storage equipment (e.g. oscilloscopes, voltmeters, and personal computers). "). The methodology is described as being used in a SHM application ((Giurgiutiu, ¶) " The current disclosure may be used in acoustic emission (AE) for nondestructive evaluation (NDE), and structural health monitoring (SHM) applications. This invention provides a novel methodology to indicate early signs of fatigue cracks."). The method is described as being executed on real-time signals and further described as being performed so as to identify fatigue cracks on aircraft grade aluminum, which is understood to be a metallic sheet structure ((Giurgiutiu, ¶244) " The fatigue experiments were designed to capture the real-time AE signals. Aircraft grade aluminum Al-2024 T3 test coupons of 100-mm-wide, 300-mm-long and 1-mmthick dimension were used."); ((Giurgiutiu, ¶100) " The crack can be evaluated online, during normal operation conditions. "). The fatigue crack is characterized by crack tips on either end ((Giurgiutiu, ¶123) " The current disclosure proposes that the observed peaks are due to local resonances of the crack due to standing waves pinned between the crack tips.")
one or more processors; and ((Giurgiutiu, ¶91) " The acoustic emission signals received from a growing crack are processed pursuant to the current disclosure to yield information about the crack length and other geometric properties.")(( Giurgiutiu, ¶136) " The acoustic emission signals received from a growing crack may be processed to yield information about the crack length and other geometric properties. "); ((Giurgiutiu, ¶100) " The crack can be evaluated online, during normal operation conditions. ")
one or more non-transitory computer-readable media that collectively store: ((Giurgiutiu, ¶88) " Typically, systems contain a sensor, preamplifier, filter, and amplifier, along with measurement, display, and storage equipment (e.g. oscilloscopes, voltmeters, and personal computers). "); ((Giurgiutiu, ¶89) " Following completion of this process, the signal travels to the acoustic system mainframe and eventually to a computer or similar device for analysis and storage. ")
[[a machine-learned convolutional neural network (CNN) model configured to receive]] a time- frequency representation of high-frequency Acoustic Emission (AE) signal data sensed from a structure ((Giurgiutiu, ¶95) " The acoustic emission signals are then analyzed by frequency analysis and time-frequency analysis to extract signal signatures that are associated with the fatigue crack geometric features. "). See also Figure 29 which depicts a time-frequency representation of data. and to quantitatively estimate crack geometric features of the structure in real-time; and ((Giurgiutiu, ¶96) " Frequency and time-frequency analysis are applied to the extract the frequency response and the propagation wave components of the acoustic emission signals. By using the results obtained in the above described vibration resonance experiments, signal signatures that are associated with the fatigue crack resonance modes are extracted. Fatigue crack generated acoustic emission waves were studied with analytical simulation, numerical simulation, and experiments. Using the analytical and numerical simulation models validated by the above described study, the structural geometric features related signal signatures are predicted. A library of features in the acoustic emission waveforms can be used to identify fatigue crack geometric features. "). Crack length is described as a geometric feature ((Giurgiutiu, ¶15) " Further, the method may comprise identifying at least one geometric feature of the fatigue crack from analysis of the at least one secondary emission wave. Still further, the method may include identifying fatigue crack length and crack tip locations as part of the at least one geometric feature. "). The methodology is described as being performed online during normal conditions, thereby indicating real-time functionality ((Giurgiutiu, ¶100) " The crack can be evaluated online, during normal operation conditions.")
instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising: The methodology is described as being performed on a computer, which would be understood by a person having skill in the art to store instructions and a processor that enable the computer to perform operations ((Giurgiutiu, ¶88) " Typically, systems contain a sensor, preamplifier, filter, and amplifier, along with measurement, display, and storage equipment (e.g. oscilloscopes, voltmeters, and personal computers)."). The methodology is further described as performing functions automatically, thereby indicating a computerized instruction ((Giurgiutiu, ¶198) " Once the AE hits were sorted based on the similarity of waveform signatures (timedomain signals and frequency spectra), other similarities, for example, the load level, duration, hit amplitude of the waveform were automatically obtained.");( Giurgiutiu (, ¶268) "In a further embodiment, the current disclosure is directed to using original fingerprints of fatigue crack generated acoustic emission waveforms. These fingerprints may be used as a standard to distinguish crack-generated and non-crack generated AE signals. Each fingerprint had a particular time domain signal pattern and unique frequency spectrum. The current disclosure has discovered that the huge amount of fatigue crack generated AE hits can be sorted into groups based on these fingerprints. These fingerprints explained the complex fatigue crack growth mechanisms which would enable proper fatigue damage monitoring solutions for the safety of public infrastructures.") ((Giurgiutiu, ¶270) " Further, AE analysis software algorithms may be developed based on the fingerprints ")
[[performing a transformation on the AE signal data to generate a fixed- dimension visual feature map;]]
[[inputting the fixed-dimension visual feature map into the machine- learned CNN model;]]
determining a characteristic dominant frequency of a crack-length-dependent standing wave pattern resulting from AE energy generated at one crack tip and traveling to the other crack tip of a crack formed in the monitored structure; Dominant frequency peaks are located at various frequencies wherein the fact that particular frequencies are distinguished from all those tested in the spectrum indicates that the particular dominant frequencies are characteristic of the phenomenon ((Giurgiutiu, ¶174) " The firstAE hit of a particular cluster is shown in FIG. 18. The frequency spectrum shows that the dominant frequency peaks are located at 30, 60, 200 kHz."). The standing wave modes are described as being dependent on crack length ((Giurgiutiu, ¶187) " The AE signals may differ slightly at various crack lengths because of any standing wave modes along the crack length. A single parameter from the frequency spectrum, for example, centroid may help to quantify the groupings. ")
[[as an output of the machine-learned CNN model,]] quantitatively determining the crack length of the crack generating the AE signal data; and Figure 37 shows a flow chart of determining crack length from a recorded AE signal, wherein a length value is understood to be a numeric value that would be a quantification. The AE signal is described as being generated by a fatigue crack ((Giurgiutiu, ¶91) "Fatigue crack generated acoustic emission waves are characterized by analytical simulation, numerical simulation, and experiments ")
using the quantitative crack length determination to monitor crack growth and predict remaining useful life of the monitored structure. Information of AE testing is described as being useful for assessing structural integrity and monitoring dynamic changes in a material (Giurgiutiu, ¶82-86); ((Giurgiutiu, ¶271) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "). Failure of a structure is described as occurring when cracks reach a critical length, thereby indicating that the length of the crack is a parameter used to determine if failure has occurred (which would indicate a prediction of no remaining useful life) ((Giurgiutiu, ¶8) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "); ((Giurgiutiu, ¶270) " This explains the complex nature of the progressive fatigue damage such as: crack closure; brittle growth of the crack; blunting growth of the crack; and near-critical fast growth of the crack."); ((Giurgiutiu, ¶13) " What is needed in the industry is acoustic emission and ultrasonic testing to determine if structures possess faults and to gauge the extent of such faults, such as evaluation of the crack geometric features. "). Crack geometric features include crack length ((Giurgiutiu, ¶15) "Further, the method may comprise identifying at least one geometric feature of the fatigue crack from analysis of the at least one secondary emission wave. Still further, the method may include identifying fatigue crack length and crack tip locations as part of the at least one geometric feature."). The signal fingerprints are also disclosed as being useful for damage diagnosis and prognosis ((Giurgiutiu, ¶270) " Further, the amplitude and frequency contents of the fingerprints may be used for the fatigue damage diagnosis, quantification, and prognosis")
Giurgiutiu does not disclose the utilization of a CNN for enabling the prediction of crack length; however; Khan discloses a machine-learned convolutional neural network (CNN) model configured to receive spectrograms as input. Figure 6 depicts a proposed architecture of a network, wherein an input is shown, demonstrating that the CNN is configured to receive such input ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN)."). The input is described as a spectrogram of vibrational data((Khan, Page 590, Col 1, ¶2) "Section 4.2 discusses the construction of vibration-based spectrograms for input to CNN, section 4.3 focuses on network architecture, section 4.4 presents the classification of delaminations from the proposed network architecture, and section 4.5 discusses the prediction of labels for new cases of delaminations."); See also Table 2 that describes the Input as a 256x256 FFT Spectrum Image. Spectrograms are described as capturing the time and frequency domain information of a signal ((Khan, Page 590, Col 2, ¶1-2) ". Fig. 5 shows the preprocess of transforming structural vibration into vibration spectrograms, which is explained in the following steps: 1. To capture the time and frequency domain information of the signal, a sliding window of 512 data points is selected that is moved from left to right each time step (0.0001 s), and Fast Fourier Transform (FFT) is computed for each sliding window. 2. The sliding window of 512 data points moved over each time step (0.0001 s) and the corresponding FFT result in a matrix of size 512 × 256 for each transient history. 3. The matrix of each transient history is transformed into a spectrogram, and each spectrogram is split half in width, and then supplied to CNN as a two-dimensional array of size 256 × 256 × 2. The process of transforming the transient histories of the healthy and delaminated cases into vibration-based spectrograms via STFT resulted in a total of 13,000 images that corresponded to 13 cases to be classified via CNN. In the next section, the architecture of CNN is developed to autonomously extract discriminative features from the vibration-based spectrograms, and employ those features to classify the healthy and delaminated cases into different classes.")
performing a transformation on the AE signal data to generate a fixed- dimension visual feature map; Vibration signals are subject to a transform so as to generate 2-d spectrograms which characterize the features of the data ((Khan, Page 593, Col 1, ¶1) "Random structural vibration signals are obtained for the healthy and delaminated smart composites via a piezoelectric sensor, and Short Time Fourier Transform (STFT) is employed to transform those signals into 2-dimensional spectrograms. CNN is employed to automatically extract discriminative features from the vibration-based spectrograms, and employ those features to make a distinction between the healthy and delaminated scenarios, as well between various different scenarios of delamination"). Spectrograms are described as comprising specified dimensions ((Khan, Page 590, ¶1) " To capture the time and frequency domain information of the signal, a sliding window of 512 data points is selected that is moved from left to right each time step (0.0001 s), and Fast Fourier Transform (FFT) is computed for each sliding window. 2. The sliding window of 512 data points moved over each time step (0.0001 s) and the corresponding FFT result in a matrix of size 512 × 256 for each transient history. 3. The matrix of each transient history is transformed into a spectrogram, and each spectrogram is split half in width, and then supplied to CNN as a two-dimensional array of size 256 × 256 × 2. "). A feature map is described as being obtained in a convolution operation wherein a transform of the input image is described ((Khan, Page 590, Col 1, ¶1) "Pooling or Subsampling is employed to reduce the dimensionality of the feature map obtained in the convolution operation. The convolutional and pooling layers transform the input image into a high-level feature map that is employed by the Fully-Connected layer (output layer) for the classification of the input image [42]. ").
inputting the fixed-dimension visual feature map into the machine- learned CNN model; The spectrogram is supplied as an input to the CNN with specified dimensions ((Khan, Page 590, Column 1, ¶2) "Section 4.2 discusses the construction of vibration-based spectrograms for input to CNN, section 4.3 focuses on network architecture, section 4.4 presents the classification of delaminations from the proposed network architecture, and section 4.5 discusses the prediction of labels for new cases of delaminations."); ((Khan, Page 590, Col 2, ¶1-2) "The matrix of each transient history is transformed into a spectrogram, and each spectrogram is split half in width, and then supplied to CNN as a two-dimensional array of size 256 × 256 × 2. The process of transforming the transient histories of the healthy and delaminated cases into vibration-based spectrograms via STFT resulted in a total of 13,000 images that corresponded to 13 cases to be classified via CNN.")
as an output of the machine-learned CNN model, An output classification is provided by the CNN ((Khan, Page 590, col 1, ¶1) "The convolutional and pooling layers transform the input image into a high-level feature map that is employed by the Fully-Connected layer (output layer) for the classification of the input image [42]. "); ((Khan, Page 590, Col 2, ¶3) " Fig. 6 shows that the network consists of six convolutional and pooling layers for automatically extracting discriminative features from the vibration-based spectrograms of the healthy and delaminated cases. Fully connected layer and Softmax layer is used to classify the healthy and delaminated cases on the basis of features extracted in the convolutional and pooling layers.")
Giurgiutiu is analogous art to the claimed invention because it is directed toward the same field of endeavor of characterizing crack lengths in structures using acoustic emissions. Khan is analogous to the claimed invention because it is related to the same field of endeavor of utilizing machine learning techniques for characterizing defects in structures using vibration-based data. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have combined the prior art references so as to impart a CNN approach (as in Khan) on the crack length estimation methodology (as discloses by Giurgiutiu) because some teaching, suggestion, or motivation would have led one having skill in the art to do so in order to arrive at the claimed invention. Giurgiutiu discloses the discovery of a relationship between AE data and crack length that can be characterized by time-frequency data. Khan discloses the utilization of a CNN to autonomously characterize structures based on features contained in time-frequency spectrogram data. While Giurgiutiu discloses the prediction of a crack length as a quantified value, and Khan discloses the classification of a delamination defect instead of a quantified value characterizing the defect, it would have still be obvious to combine the references in order to arrive at the claimed invention because it would be known to one having skill in the art, at the time of the filing of the invention, that a CNN tailored to a classification task could have been further modified so as to produce a numeric prediction result. This is supported by the evidence noted in Matlab (Matlab, “Convert Classification Network into Regression Network”, Available online December 2, 2020, mathworks.com), hereinafter referred to as Matlab, wherein a tutorial is given for how one would modify the capabilities of a CNN for regression purposes instead of classification. Furthermore, while Khan is directed towards the classification of delamination in composite materials and not particularly directed towards detecting crack length in metallic structures, it would have been obvious to use a CNN approach because Giurgiutiu explicitly demonstrates that a relationship between crack length and frequency response is identifiable from time-frequency data and Khan provides an automated mechanism by which to make estimates from such data sources. Khan further touts the major advantage of leveraging deep learning for autonomous extraction from raw data which is particularly advantageous for damage assessment and structural health monitoring applications ((Khan, Page 589, Col 2, ¶1) "One of the major advantages of deep learning is the autonomous extraction of discriminative features from the raw data [36]. Deep learning has been employed for the damage assessment of laminated composites [37], aircraft engines and power transformers [38], rolling element bearings [39], and the structural health monitoring of civil infrastructures [40]. The discussions in sections 1 and 3 show that the detection, quantification, and localization of delaminations in smart composite laminates from their structural vibration responses have difficulties that are inherited from the dependency of the acquired transient histories on the operating conditions, and the fact that the vibratory responses in the fundamental modes capture the global behavior of the structure"). Accordingly, given the information available in the prior art at the time the invention was filed, the combination would have been obvious to one having skill in the art.
Regarding claim 2, the proposed combination discloses The computing system according to claim 1, as stated previously. The proposed combination in further view of Giurgiutiu discloses wherein the one or more processors are further configured so that the determining operations include detecting peaks in a detected frequency spectrum that shift as crack length changes. A dynamic relationship between the crack geometry and the vibrational modes around the crack as the crack grows is acknowledged. ((Giurgiutiu, ¶113) "As the geometry changes due to the crack growth, so does the local vibrational modes around the crack. The current disclosure's aim is to understand these changing local vibrational modes and find possible relation between the AE waveforms features and the crack geometric features. The main challenge is to identify crack resonances in the collected AE signals."). Further, investigation of EA waveforms indicates that a relationship exists between the presence and number of resonance peaks and the length of the associated crack ((Giurgiutiu, ¶122) "These studies seemed to indicate a clear relationship between the presence and number of resonance peaks and the length of the crack.") A specific frequency spectrum was investigated which contained the peaks, as indicated in Figure 6.
Regarding claim 3, the proposed combination discloses The computing system according to claim 2, as stated previously. The proposed combination further in view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured [[so that the machine-learned CNN model learns to]] predict crack length directly from individual AE signal data signals, for estimating in real-time the crack length information from high-frequency AE signal waveforms during fatigue crack growth. Signal processing is performed on a single AE data signal, wherein a singular data signal contains information corresponding to crack length ((Giurgiutiu, ¶128) " These numerical investigations have revealed that the AE wave signal measured at a distance from the crack may carry information about the crack length. […] Based on our numerical investigation, the current disclosure proposes that (a) a crack exhibits specific resonances related to its length; and (b) the AE signal generated by the energy discharged at the crack tip during crack growth may contain traces of these resonances that, upon signal processing, may reveal information about the crack length."). During crack growth, AE energy discarded at the tip of the crack generates standing waves that engage the crack into local resonances, wherein the resonances are high-frequency ((Giurgiutiu, ¶133) "This embedded information is generated by the fact that the AE energy discarded at the crack tip during crack growth may generate standing waves that would engage the crack into local resonances. These resonances are of high frequency, typically hundreds of kHz and low MHz. The current disclosure proposes that these local vibration resonances would modulate the AE wave signal that travels away from the crack thus embedding crack-size information in the AE wave signal. We also hypothesize that theseAE wave signals would travel at a distance from the crack and could be capture with appropriate AE transducers that sufficiently sensitive for this task. These AE wave signals, could be processed and decoded such as to reveal the embedded crack-size information. "). Giurgiutiu indicates previous methodologies require analysis during maintenance periods ((Giurgiutiu, ¶101) " In contrast, conventional AE signal analysis methods are based on the statistical characteristics of AE hit event parameters, and can only provide qualitative estimation of the crack severity. Evaluation of the crack geometric features needs to be performed offline, during maintenance period. The current disclosure may improve the safety and the availability of critical vehicles and infrastructures, as well as reduce maintenance costs.") but that the proposed methodology can be performed online and under actual operating-conditions, thereby indicating that the monitoring and prediction can be performed in real-time ((Giurgiutiu, ¶100) " The current disclosure is founded on the science and understanding of how AE wave signals are generated by crack growth and their interaction with the crack. The signal processing method developed in this methodology extracts crack geometric features from the received AE waveforms, such as crack length and orientation. The crack can be evaluated online, during normal operation conditions. ")
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Khan discloses so that the machine-learned CNN model learns to The CNN is described as learning patterns for predictions ((Khan, Page 592, Col 2, ¶3) "Table 3 reveals that the labels predicted by the learned CNN are consistent with the physics of the problem. For example, the dynamic characteristics of the L1D1 delaminated case should be similar to either the L1D0 or L1D2 delaminated case; the predicted label is L1D0, which shows the ability of the learned network to correctly predict those cases of delamination that were not even considered in the training and crossvalidation of the network."). The CNN is trained on spectrograms to make predictions based on the data contained within the data ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN). ")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 4, the proposed combination discloses The computing system according to claim 3, as stated previously. The proposed combination further discloses in view of Giurgiutiu wherein the one or more processors are further configured [[so that the machine-learned CNN model]] estimates fatigue crack length in sheet metal structures using the crack length information contained in the high- frequency AE signal signatures. Fatigue crack length can be estimated using information contained in AE signal signatures, wherein the high-frequency information of the AE signal signatures is what contains embedded data correlated to crack length ((Giurgiutiu, ¶133) "This embedded information is generated by the fact that the AE energy discarded at the crack tip during crack growth may generate standing waves that would engage the crack into local resonances. These resonances are of high frequency, typically hundreds of kHz and low MHz. The current disclosure proposes that these local vibration resonances would modulate the AE wave signal that travels away from the crack thus embedding crack-size information in the AE wave signal. We also hypothesize that theseAE wave signals would travel at a distance from the crack and could be capture with appropriate AE transducers that sufficiently sensitive for this task. These AE wave signals, could be processed and decoded such as to reveal the embedded crack-size information. "). Experiments were performed using a thin metal specimen, understood as the equivalent of sheet metal structures. ((Giurgiutiu, ¶198) "The particular waveform signature may explain a particular behavior during the fatigue crack growth. In this particular fatigue crack growth in a thin metal specimen, nine different AE wave forms groups were identified.".)
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Khan discloses so that the machine-learned CNN model estimates values based on signal data. The CNN is described as learning patterns for predictions ((Khan, Page 592, Col 2, ¶3) "Table 3 reveals that the labels predicted by the learned CNN are consistent with the physics of the problem. For example, the dynamic characteristics of the L1D1 delaminated case should be similar to either the L1D0 or L1D2 delaminated case; the predicted label is L1D0, which shows the ability of the learned network to correctly predict those cases of delamination that were not even considered in the training and crossvalidation of the network."). The CNN is trained on spectrograms to make predictions based on the data contained within the data ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN). ")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 5, the proposed combination discloses The computing system according to claim 3, as stated previously. The proposed combination in further view of Khan discloses wherein the one or more processors are further configured so that the machine-learned CNN model uses physics-based modeling to generate synthetic datasets for training Al algorithms An electromechanically coupled dynamic model (understood as physics-based model) is used to obtain transient responses which are then used to generate spectrograms which are used for training the CNN ((Khan, Page 587, Col 1, ¶1) "In this work, an output-based data-driven technique is proposed for the global and local assessment of delamination in smart composite laminates from their low-frequency structural vibration response. Experimentally verified improved layerwise theory [31], higher order electric potential field [32], and the finite element method are employed to develop an electromechanically coupled dynamic model for the pristine and delaminated smart composite structures. Transient responses of the healthy and delaminated composites are obtained via a piezoelectric sensor by solving the dynamic model in the time domain. The transient histories are transformed into spectrograms via discrete Fast Fourier Transform (FFT) and a sliding window technique. The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN). ".); ((Khan, Page 581, Col 1, ¶3) "This section describes the development of an electromechanically coupled dynamic mathematical model for the pristine and delaminated smart composite laminates. The displacement and electric potential fields are modelled on the basis of improved layerwise theory [33] and higher-order electric potential field [32], respectively. The two fields are implemented via finite element method, and variational principles are employed to derive the governing equation of motion."); ((Khan, Page 588, Col 2, ¶2) "The transient responses are obtained by solving the electromechanically coupled equation of motion (Eq. (5)) in the time domain via Newmark's time integration algorithm [34]. For the 13 cases (i.e., healthy and 12 delaminated cases), we obtained a data set of size 13,000 × 1,001, where 13,000 refers to the responses of the 13 cases (1000 responses in each case), and 1001 represents the data points in 0.1 s of the transient response. ")
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the combined prior art references to include synthetic data generation for training because combining prior art elements according to known methods would yield predictable results. Giurgiutiu discloses the generation of experimental data and FEM data (synthetic) by which to apply to the prediction methodology but does not particularly describe using synthetic data for training AI algorithms. Khan discloses the development of an electromechanically coupled model for use in FEM simulations, wherein the data generated from leveraging these tools is used by the CNN for training (Khan, Page 588, Col 2, ¶2); (Khan, Page 591, Col 1, ¶3). One having skill would be particularly compelled to further modify the references in order to arrive at the claimed invention in this way because using synthetic data for training is a known approach in the art that enables faster and more cost-effective training to create robust predictions, especially in circumstances where real-world data is sparse.
Regarding claim 6, the proposed combination discloses The computing system according to claim 5, as stated previously. The proposed combination in further view of Giurgiutiu discloses wherein the one or more processors are further configured so that [[the machine-learned CNN model]] comprises finite element modeling (FEM) simulation conducted to identify the correlation between the AE signal and crack length during a fatigue crack growth event. FEM simulation to simulate a fatigue crack growth event was conducted to obtain the correlation of the crack length with resonance frequency, wherein the resonance frequencies are obtained from the AE signal ((Giurgiutiu, ¶219) "FIG. 33 shows the frequency content of the simulated acoustic emission in terms of out of plane displacement measured at 20 mm away from the hole. We can clearly see multiple resonances from the simulated acoustic emission signal. Upon comparing FIGS. 32 and 33, we can see that these resonance frequencies are same as the resonance frequencies associated with the crack opening motion. Therefore, we confirm through simulation that a wideband acoustic source located at the tip of a crack causes the crack to resonate and this resonance can be detected from the acoustic emission signal at a distance from the crack. Since the crack resonance frequency depends on the crack length, theoretically it is possible to detect crack length from the acoustic emission signals. The correlation of the crack length with the resonance frequency can be obtained by FE models similar to the ones presented.")
The proposed combination in further view of Giurgiutiu does not disclose; however the proposed combination in further view of Khan discloses the machine-learned CNN model leveraging FEM to identify the relationship between sensor signals and structural displacement phenomenon. ((Khan, Page 587, Col 2 ¶3-4) "From Eq. (1), the displacement field of a healthy laminated composite with N-layers can be expressed in terms of the variables u1, u2, w, w,x , w,y, 1, and 2, whereas the displacement field of a delaminated composite can be expressed in terms of the variables u1, u2, w, w,x , w,y, 1, 2, u¯ j 1 , u¯ j 2 , w¯ j , w¯ x j , , and w¯ y j , . On the other hand, the higher-order electric potential field of piezoelectric patches can be expressed in terms of the variables q 0 and Ez q . The two fields of Eqs. (1) and (2) are implemented via finite element method for a four-node plate element. The in-plane structural unknowns (u1, u2, 1, 2, u¯ j 1 , u¯ j 2 ) and electrical unknowns ( , E q z q 0 ) are interpolated via linear Lagrange interpolation function ( Nm), whereas the out-of-plane structural unknowns (w, w¯ j ) are interpolated via Hermite cubic interpolation function ( Hm, Hxm, Hym). In this way, the structural and electrical unknowns can be expressed in terms of nodal values and interpolation functions, as given by Eq. (3):")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim when modifying the CNN per the particular data and use case directed towards predicting crack length of metal structures presented by Giurgiutiu over the alternate use case for delamination in composites presented by Khan.
Regarding claim 7, the proposed combination discloses The computing system according to claim 6, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model]] comprises fatigue crack growth source modeling due to a crack growth event modeled using the dipole moment excitation concept. Dipole moment excitation is leveraged to model a fatigue crack growth source ((Giurgiutiu, ¶258) "The AE source was modeled using dipole concept suggested by Ramstad et al. To model the fatigue crack growth, the concept was extended and two dipoles were modeled along the thickness that represents a line dipole source. The illustration of the 3D FEM modeling is shown in FIG. 52. FIG. 52 shows 3D FEM for harmonic analysis: (a) top view, (b) front view, (c) dipole loading at the crack tip, and ( d) line load along the thickness."); ((Giurgiutiu, ¶118) "For wave propagation analysis, we used dipole AE sources placed on the crack tip following previous work by Ramstad and Prosser."); ((Giurgiutiu, ¶135) "To summarize, fatigue crack generated AE waves were studied with analytical simulation, numerical simulation, and experiments. Finite element method (FEM) analysis was used to model AE events due to fatigue crack growth. This was done using dipole excitation at the crack tips. Harmonic analysis was also performed on these FE models "); ((Giurgiutiu, ¶209) "Also, we place these dipoles at both the ends of the elements at the crack tips to approximate AE due to crack growth of one element length (0.25 mm). This makes the AE source as an extended source instead of a point source. By incorporating a temporal variation of the dipole strength, we simulate generation of acoustic emission from the crack tips.")
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Khan discloses the machine-learned CNN model leveraging simulated (modeled) harmonic excitations. ((Khan, Page 588, Col 2, ¶2-3 - Page 589, Col 1, ¶1-3) "In real life applications, engineering structures are subjected to random input excitations from unknown sources. To simulate the actual scenario, the pristine and 12 delaminated cases of the smart composite laminates are subjected to 1000 random harmonic excitations. The random harmonic excitations are applied in the form of voltages through the piezoelectric actuator, and the corresponding responses are measured in the form of voltage signals through the piezoelectric sensor. [[…]] Therefore, in the next section, a convolutional neural network (CNN) based methodology is proposed to deal with the current problem.")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 8, the proposed combination discloses The computing system according to claim 7, as stated previously. The proposed combination in further view of Giurgiutiu discloses wherein the fatigue crack growth event comprises self-equilibrating dipole forces acting at one of the opposing crack tips. ((Giurgiutiu, ¶208) " We model the source as equal strength dipoles distributed across the thickness of the plate (FIG. 3) approximating to a line source of acoustic emission. "); ((Giurgiutiu, ¶210) " FIG. 26 shows at (a) shows Dipoles at crack tips for simulation of acoustic emission due to crack growth of one element length. FIG. 26 at (b) illustrates distribution of dipoles across the thickness of the plate."); ((Giurgiutiu, ¶118) "For wave propagation analysis, we used dipole AE sources placed on the crack tip following previous work by Ramstad and Prosser.")
Regarding claim 9, the proposed combination discloses The computing system according to claim 8, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that the surface strain (Exx and Eyy) captured by a PWAS sensor is extracted from FEM simulation [[so that the machine-learned CNN model learns]] wavefield patterns due to fatigue crack growth. The source of the fatigue crack growth is modeled using modeled dipole forces in a simulation ((Giurgiutiu, ¶208) "We model the source as equal strength dipoles distributed across the thickness of the plate (FIG. 3) approximating to a line source of acoustic emission."). Acoustic emission recordings during a simulated fatigue test are obtained ((Giurgiutiu, ¶207) "The current disclosure seeks to simulate acoustic emission recorded during uniaxial tensile fatigue test."); See also Figure 7. The simulation is representative of experimental tests, wherein the experimental test employs a PWAS sensor ((Giurgiutiu, ¶117) “One initial aim was to simulate AE signals recorded during uniaxial tensile fatigue test in a thin-sheet specimen representative for aerospace applications. We assume that the specimen is under pure tension and the crack is fully penetrated through the specimen thickness. Therefore, we also assume symmetric emission of acoustic energy across the plate thickness. These conditions are similar to those encountered during experimental AE work."); ((Giurgiutiu, ¶221)" The current disclosure seeks to validate the simulation results with a fatigue test experiment. During fatigue tests, when the crack grows, the plate causes acoustic emissions from one of the crack tips. In our simulation, the crack surfaces are assumed to be stress free which is not the case in a fatigue crack. Therefore, to confirm the phenomenon of crack resonance due an acoustic emission source at the tip, we use a slit instead of a fatigue crack.[[..]] Then, piezoelectric wafer active sensors (PWAS) are bonded at one of the tips of the slit to emulate an acoustic source. Two PWAS transducers are bonded at the slit tip on the top and bottom surfaces of the plate."). It would be understood by one of ordinary skill in the art that a PWAS sensor captures strain. Standing wavefield plots are obtained due to fatigue crack growth in experimental setup ((Giurgiutiu, ¶132) "FIG. 8 shows resonance of the slit at multiple frequencies due to acoustic emission from PWAS (a) measured at 20 mm from the slit (b )-( e) area scan results showing standing wave field around the slit.").
The proposed combination in further view of Giurgiutiu does not disclose; however the proposed combination in further view of Khan discloses the responses of piezoelectric sensors are obtained so that the machine-learned CNN model learns displacement and electric potential fields ((Kham, Page 587, Col 1, ¶3) "This section describes the development of an electromechanically coupled dynamic mathematical model for the pristine and delaminated smart composite laminates. The displacement and electric potential fields are modelled on the basis of improved layerwise theory [33] and higher-order electric potential field [32], respectively. The two fields are implemented via finite element method, and variational principles are employed to derive the governing equation of motion."); ((Khan, Page 587, Col 2 ¶3) "From Eq. (1), the displacement field of a healthy laminated composite with N-layers can be expressed in terms of the variables u1, u2, w, w,x , w,y, 1, and 2, whereas the displacement field of a delaminated composite can be expressed in terms of the variables u1, u2, w, w,x , w,y, 1, 2, u¯ j 1 , u¯ j 2 , w¯ j , w¯ x j , , and w¯ y j , . On the other hand, the higher-order electric potential field of piezoelectric patches can be expressed in terms of the variables q 0 and Ez q ."). The CNN is described as learning patterns for predictions ((Khan, Page 592, Col 2, ¶3) "Table 3 reveals that the labels predicted by the learned CNN are consistent with the physics of the problem. For example, the dynamic characteristics of the L1D1 delaminated case should be similar to either the L1D0 or L1D2 delaminated case; the predicted label is L1D0, which shows the ability of the learned network to correctly predict those cases of delamination that were not even considered in the training and crossvalidation of the network."). The CNN is trained on spectrograms to make predictions based on the data contained within the data ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN). ")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 12, the proposed combination discloses The computing system according to claim 3, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model]] determines a proportional relation between the crack length and peaks in the frequency spectrum of the AE signal. Investigation of EA waveforms indicates that a relationship exists between the presence and number of resonance peaks and the length of the associated crack ((Giurgiutiu, ¶122) "These studies seemed to indicate a clear relationship between the presence and number of resonance peaks and the length of the crack."). The crack length is determined to depend on crack resonance frequency and the correlation can be obtained by FE models disclosed ((Giurgiutiu, ¶219) "Upon comparing FIGS. 32 and 33, we can see that these resonance frequencies are same as the resonance frequencies associated with the crack opening motion. Therefore, we confirm through simulation that a wideband acoustic source located at the tip of a crack causes the crack to resonate and this resonance can be detected from the acoustic emission signal at a distance from the crack. Since the crack resonance frequency depends on the crack length, theoretically it is possible to detect crack length from the acoustic emission signals. The correlation of the crack length with the resonance frequency can be obtained by FE models similar to the ones presented.");
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Khan discloses using the machine-learned CNN model to determine features from frequency data ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN). The confusion matrix of CNN showed physically consistent results, and an overall classification accuracy of 90.1% is obtained. The essence of the proposed technique is that it requires only low-frequency structural vibration responses for the detection and localization of delamination in smart composite laminates")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 13, the proposed combination discloses A computing system according to claim 3, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model]] uses AE signals to monitor crack growth and predict remaining useful life of the monitored structure. Information of AE testing is described as being useful for assessing structural integrity and monitoring dynamic changes in a material (Giurgiutiu, ¶82-86); ((Giurgiutiu, ¶271) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "). Failure of a structure is described as occurring when cracks reach a critical length, thereby indicating that the length of the crack is a parameter used to determine if failure has occurred (which would indicate a prediction of no remaining useful life) ((Giurgiutiu, ¶8) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "); ((Giurgiutiu, ¶270) " This explains the complex nature of the progressive fatigue damage such as: crack closure; brittle growth of the crack; blunting growth of the crack; and near-critical fast growth of the crack."); ((Giurgiutiu, ¶13) " What is needed in the industry is acoustic emission and ultrasonic testing to determine if structures possess faults and to gauge the extent of such faults, such as evaluation of the crack geometric features. "). Crack geometric features include crack length ((Giurgiutiu, ¶15) "Further, the method may comprise identifying at least one geometric feature of the fatigue crack from analysis of the at least one secondary emission wave. Still further, the method may include identifying fatigue crack length and crack tip locations as part of the at least one geometric feature."). The signal fingerprints are also disclosed as being useful for damage diagnosis and prognosis ((Giurgiutiu, ¶270) " Further, the amplitude and frequency contents of the fingerprints may be used for the fatigue damage diagnosis, quantification, and prognosis")
The proposed combination in further view of Giurgiurtiu does not disclose; however, in further view of Khan suggests using deep learning such as the machine-learned CNN model for evaluating vibration signal data for structural health monitoring purposes. ((Khan, Page 589, Col 2, ¶1) "One of the major advantages of deep learning is the autonomous extraction of discriminative features from the raw data [36]. Deep learning has been employed for the damage assessment of laminated composites [37], aircraft engines and power transformers [38], rolling element bearings [39], and the structural health monitoring of civil infrastructures [40]. The discussions in sections 1 and 3 show that the detection, quantification, and localization of delaminations in smart composite laminates from their structural vibration responses have difficulties that are inherited from the dependency of the acquired transient histories on the operating conditions, and the fact that the vibratory responses in the fundamental modes capture the global behavior of the structure. In this section, a CNN-based methodology is proposed to address the current problem of detection and localization of delamination from low-frequency structural vibration response")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
Regarding claim 20, Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) A computer-implemented method for real-time structural health monitoring of a metallic sheet structure potentially having a fatigue crack extending between opposing crack tips, comprising: : A methodology is described as being carried out using a system comprising a personal computer connected to other equipment ((Giurgiutiu, ¶88) " Acoustic emission testing can be performed in the field with portable instruments or in a stationary laboratory setting. Typically, systems contain a sensor, preamplifier, filter, and amplifier, along with measurement, display, and storage equipment (e.g. oscilloscopes, voltmeters, and personal computers). "). The methodology is described as being used in a SHM application ((Giurgiutiu, ¶) " The current disclosure may be used in acoustic emission (AE) for nondestructive evaluation (NDE), and structural health monitoring (SHM) applications. This invention provides a novel methodology to indicate early signs of fatigue cracks."). The method is described as being executed on real-time signals and further described as being performed so as to identify fatigue cracks on aircraft grade aluminum, which is understood to be a metallic sheet structure ((Giurgiutiu, ¶244) " The fatigue experiments were designed to capture the real-time AE signals. Aircraft grade aluminum Al-2024 T3 test coupons of 100-mm-wide, 300-mm-long and 1-mmthick dimension were used."); ((Giurgiutiu, ¶100) " The crack can be evaluated online, during normal operation conditions. "). The fatigue crack is characterized by crack tips on either end ((Giurgiutiu, ¶123) " The current disclosure proposes that the observed peaks are due to local resonances of the crack due to standing waves pinned between the crack tips.")
obtaining, by a computing system comprising one or more computing devices, detected Acoustic Emission (AE) signal data from sensors used with an associated structure to be monitored; A methodology is described as being carried out using a system comprising a personal computer connected to other equipment including sensors to obtain AE data regarding a monitored system ((Giurgiutiu, ¶88) " Acoustic emission testing can be performed in the field with portable instruments or in a stationary laboratory setting. Typically, systems contain a sensor, preamplifier, filter, and amplifier, along with measurement, display, and storage equipment (e.g. oscilloscopes, voltmeters, and personal computers). "); ((Giurgiutiu, ¶81) " The SHM process involves the observation of a system over time using periodically sampled dynamic response measurements from an array of sensors, the extraction of damage-sensitive features from these measurements, and the statistical analysis of these features to determine the current state of system health."); ((Giurgiutiu, ¶82) " One non-obtrusive way to determine structural integrity and conduct SHM is via Acoustic Emission (AE) testing. AE refers to the generation of transient elastic waves produced by a sudden redistribution of stress in a material. When a structure is subjected to an external stimulus ( change in pressure, load, or temperature), localized sources trigger the release of energy, in the form of stress waves, which propagate to the surface and are recorded by sensors.").
[[performing a transformation on the AE signal data to generate a fixed-dimension visual feature map;]]
[[inputting, by the computing system, the fixed-dimension visual feature map into a machine-learned convolutional neural network (CNN) model configured to receive]] AE signal data sensed from a structure ((Giurgiutiu, ¶95) " The acoustic emission signals are then analyzed by frequency analysis and time-frequency analysis to extract signal signatures that are associated with the fatigue crack geometric features. "). See also Figure 29 which depicts a time-frequency representation of data. and to quantitatively estimate crack geometric features of the structure in real-time; ((Giurgiutiu, ¶96) " Frequency and time-frequency analysis are applied to the extract the frequency response and the propagation wave components of the acoustic emission signals. By using the results obtained in the above described vibration resonance experiments, signal signatures that are associated with the fatigue crack resonance modes are extracted. Fatigue crack generated acoustic emission waves were studied with analytical simulation, numerical simulation, and experiments. Using the analytical and numerical simulation models validated by the above described study, the structural geometric features related signal signatures are predicted. A library of features in the acoustic emission waveforms can be used to identify fatigue crack geometric features. "). Crack length is described as a geometric feature ((Giurgiutiu, ¶15) " Further, the method may comprise identifying at least one geometric feature of the fatigue crack from analysis of the at least one secondary emission wave. Still further, the method may include identifying fatigue crack length and crack tip locations as part of the at least one geometric feature. "). The methodology is described as being performed online during normal conditions, thereby indicating real-time functionality ((Giurgiutiu, ¶100) " The crack can be evaluated online, during normal operation conditions.")
receiving, by the computing system, [[as an output of the machine-learned convolutional neural network (CNN) model,]] a characteristic dominant frequency of a crack-length-dependent standing wave pattern resulting from AE energy generated at one crack tip and traveling to the other crack tip of a crack formed in the monitored structure; Dominant frequency peaks are located at various frequencies wherein the fact that particular frequencies are distinguished from all those tested in the spectrum indicates that the particular dominant frequencies are characteristic of the phenomenon ((Giurgiutiu, ¶174) " The firstAE hit of a particular cluster is shown in FIG. 18. The frequency spectrum shows that the dominant frequency peaks are located at 30, 60, 200 kHz."). The standing wave modes are described as being dependent on crack length ((Giurgiutiu, ¶187) " The AE signals may differ slightly at various crack lengths because of any standing wave modes along the crack length. A single parameter from the frequency spectrum, for example, centroid may help to quantify the groupings. "). See also figs 16-18 depicting waveforms generated by a computer, thereby indicative that the computing system receives such data.
quantitatively determining, [[as an output of the machine-learned CNN model,]] the crack length of the crack generating the AE signal data; and Figure 37 shows a flow chart of determining crack length from a recorded AE signal, wherein a length value is understood to be a numeric value that would be a quantification. The AE signal is described as being generated by a fatigue crack ((Giurgiutiu, ¶91) "Fatigue crack generated acoustic emission waves are characterized by analytical simulation, numerical simulation, and experiments ")
using the quantitative crack length determination to monitor crack growth and predict remaining useful life of the monitored structure. Information of AE testing is described as being useful for assessing structural integrity and monitoring dynamic changes in a material (Giurgiutiu, ¶82-86); ((Giurgiutiu, ¶271) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "). Failure of a structure is described as occurring when cracks reach a critical length, thereby indicating that the length of the crack is a parameter used to determine if failure has occurred (which would indicate a prediction of no remaining useful life) ((Giurgiutiu, ¶8) " The potential industrial applications of the current disclosure include but are not limited to those which use nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for monitoring structural integrity using acoustic emissions. "); ((Giurgiutiu, ¶270) " This explains the complex nature of the progressive fatigue damage such as: crack closure; brittle growth of the crack; blunting growth of the crack; and near-critical fast growth of the crack."); ((Giurgiutiu, ¶13) " What is needed in the industry is acoustic emission and ultrasonic testing to determine if structures possess faults and to gauge the extent of such faults, such as evaluation of the crack geometric features. "). Crack geometric features include crack length ((Giurgiutiu, ¶15) "Further, the method may comprise identifying at least one geometric feature of the fatigue crack from analysis of the at least one secondary emission wave. Still further, the method may include identifying fatigue crack length and crack tip locations as part of the at least one geometric feature."). The signal fingerprints are also disclosed as being useful for damage diagnosis and prognosis ((Giurgiutiu, ¶270) " Further, the amplitude and frequency contents of the fingerprints may be used for the fatigue damage diagnosis, quantification, and prognosis")
Giurgiutiu does not disclose the utilization of a CNN for enabling the prediction of crack length; however; Khan discloses performing a transformation on the AE signal data to generate a fixed-dimension visual feature map; Vibration signals are subject to a transform so as to generate 2-d spectrograms which characterize the features of the data ((Khan, Page 593, Col 1, ¶1) "Random structural vibration signals are obtained for the healthy and delaminated smart composites via a piezoelectric sensor, and Short Time Fourier Transform (STFT) is employed to transform those signals into 2-dimensional spectrograms. CNN is employed to automatically extract discriminative features from the vibration-based spectrograms, and employ those features to make a distinction between the healthy and delaminated scenarios, as well between various different scenarios of delamination"). Spectrograms are described as comprising specified dimensions ((Khan, Page 590, ¶1) " To capture the time and frequency domain information of the signal, a sliding window of 512 data points is selected that is moved from left to right each time step (0.0001 s), and Fast Fourier Transform (FFT) is computed for each sliding window. 2. The sliding window of 512 data points moved over each time step (0.0001 s) and the corresponding FFT result in a matrix of size 512 × 256 for each transient history. 3. The matrix of each transient history is transformed into a spectrogram, and each spectrogram is split half in width, and then supplied to CNN as a two-dimensional array of size 256 × 256 × 2. "). A feature map is described as being obtained in a convolution operation wherein a transform of the input image is described ((Khan, Page 590, Col 1, ¶1) "Pooling or Subsampling is employed to reduce the dimensionality of the feature map obtained in the convolution operation. The convolutional and pooling layers transform the input image into a high-level feature map that is employed by the Fully-Connected layer (output layer) for the classification of the input image [42]. ").
inputting, by the computing system, the fixed-dimension visual feature map into a machine-learned convolutional neural network (CNN) model configured to receive. Figure 6 depicts a proposed architecture of a network, wherein an input is shown, demonstrating that the CNN is configured to receive such input ((Khan, Page 587, Col 1, ¶1) "The structural vibration-based spectrograms are employed for the training, cross-validation, and testing of convolutional neural network (CNN)."). The input is described as a spectrogram of vibrational data((Khan, Page 590, Col 1, ¶2) "Section 4.2 discusses the construction of vibration-based spectrograms for input to CNN, section 4.3 focuses on network architecture, section 4.4 presents the classification of delaminations from the proposed network architecture, and section 4.5 discusses the prediction of labels for new cases of delaminations."); See also Table 2 that describes the Input as a 256x256 FFT Spectrum Image. Spectrograms are described as capturing the time and frequency domain information of a signal ((Khan, Page 590, Col 2, ¶1-2) ". Fig. 5 shows the preprocess of transforming structural vibration into vibration spectrograms, which is explained in the following steps: 1. To capture the time and frequency domain information of the signal, a sliding window of 512 data points is selected that is moved from left to right each time step (0.0001 s), and Fast Fourier Transform (FFT) is computed for each sliding window. 2. The sliding window of 512 data points moved over each time step (0.0001 s) and the corresponding FFT result in a matrix of size 512 × 256 for each transient history. 3. The matrix of each transient history is transformed into a spectrogram, and each spectrogram is split half in width, and then supplied to CNN as a two-dimensional array of size 256 × 256 × 2. The process of transforming the transient histories of the healthy and delaminated cases into vibration-based spectrograms via STFT resulted in a total of 13,000 images that corresponded to 13 cases to be classified via CNN. In the next section, the architecture of CNN is developed to autonomously extract discriminative features from the vibration-based spectrograms, and employ those features to classify the healthy and delaminated cases into different classes.")
as an output of the machine-learned convolutional neural network (CNN) model, An output classification is provided by the CNN ((Khan, Page 590, col 1, ¶1) "The convolutional and pooling layers transform the input image into a high-level feature map that is employed by the Fully-Connected layer (output layer) for the classification of the input image [42]. "); ((Khan, Page 590, Col 2, ¶3) " Fig. 6 shows that the network consists of six convolutional and pooling layers for automatically extracting discriminative features from the vibration-based spectrograms of the healthy and delaminated cases. Fully connected layer and Softmax layer is used to classify the healthy and delaminated cases on the basis of features extracted in the convolutional and pooling layers.")
as an output of the machine-learned CNN model, An output classification is provided by the CNN ((Khan, Page 590, col 1, ¶1) "The convolutional and pooling layers transform the input image into a high-level feature map that is employed by the Fully-Connected layer (output layer) for the classification of the input image [42]. "); ((Khan, Page 590, Col 2, ¶3) " Fig. 6 shows that the network consists of six convolutional and pooling layers for automatically extracting discriminative features from the vibration-based spectrograms of the healthy and delaminated cases. Fully connected layer and Softmax layer is used to classify the healthy and delaminated cases on the basis of features extracted in the convolutional and pooling layers.")
Giurgiutiu is analogous art to the claimed invention because it is directed toward the same field of endeavor of characterizing crack lengths in structures using acoustic emissions. Khan is analogous to the claimed invention because it is related to the same field of endeavor of utilizing machine learning techniques for characterizing defects in structures using vibration-based data. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have combined the prior art references so as to impart a CNN approach (as in Khan) on the crack length estimation methodology (as discloses by Giurgiutiu) because some teaching, suggestion, or motivation would have led one having skill in the art to do so in order to arrive at the claimed invention. Giurgiutiu discloses the discovery of a relationship between AE data and crack length that can be characterized by time-frequency data. Khan discloses the utilization of a CNN to autonomously characterize structures based on features contained in time-frequency spectrogram data. While Giurgiutiu discloses the prediction of a crack length as a quantified value, and Khan discloses the classification of a delamination defect instead of a quantified value characterizing the defect, it would have still be obvious to combine the references in order to arrive at the claimed invention because it would be known to one having skill in the art, at the time of the filing of the invention, that a CNN tailored to a classification task could have been further modified so as to produce a numeric prediction result. This is supported by the evidence noted in Matlab (Matlab, “Convert Classification Network into Regression Network”, Available online December 2, 2020, mathworks.com), hereinafter referred to as Matlab, wherein a tutorial is given for how one would modify the capabilities of a CNN for regression purposes instead of classification. Furthermore, while Khan is directed towards the classification of delamination in composite materials and not particularly directed towards detecting crack length in metallic structures, it would have been obvious to use a CNN approach because Giurgiutiu explicitly demonstrates that a relationship between crack length and frequency response is identifiable from time-frequency data and Khan provides an automated mechanism by which to make estimates from such data sources. Khan further touts the major advantage of leveraging deep learning for autonomous extraction from raw data which is particularly advantageous for damage assessment and structural health monitoring applications ((Khan, Page 589, Col 2, ¶1) "One of the major advantages of deep learning is the autonomous extraction of discriminative features from the raw data [36]. Deep learning has been employed for the damage assessment of laminated composites [37], aircraft engines and power transformers [38], rolling element bearings [39], and the structural health monitoring of civil infrastructures [40]. The discussions in sections 1 and 3 show that the detection, quantification, and localization of delaminations in smart composite laminates from their structural vibration responses have difficulties that are inherited from the dependency of the acquired transient histories on the operating conditions, and the fact that the vibratory responses in the fundamental modes capture the global behavior of the structure"). Accordingly, given the information available in the prior art at the time the invention was filed, the combination would have been obvious to one having skill in the art.
Claims 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Giurgiutiu in view of Khan as applied to claim 5 above, and further in view of Joseph (Joseph, R., Bhuiyan, Y., Giurgiutiu, V., "Acoustic emission source modeling in a plate using buried moment tensors", April 28, 2017, Proc. SPIE 10170, Health Monitoring of Structural and Biological Systems 2017, 1017028; https://doi.org/10.1117/12.2260167), hereinafter referred to as Joseph.
Regarding claim 10, the proposed combination discloses The computing system according to claim 5, as stated previously. The proposed combination further in view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model]] uses finite element modeling [[using the moment tensor concept]] for achieving prediction of how crack length values affect the high-frequency content of AE signals. FE analysis is performed to understand the relationship between crack length and recorded AE signals. ((Giurgiutiu, ¶110) "Therefore, to understand generation of plate guided waves due to crack growth and their interaction with cracks, the current disclosure performed FE analysis along with experimental studies. First we introduce the experimental procedure and simplified FE modeling assumptions based on the experiment. Then we present detailed 3D FE models to elaborate our method of estimating crack length from recorded AE signal.").
The proposed combination in further view of Giurgiutiu does not explicitly disclose; however the proposed combination in further view of Khan discloses the machine-learned CNN model using finite element modeling ((Khan, page 593, Col 1, ¶1) " An electromechanically coupled mathematical model is developed for the healthy and delaminated smart composite laminates by incorporating improved layerwise theory, higher-order electric potential field, and finite element method. Random structural vibration signals are obtained for the healthy and delaminated smart composites via a piezoelectric sensor, and Short Time Fourier Transform (STFT) is employed to transform those signals into 2-dimensional spectrograms. CNN is employed to automatically extract discriminative features from the vibration-based spectrograms, and employ those features to make a distinction between the healthy and delaminated scenarios, as well between various different scenarios of delamination.")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
The proposed combination in view of Giurgiutiu and Khan does not explicitly teach; however, Joseph teaches using the moment tensor concept ((Joseph, Page 2, ¶3) "This excitation is a second order tensor. It has the unit of moment per unit area and it is commonly called “moment density tensor”. An integral of this moment density tensor over a finite area is called “moment tensor”. The tensor represents a point excitation acting at the center of gravity of the finite area. So the excitation due to an internal surface motion that produces wave propagation can be approximated by a moment tensor which is acting at a point.")
Giurgiutiu discloses simulating fatigue crack growth using dipole excitation at the crack tips but does not explicitly disclose details as to how the dipole excitation is achieved. Joseph provides the explicit numerical derivation of the moment tensor concept to represent a point excitation. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the system of the proposed combination to use the moment tensor concept taught by Joseph as a mechanism to generate the simulated dipole excitation because Joseph demonstrates that the methodology can be utilized and expected to perform with predictable results.
Regarding claim 11, the proposed combination discloses The computing system according to claim 5, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model uses adaptation of the three-dimensional (3D) moment-tensor concept from geophysics]] to enable the prediction of AE signals in thin-plates using guided-wave theory. FE analysis and experimental studies are performed to understand how plate guided waves correlate to crack growth and recorded AE signals can be utilized to estimate crack length ((Giurgiutiu, ¶110) " Therefore, to understand generation of plate guided waves due to crack growth and their interaction with cracks, the current disclosure per formed FE analysis along with experimental studies. First we introduce the experimental procedure and simplified FE modeling assumptions based on the experiment. Then we present detailed 3D FE models to elaborate our method of estimating crack length from recorded AE signal. "). A 3D simulation is performed, wherein an AE source is modeled ((Giurgiutiu, ¶258) "In order to prove the concept of crack resonance, 3D FEM simulation was performed. The AE source was modeled using dipole concept suggested by Ramstad et al. To model the fatigue crack growth, the concept was extended and two dipoles were modeled along the thickness that represents a line dipole source. The illustration of the 3D FEM modeling is shown in FIG. 52. FIG. 52 shows 3D FEM for harmonic analysis: (a) top view, (b) front view, (c) dipole loading at the crack tip, and ( d) line load along the thickness.")
The proposed combination in further view of Giurgiutiu does not disclose; however the proposed combination in further view of Khan discloses the machine-learned CNN model an excitation to enable the prediction of signal data using low frequency vibration outputs ((Khan, Page 592, Col 2, ¶5 – Page 593, Col 1, ¶1) " Although structural vibration responses obtained via smart elements, such as piezoelectric sensor, are readily available to assess laminated composites for the presence, localization and quantification of delamination, in the published literature, the attributes of low-frequency structural vibrations have been used for global assessment (i.e., the presence of damage), while features from high-frequency outputs (e.g., guided waves) have been employed for the local assessment (i.e., localization of damage) of smart composite laminates. In this paper, a method based on convolutional neural network (CNN) is proposed for the global and local assessment of delamination in smart composite laminates while using only low-frequency structural vibration outputs ")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
The proposed combination in further view of Giurgiutiu and Khan does not disclose; however in view of Joseph discloses uses adaptation of the three-dimensional (3D) moment-tensor concept from geophysics ((Joseph, Page 2, ¶3) "This excitation is a second order tensor. It has the unit of moment per unit area and it is commonly called “moment density tensor”. An integral of this moment density tensor over a finite area is called “moment tensor”. The tensor represents a point excitation acting at the center of gravity of the finite area. So the excitation due to an internal surface motion that produces wave propagation can be approximated by a moment tensor which is acting at a point."); See also equation 47
Giurgiutiu discloses simulating fatigue crack growth using dipole excitation at the crack tips in a 3D model but does not explicitly disclose details as to how the dipole excitation is achieved. Joseph provides the explicit numerical derivation of the 3D moment tensor concept to represent a point excitation. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the system of the proposed combination to use the adapted moment tensor concept taught by Joseph as a mechanism to generate the simulated dipole excitation because Joseph demonstrates that the methodology can be utilized and expected to perform with predictable results.
Claims 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Giurgiutiu in view and Khan as applied to claim 3 above, and further in view of Azimi et al. (Azimi, M., Eslamlou, A., Pekcan, G., “Data-Driven Structural Health Monitoring and Damage Detection through Deep Learning: State-of-the-Art Review”, May 13, 2020, Sensors 20(10), 2778, https://doi.org/10.3390/s20102778), hereinafter referred to as Azimi.
Regarding claim 14, the proposed combination discloses The computing system according to claim 3, as stated previously. The proposed combination in view of Azimi discloses wherein the one or more processors are further configured so that the machine-learned CNN model is tuned so that predictive AE models achieve concurrence with experimentally observed AE signals. Transfer learning is described as a methodology that can be used as an efficient way to fine-tune existing networks (which entail predictive models) for similar classification tasks ((Azimi, Page 14, ¶4) "When the dataset is relatively small and there is a pre-trained network that has already been trained on a larger dataset, an efficient way is to fine-tune the existing network for the new similar classification task. Using transfer learning techniques (TL), the training time can be minimized by transferring the coefficients from the base model instead of starting with randomly assigned weights.").Utilization of real-world data, which encompasses experimental data, is suggested as being utilized to fine-tune ((Azimi, page 23, ¶2) "Therefore, such data can be used for pre-training a network before fine-tuning using real-work data."). Vibration-based data that can be applied to neural networks can include acoustic emissions data ((Azimi, Page 4, ¶3) "DL-based SHM techniques have been used for: general SHM [30], multi-level damage detection, corrosion detection [31], concrete surface bughole recognition [32], concrete crack detection [33], pavement crack detection [34], acoustic emissions source detection [35], etc.").
Azimi is analogous to the claimed invention because it is directed towards outlining deep learning methodologies for structural health monitoring applications and emphasizes the utilization of vision-based and vibration-based damage assessment methods in conjunction with deep learning techniques. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have modified the proposed combination so the CNN model is tuned so as to achieve concurrence between predictive models and experimentally observed signals because combining prior art elements according to known methods would yield predictable results. The proposed combination in view of Giurgiutiu teaches the utilization of a CNN for predicting crack length in structures using AE data, as stated previously. It is understood in the art that a neural network architecture comprises a predictive model so as to enable the prediction. Azimi teaches the utilization of real-world data to fine tune neural networks via training and emphasizes that data plays a key role in deep learning ((Azimi, Page 23, ¶2) "It is obvious that data play a key role in deep learning, and, as explained in previous sections, data collection is not always possible. One option is using transfer learning to shortcut the learning process by using some level of knowledge from earlier studies. An emerging technique for increasing the training data size is using synthetic data for pre-training DL models while using finite element analysis packages (for vibration-based data) or game engines (for vision-based data). Therefore, it would be obvious to one of ordinary skill in the art to ensure that the data used for fine-tuning a predictive model is representative of quality obtained in real-world (such as experimentally observed data) in order to produce an accurate predictive model.
Regarding claim 15, the proposed combination discloses The computing system according to claim 14, as stated previously. The proposed combination in further view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that [[the machine-learned CNN model]] makes selection of representative AE signal features in time domain and frequency domain [[to enable tuning of the predictive AE models.]] AE events can be sorted based on time domain signal and frequency spectrum. ((Giurgiutiu, ¶157) "Then the AE events are sorted based on the same ( or very close) time-domain signal and frequency spectrum. In this sorting process, nine groups of AE hits were produced. The AE events in a particular group have almost same time-domain signal and frequency spectrum (as illustrated later by comparing the waveforms). "). Representative AE signal features are identified for different groups of waveforms in the time domain and frequency domain. ((Giurgiutiu, ¶166) " The representative waveforms from group A, Bis plotted in FIG. 13 at (b) and ( c ), respectively. Both time domain signals and frequency spectra are shown here "); ((Giurgiutiu, ¶168) " The time-domain signal and the frequency spectrum of a representative group C waveform are shown in FIG. 16 at (b ). "); ((Giurgiutiu, ¶170) " time-domain signal of a representative group D waveform is shown at FIG. 17 at (b).");
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Khan discloses the machine-learned CNN model extracting features for employing the features to enable classification by the CNN. ((Khan, Page 590, Col 1, ¶1) "The primary purpose of the convolution is to extract features from the input image by sliding a filter or kernel over small patches of the image. "); ((Khan, page 590, Col 2, ¶2) "In the next section, the architecture of CNN is developed to autonomously extract discriminative features from the vibration-based spectrograms, and employ those features to classify the healthy and delaminated cases into different classes.")
As stated above, it would have been obvious to incorporate the methodology taught by Giurgiutiu into system leveraging an CNN so as to realize the benefits of automating the crack length estimates from sensor data. The further modification of the prior art references would flow consequentially to the combined references such that the CNN would carry out the functionality described herein this claim.
The proposed combination in further view of Giurgiutiu and Khan does not disclose; however in further view of Azimi discloses to enable tuning of the predictive AE models. Transfer learning is described as a methodology that can be used as an efficient way to fine-tune existing networks (which entail predictive models) for similar classification tasks ((Azimi, Page 14, ¶4) "When the dataset is relatively small and there is a pre-trained network that has already been trained on a larger dataset, an efficient way is to fine-tune the existing network for the new similar classification task. Using transfer learning techniques (TL), the training time can be minimized by transferring the coefficients from the base model instead of starting with randomly assigned weights."). Transfer learning requires the selection of a source data (data samples) set that shares relevant features for the target task. ((Azimi, Page 14, ¶4) " Transfer Learning (TL) can alleviate this issue by providing prior knowledge (weights) that was obtained from a similar problem [174]; therefore, fine-tuning can be carried out with a lower computational cost and fewer data samples.").
The proposed combination teaches a neural network architecture utilized for the estimation of crack length in structures. Azimi teaches the concept of transfer learning, wherein it is understood that selecting particular data samples is required to enable execution of transfer learning. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the system of the proposed combination with the additional teachings of Azimi to select data for fine-tuning of the predictive model because leveraging transfer learning offers reduced model training time and enables the utilization of pre-trained models for similar classification tasks ((Azimi, Page 14, ¶4) "When the dataset is relatively small and there is a pre-trained network that has already been trained on a larger dataset, an efficient way is to fine-tune the existing network for the new similar classification task. Using transfer learning techniques (TL), the training time can be minimized by transferring the coefficients from the base model instead of starting with randomly assigned weights."). Pre-trained models already exist for crack detection applications and one having ordinary skill in the art would recognize that crack characterization (by crack length), such as that taught by the proposed combination, is a related task and thus the benefits of transfer learning could be realized. One could have a reasonable expectation of success with decreased computational cost and memory demands, as well as decreased training time by leveraging this methodology ((Azimi, Page 15, ¶1) "They showed that the proposed approach reduces memory demands and inference time. This technique can be applied to decrease the need for a huge amount of training data without losing performance in damage detection.").
Regarding claim 16, the proposed combination discloses The computing system according to claim 3, as stated previously. The proposed combination further in view of Giurgiutiu discloses (except the limitations surrounded by brackets ([[..]])) [[wherein the one or more processors are further configured so that the machine-learned CNN model sifts through experimental AE signals datasets to identify dominant trends correlated]] with crack length information. Crack length information is described as being embedded in experimental AE signal data ((Giurgiutiu, ¶93) "Fatigue loading of several thousands of cycles was applied on test coupons to grow fatigue cracks. Acoustic emissions were generated by the crack growth and these acoustic emissions were recorded using PWAS sensors and acoustic emission sensors. The acoustic emission signals received from a growing crack are processed pursuant to the current disclosure to yield information about the crack length and other geometric properties.")
The proposed combination in further view of Giurgiutiu does not disclose; however in further view of Azimi discloses wherein the one or more processors are further configured so that the machine-learned CNN model sifts through experimental AE signals datasets to identify dominant trends correlated Deep learning models are recognized for the ability to learn information (such as dominant trends) hidden in data to predict patterns ((Azimi, Page 2, ¶2) "The DL models can capture and learn information that is hidden in the data to predict different patterns via stacked blocks of layers that form the DL skeleton [10,12]. ") Sensor data for vibration-based data can be leveraged in deep learning models ((Azimi, Page 2, ¶2) "The limitations on sensor measurement capabilities and challenges in deploying sensor networks due to power and data communication requirements have historically hindered the deployment of dense sensor arrays on civil infrastructure. Large amounts of heterogeneous data are becoming available from different types of sensors, as these limitations have been overcome."). Deep learning methods are known to utilize acoustic emission signals as well as for crack detection purposes ((Azimi, Page 4, ¶3) " DL-based SHM techniques have been used for: general SHM [30], multi-level damage detection, corrosion detection [31], concrete surface bughole recognition [32], concrete crack detection [33], pavement crack detection [34], acoustic emissions source detection [35], etc.")
Giurgiutiu discloses the recognition of crack length information embedded within experimental data. Azimi discloses the utilization of deep learning models (such as CNN) being used to learn information hidden in data. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the proposed combination in view of Azimi because combining prior art elements according to known methods would yield predictable results. Identifying dominant trends in complex datasets is understood in the art as a primary intended use for employing deep learning and neural network architectures. By applying the known AE signal data to a deep learning framework, it would be expected that the network have the capability of identifying such pattern embedded in the data.
Regarding claim 17, the proposed combination discloses The computing system according to claim 16, as stated previously. The proposed combination in further view of Azimi discloses wherein the machine-learned CNN model comprises an AlexNet convolutional neural network (CNN). ((Azimi, Page 15, ¶2) "AlexNet: AlexNet [18], one of the earlier DL models, has been developed to classify objects in the images, and it won the ImageNet [180] classification competition in 2012. It has five convolutional and max-pooling layers, three fully-connected layers, and a 1000-way softmax output layer (25 layers in total). When considering the concrete surface cracks as objects, AlexNet can be fine-tuned for crack detection purposes through transfer learning [96,122]. AlexNet can be loaded to Matlab or Python using the dedicated toolboxes.").
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have further modified the proposed combination with the teachings of Azimi because some teaching, suggestion, or motivation in the prior art references would have led one having skill in the art to do so in order to arrive at the claimed invention. The proposed combination, particularly in view of Khan discloses the utilization of a CNN for predictive tasks but specifies a custom architecture and not a named product. Azimi provides and exemplary CNN that can be used for object detection and further fine tuned for crack detection. Azimi further notes that Alexnet is readily available for use in multiple programming languages and is widely recognized in the art for its capabilities. Accordingly, one having skill in the art and looking to realize the benefits would be compelled to utilize the Alexnet over the custom CNN disclosed by Khan.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Giurgiutiu in view of Khan in view of Azimi as applied to claim 17 above, and further in view of Joseph et al (Joseph, R. Giurgiutiu, V., “Analytical and Experimental Study of Fatigue-Crack-Growth AE Signals in Thin Sheet Metals”, October 2020, Sensors 20, no. 20: 5835. https://doi.org/10.3390/s20205835), hereinafter referred to as Joseph2 to distinguish from the previously referenced Joseph, and even further in view of Kim et al. (Kim, B., Cho, S., “Automated Vision-Based Detection of Cracks on Concrete Surfaces Using a Deep Learning Technique”, October 14, 2018, Sensors 18, no. 10: 3452. https://doi.org/10.3390/s18103452), hereinafter referred to as Kim.
Regarding claim 18, the proposed combination discloses The computing system according to claim 17, as stated previously. The proposed combination in view of Joseph2 discloses (except the limitations surrounded by brackets ([[..]])) wherein the one or more processors are further configured so that a Choi-Williams transform of the acoustic emission signals [[is cropped and augmented to fit 227 x 227-pixel criteria before being entered into an input layer of the AlexNet convolutional neural network architecture. ]] A Choi-William transform of acoustic emission signals is depicted in Figures 6, 18, and 20. ((Joseph2, Page 22, ¶2) "Next, the time-frequency representation of the AE signal prediction and experimental AE signals was obtained using Choi-Williams transform, and the results were compared.")
Joseph2 is analogous art in that it pertains to analysis and predictive modeling of AE signals for fatigue crack growth in thin sheet metals. The proposed combination teaches utilization of AE signals for analysis in the AlexNet neural network architecture, which is known to accept image data as input. Joseph2 teaches the utilization of a Choi-Williams transform of AE signals. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have incorporated the Choi-Williams transform of the AE data as the input of the AlexNet architecture because the Choi-Williams transform is known in the art to provide comprehensive information about a signal and enables visual analysis of the time-frequency content of a signal.
The proposed combination in view of Joseph2 does not disclose; however the combination in view of Kim discloses is cropped and augmented to fit 227 x 227-pixel criteria before being entered into an input layer of the AlexNet convolutional neural network architecture. Images are augmented to have 227x227 pixel resolutions for utilization in an AlexNet CNN ((Kim, Page 14, ¶5) "AlexNet was fine-tuned for five categories, Crack, Joint/Edge (ML), Joint/Edge (SL), Intact Surface, and Plant on 42,000 augmented images with 227 × 227 pixel resolutions. ")
The proposed combination teaches the utilization of AlexNet neural network to estimate crack length of AE data. With the proposed combination utilizing the Choi-Williams transform image of the AE signal for signal analysis, as stated previously, it would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have applied the image augmentation of the Choi-Williams transform image to fit 227x227 pixel criteria as taught by Kim prior to being entered into AlexNet for analysis because it is well-known in the art that AlexNet accepts inputs that follow such dimension criteria. Combining prior art elements according to known methods would yield the predictable results of the AlexNet functioning in its normal capacity when using image dimensions that were specifically designed for the network’s input.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Giurgiutiu in view of Khan in view Azimi as applied to claim 16 above, and further in view of Popescu et al. (Popescu, M. C., Balas, V. E., Perescu-Popescu, L., & Mastorakis, N. (2009). Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems, 8(7), 579-588.), hereinafter referred to as Popescu.
Regarding claim 19, the proposed combination discloses The computing system according to claim 16, as stated previously. The proposed combination in view of Popescu discloses wherein the machine-learned CNN model comprises neural network architecture following a standard multilayer perception model for training its neural connections by backpropagating error and adjusting connection weights following standard steepest gradient descent. . A multilayer perceptron model is described that utilizes backpropagation as the learning (training) algorithm and weight adjustment using the downward gradient method ((Popescu, Page 587, Col 2, ¶2) " Multilayer perceptrons are the most commonly used types of neural networks. Using the backpropagation algorithm for training, they can be used for a wide range of applications, from the functional approximation to prediction in various fields, such as estimating the load of a calculating system or modelling the evolution of chemical reactions of polymerization, described by complex systems of differential equations. "); ((Popescu, Page 581, Col 2, ¶3) "Therefore, the gradients of the weights must be memorized and adjusted after each model in the training set, and the end of an epoch of training, the weights will be changed only one time (there is an „on-line” variant, more simple, in which the weights are updated directly, in this case, the order in which the vectors of the network are presented might matter."); ((Popescu, Page 579, Abstract) "The backpropagation algorithm is the most known and used supervised learning algorithm.")
The proposed combination teaches utilization of a neural network architecture, as stated previously. Popescu teaches utilization of multilayer perceptrons as a type of neural network, wherein the multilayer perceptron employs backpropagation and weight adjustment following the downward gradient method. It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have utilized the standard multilayer perceptron as the particular neural network architecture in the proposed combination because it is the most commonly used type of neural network, indicating that its intricacies are widely understood and that a person having ordinary skill in the art would have a reasonable expectation of predictable results employing the network ((Popescu, Page 587, Col 2, ¶2) "Multilayer perceptrons are the most commonly used types of neural networks. Using the backpropagation algorithm for training, they can be used for a wide range of applications, from the functional approximation to prediction in various fields, such as estimating the load of a calculating system or modelling the evolution of chemical reactions of polymerization, described by complex systems of differential equations. ")
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY GORMAN LEATHERS whose telephone number is (571)272-1880. The examiner can normally be reached Monday-Friday, 9:00 am-5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, EMERSON PUENTE can be reached at (571) 272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.G.L./Examiner, Art Unit 2187
/JOHN E JOHANSEN/Examiner, Art Unit 2187
Prosecution Insights