DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The inclusion of Search Reports and the Written Opinion in the submitted files is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office. Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered.
Claim Rejections - 35 USC § 112
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitation "assigning a value representing the characteristic to the physical article at the machine learning model based on the representation of the signal" In Ln. 7. It is unclear what assigning a value at a machine learning model means. For the purposes of examination, the limitation will be viewed as “assigning a value representing the characteristic to the physical article using the trained machine learning model based on the representation of the signal”
Claim 3 recites the limitation "displaying the assigned defect feature value to a user at an associated display." In Ln. 7. There is insufficient antecedent basis for this limitation in the claim. For the purposes of examination, the limitation will be viewed as “displaying the assigned ”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
` Claims 1-20, are rejected under 35 U.S.C. 101 because the claimed invention is
directed to an abstract idea without significantly more.
With respect to claims 1 and 16,
Step 1:
The claims are directed to a process as they are a method for training a machine learning algorithm.
Step 2A Prong One:
The following bold limitations are considered abstract:
“A method for training a machine learning model for non-destructive evaluation, the method comprising:
generating a plurality of training samples, wherein each of a subset of the plurality of training samples represents an article with a defect and generating a given training sample comprises:
generating a virtual model of the article on a computer system;
generating a simulated signal representing an output of a non-destructive evaluation system scanning the article based on the virtual model at the computer system;
and associating a representation of the simulated signal with a parameter representing a characteristic of the virtual model of the article to generate the given training sample;
and training the machine learning model on the plurality of training samples.”
The above bolded limitations are directed to abstract ideas and would fall within the
“Mathematical Concept” and “Mental Process” groupings of abstract ideas. Generating a training sample by generating models of articles, signals, and responses is a mathematical concept as modeling is a mathematical representation. This can be further seen in specification pages 29-36. According to MPEP 2106.04(C) “A claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation. There is no particular word or set of words that indicates a claim recites a mathematical calculation. That is, a claim does not have to recite the word "calculating" in order to be considered a mathematical calculation. For example, a step of "determining" a variable or number using mathematical methods or "performing" a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation.” associating a representation of the simulated signal with a parameter amounts to labeling data. This can be done in the human mind using observation judgement and opinion.
Step 2A Prong Two:
This judicial exception is not integrated into a practical application. In particular, the
claim recites the additional elements –
“a machine learning model for non-destructive evaluation and a computer system.”
Examiner views these limitations amount to generally linking the use of the judicial exception to
a particular technological environment or field of use – see MPEP 2106.05(h)
As such Examiner does NOT view that the claims
-Improve the functioning of a computer, or to any other technology or technical field
-Apply the judicial exception with, or by use of, a particular machine - see MPEP 2106.05(b)
-Effect a transformation or reduction of a particular article to a different state or thing-see MPEP 2106.05(c)
-Apply or use the judicial exception in some other meaningful way beyond generally
linking the use of the judicial exception to a particular technological environment, such that the
claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP
2106.05(e) and Vanda Memo.
Moreover, Examiner views the claims to be merely generally linking the use of the judicial
exception to a non-destructive evaluation. Furthermore, using a computer system is viewed as using a computer as a tool and does not integrate the abstract idea into a practical application.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception. Considering the claim as a whole, one of ordinary skill in the art would
not know the practical application of the present invention since the claims do not apply or use the
judicial exception in some meaningful way. As currently claimed, Examiner views that the additional
elements do not apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit
on the judicial exception, because the claim fails to recite clearly how the judicial exception is applied in
a manner that does not monopolize the exception because the limitations “a machine learning model for non-destructive evaluation and a computer system.” just tie the claim to non-destructive evaluation. Examiner further notes that such additional elements are viewed to be well known routine and conventional as evidenced by Belanger (US 20220276207 A1) and Luo (CN 112150407 A).
With respect to claim 9,
Step 1:
The claim is directed to a machine as it is a system for training a machine learning algorithm.
Step 2A Prong One:
The following bold limitations are considered abstract:
“A system comprising: a processor; and a non-transitory computer readable medium storing instructions for training a machine learning model for non-destructive testing, the instructions being executable by the processor to provide:
a sample generation system that generates a plurality of training samples, wherein each of a subset of the plurality of training samples represents an article with a defect, the sample generation system comprising:
a computational modeling component that generates a virtual model of a given article and generates a simulated signal representing an output of a non-destructive evaluation system scanning the article based on the virtual model;
and a sample labeler that associates a representation of the simulated signal with a parameter representing a characteristic of the virtual model of the given article to generate the given training sample;
and a training component that trains the machine learning model on the plurality of training samples to produce a trained machine learning model.”
The above bolded limitations are directed to abstract ideas and would fall within the
“Mathematical Concept” and “Mental Process” groupings of abstract ideas. Generating a training sample by generating models of articles, signals, and responses is a mathematical concept as modeling is a mathematical representation. This can be further seen in specification pages 29-36. According to MPEP 2106.04(C) “A claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation. There is no particular word or set of words that indicates a claim recites a mathematical calculation. That is, a claim does not have to recite the word "calculating" in order to be considered a mathematical calculation. For example, a step of "determining" a variable or number using mathematical methods or "performing" a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation.” associating a representation of the simulated signal with a parameter amounts to labeling data. This can be done in the human mind using observation judgement and opinion.
Step 2A Prong Two:
This judicial exception is not integrated into a practical application. In particular, the
claim recites the additional elements –
“A system comprising: a processor; and a non-transitory computer readable medium storing instructions for training a machine learning model for non-destructive testing, the instructions being executable by the processor to provide: a sample generation system, a computational modeling component, and a sample labeler, and a training component.”
Examiner views these limitations amount to generally linking the use of the judicial exception to
a particular technological environment or field of use – see MPEP 2106.05(h)
As such Examiner does NOT view that the claims
-Improve the functioning of a computer, or to any other technology or technical field
-Apply the judicial exception with, or by use of, a particular machine - see MPEP 2106.05(b)
-Effect a transformation or reduction of a particular article to a different state or thing-see MPEP 2106.05(c)
-Apply or use the judicial exception in some other meaningful way beyond generally
linking the use of the judicial exception to a particular technological environment, such that the
claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP
2106.05(e) and Vanda Memo.
Moreover, Examiner views the claims to be merely generally linking the use of the judicial
exception to a non-destructive evaluation. Furthermore, using a processor and memory as well as the components and systems are viewed as using a computer as a tool and do not integrate the abstract idea into a practical application.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception. Considering the claim as a whole, one of ordinary skill in the art would
not know the practical application of the present invention since the claims do not apply or use the
judicial exception in some meaningful way. As currently claimed, Examiner views that the additional
elements do not apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit
on the judicial exception, because the claim fails to recite clearly how the judicial exception is applied in
a manner that does not monopolize the exception because the limitations “A system comprising: a processor; and a non-transitory computer readable medium storing instructions for training a machine learning model for non-destructive testing, the instructions being executable by the processor to provide: a sample generation system, a computational modeling component, and a sample labeler, and a training component” just tie the claim to non-destructive evaluation. Examiner further notes that such additional elements are viewed to be well known routine and conventional as evidenced by Belanger (US 20220276207 A1) and Luo (CN 112150407 A).
Dependent claims 2-8, 10-15, and 17-20 when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitation(s) fail(s) to establish that the claims are not directed to an abstract idea, as detailed below:
The dependent claims are directed to further limit the training samples by generating further mathematical models which are mathematical concepts. Claim 3 includes scanning a physical article and displaying a feature value. However, these additional elements are viewed as mere data gathering and necessary data outputting as it is unclear if the signal is fed into the trained machine learning model. Claim 11 is directed to an integrated circuit and a non-destructive evaluation element which are additional elements, however, they amount to using a computer as a tool and just tie the claim to non-destructive evaluation.
Therefore, dependent claims 2-8, 10-15, and 17-20 further limit the abstract idea with an abstract idea and thus the claims are still directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5-7, 9-12, 14, and 16-18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Belanger (US 20220276207 A1).
With respect to claim 1,
Belanger teaches,
A method for training a machine learning model for non-destructive evaluation, the method comprising: (Para. [0008] teaches “In accordance with one or more broad aspects, there is provided methods and systems for non-destructive testing (NDT) using ultrasonic testing (UT). The NDT method and systems may serve to determine whether or not a given component satisfies an acceptability criterion with respect to the presence of flaws within the component. The methods and systems for performing NDT may make use of neural networks of various types, for instance Faster Region-based Convolutional Neural Networks (R-CNN).”)
generating a plurality of training samples, wherein each of a subset of the plurality of training samples represents an article with a defect and generating a given training sample comprises: (Para. [0044] teaches “In some embodiments, training of the Faster R-CNN 109 is performed using datasets of simulated ultrasonic tests, that is to say, using simulated data. The simulated data may be produced by graphical processing unit (GPU)-accelerated finite element (FE) simulations, which replicate the empirical acquisition process described above to generate simulated TTMs.” Para. [0045] teaches “For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present.”)
generating a virtual model of the article on a computer system; (Para. [0147] teaches “At step 152, the method 150 includes modeling a plurality of simulated ultrasonic testing scenarios. For example, a plurality of different models 120 may be generated, which may have differing dimensions for the blocks 122, different positioning and sizing for the defects 124, differing levels for the fluid 128, and the like.” Para. [0078] teaches “The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks.” (i.e. computer system))
generating a simulated signal representing an output of a non-destructive evaluation system scanning the article based on the virtual model at the computer system; (Para. [0148] teaches “The first training scenarios may be generated by simulating the response of the models 120 generated at step 152 to a simulated insonification using simulated acoustic waves. In some cases, different simulated insonifications may be performed for a given model.”)
and associating a representation of the simulated signal with a parameter representing a characteristic of the virtual model of the article to generate the given training sample; (Para. [0045] teaches “FE model 120 representing the component 110 within a simulated version of the experimental setup 100 of FIG. 1B is illustrated. The component 110 is simulated as an aluminium block 122 which is immersed in a fluid 128. For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.” Para. [0054] teaches “The sizes and positions of defects 112 were randomly generated in order to diversify the training data. For simulation purposes, the defects (FBH and SDH) were filled with water, and all the positions and sizes of the reflectors, as well as the heights of the aluminum block, were stored in memory in order to be used during the supervised learning of the neural network.”)
and training the machine learning model on the plurality of training samples. (Para. [0049] teaches “At step 156, the method 150 includes training the Faster R-CNN 109 using the first training dataset. Training of the Faster R-CNN 109 may be performed in any suitable fashion.”)
With respect to claim 2,
Belanger teaches,
The method of claim 1, wherein generating a virtual model of the article at a computer system represents generating a virtual model of an article having an internal defect, and the parameter represents geometrical characteristics of the internal defect. (Para. [0054] teaches “The sizes and positions of defects 112 were randomly generated in order to diversify the training data. For simulation purposes, the defects (FBH and SDH) were filled with water, and all the positions and sizes of the reflectors, as well as the heights of the aluminum block, were stored in memory in order to be used during the supervised learning of the neural network.” (i.e. Size of defect is viewed as geometrical characteristic.)
With respect to claim 3,
Belanger further teaches,
The method of claim 1, further comprising: scanning a physical article with a non-destructive evaluation system to produce a signal; (Fig. 2 see step 162)
providing a representation of the signal to the machine learning model; (Para. [0037] teaches “The bounding box may be generated by the operation of the Faster R-CNN 109. In some cases, a plurality of bounding boxes for the component 110 are generated, each having different confidence scores, and the bounding box having the highest confidence score is selected. The generation of the plurality of bounding boxes, having associated confidence scores, is performed via the Faster R-CNN 109 having been trained to perform analysis of the TTM”
assigning a value representing the characteristic to the physical article at the machine learning model based on the representation of the signal; (Para. [0038] teaches “At step 169, the method 160 includes analyzing a portion of the representation associated with the component to detect a presence of a defect in the component 110 to be tested, for instance the defect 112. Analysis of the portion of the representation may include analyzing the portion of the TTM which corresponds to the bounding box generated at step 168, and may include an analysis to identify discontinuities, changes in reflectivity, or other responses of the component to the insonification.”
and displaying the assigned defect feature value to a user at an associated display. Para. [0039] teaches “As illustrated in the diagram 220, the Faster R-CNN identifies the presence of a FBH defect within the bounding box 224. In some cases, step 168 and/or step 169 may include rejecting portions of the TTM diagram 210 which are not related to the component 110.” Para. [0042] teaches “When the defect 112 is detected at step 170, the method 160 moves to step 172. At step 172, the method 160 includes issuing an alert indicative of the defect 112. The alert may indicate a size, position, and/or orientation of the defect, as well as the nature of the defect 112, where available. The alert may, in some cases, include a copy of the representation of the component produced at step 166, a link to the representation, or the like. In this fashion, an operator of the experimental setup 100 may be able to confirm or validate the determination performed by the Faster R-CNN.”)
With respect to claim 5,
Belanger further teaches,
The method of claim 1, wherein a defect associated with a first training sample of the plurality of training samples represents a crack within the article associated with the first training sample. (Para. [0003] teaches “The method of claim 1, wherein a defect associated with a first training sample of the plurality of training samples represents a crack within the article associated with the first training sample.”)
With respect to claim 6,
Belanger further teaches,
The method of claim 1, wherein generating the simulated signal representing the output of the non-destructive evaluation system scanning the article based on the virtual model comprises generating a simulated ultrasound signal representing an output of an ultrasound system scanning the article based on the virtual model at the computer system. (Para. [0048] teaches “The first training scenarios may be generated by simulating the response of the models 120 generated at step 152 to a simulated insonification using simulated acoustic waves. In some cases, different simulated insonifications may be performed for a given model. Alternatively, or in addition, different locations of certain elements of the models 120 may be simulated, thereby diversifying the training dataset.”)
With respect to claim 7,
Belanger further teaches,
The method of claim 1, wherein the machine learning model is a convolutional neural network and associating the representation of the simulated signal with the characteristic to generate the given training sample comprises associating the simulated signal with the characteristic to generate the given training sample. (Para. [0008] teaches “The methods and systems for performing NDT may make use of neural networks of various types, for instance Faster Region-based Convolutional Neural Networks (R-CNN).” Para. [0045] teaches “FE model 120 representing the component 110 within a simulated version of the experimental setup 100 of FIG. 1B is illustrated. The component 110 is simulated as an aluminium block 122 which is immersed in a fluid 128. For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.”)
With respect to claim 9,
Belanger teaches,
A system comprising: a processor; and a non-transitory computer readable medium storing instructions for training a machine learning model for non-destructive testing, the instructions being executable by the processor to provide: (Para. [0008] teaches “In accordance with one or more broad aspects, there is provided methods and systems for non-destructive testing (NDT) using ultrasonic testing (UT). The NDT method and systems may serve to determine whether or not a given component satisfies an acceptability criterion with respect to the presence of flaws within the component. The methods and systems for performing NDT may make use of neural networks of various types, for instance Faster Region-based Convolutional Neural Networks (R-CNN).” Para. [0055] teaches “As depicted, computing device 250 includes at least one processor 252, a memory 254 storing instructions 256, and at least one I/O interface (illustrated as ‘Inputs’ and ‘Outputs’)”)
a sample generation system that generates a plurality of training samples, wherein each of a subset of the plurality of training samples represents an article with a defect, the sample generation system comprising: (Para. [0044] teaches “In some embodiments, training of the Faster R-CNN 109 is performed using datasets of simulated ultrasonic tests, that is to say, using simulated data. The simulated data may be produced by graphical processing unit (GPU)-accelerated finite element (FE) simulations, which replicate the empirical acquisition process described above to generate simulated TTMs.” Para. [0045] teaches “For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present.”)
a computational modeling component that generates a virtual model of a given article and generates a simulated signal representing an output of a non-destructive evaluation system scanning the article based on the virtual model; (Para. [0044] teaches “The simulated data may be produced by graphical processing unit (GPU)-accelerated finite element (FE) simulations, which replicate the empirical acquisition process described above to generate simulated TTMs. For example, a comparatively large portion of a training dataset is generated via the FE simulations” (i.e. computational modeling component) Para. [0147] teaches “At step 152, the method 150 includes modeling a plurality of simulated ultrasonic testing scenarios. For example, a plurality of different models 120 may be generated, which may have differing dimensions for the blocks 122, different positioning and sizing for the defects 124, differing levels for the fluid 128, and the like.” Para. [0148] teaches “The first training scenarios may be generated by simulating the response of the models 120 generated at step 152 to a simulated insonification using simulated acoustic waves. In some cases, different simulated insonifications may be performed for a given model.”)
and a sample labeler that associates a representation of the simulated signal with a parameter representing a characteristic of the virtual model of the given article to generate the given training sample; (Para. [0045] teaches “FE model 120 representing the component 110 within a simulated version of the experimental setup 100 of FIG. 1B is illustrated. The component 110 is simulated as an aluminium block 122 which is immersed in a fluid 128. For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.” Para. [0054] teaches “The sizes and positions of defects 112 were randomly generated in order to diversify the training data. For simulation purposes, the defects (FBH and SDH) were filled with water, and all the positions and sizes of the reflectors, as well as the heights of the aluminum block, were stored in memory in order to be used during the supervised learning of the neural network.” (i.e. supervised learning requires training with labeled data-sets.) Para. [0074] teaches “Training of the Faster R-CNN 109 was performed on segmented and classified data generated using graphics processing unit (GPU)-accelerated finite element simulations. A training dataset is generated based on the GPU-accelerated FE simulations.”)
and a training component that trains the machine learning model on the plurality of training samples to produce a trained machine learning model. (Para. [0049] teaches “At step 156, the method 150 includes training the Faster R-CNN 109 using the first training dataset. Training of the Faster R-CNN 109 may be performed in any suitable fashion.”)
With respect to claim 10,
Belanger further teaches,
The system of claim 9, wherein the computational modeling component is implemented as a finite element modeling system. (Para. [0044] teaches “The simulated data may be produced by graphical processing unit (GPU)-accelerated finite element (FE) simulations”)
With respect to claim 11,
Belanger further teaches,
The system of claim 9, wherein the trained machine learning model is implemented as an integrated circuit chip onboard a non-destructive evaluation system. (Para. [0028] teaches “The UT tool 101 may perform part or all of the evaluation of the component 110 itself, or may provide information regarding the recorded acoustic waves (termed “acoustic information”) to the processing system 107. The processing system 107 analyzes the acoustic information in any suitable fashion, and can produce various results or conclusions regarding the characteristics of the component 110.” Para. [0056] teaches “Each processor 252 may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof.”)
With respect to claim 12,
Belanger teaches,
The system of claim 9, wherein the computational modeling system generates a simulated ultrasound signal representing an output of an ultrasound system scanning the article as a time series of values. (Para. [0039] teaches “The TTM diagram 210 may be generated as part of step 166 as a representation of the component 110 by summing and concatenating multiple amplitude time traces as described hereinabove to produce an amplitude time series.”)
With respect to claim 14,
Belanger teaches,
The system of claim 9, wherein the parameter represents one of a plurality of classes, the plurality of classes including a first class representing an absence of a defect in the virtual model of the article, a second class representing the presence of a first type of defect in the virtual model of the article, a third class representing a second type of defect in the virtual model of the article, and a fourth class representing multiple defects in the virtual model of the article. (Para. [0045] teaches “For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.”)
With respect to claim 16,
Belanger teaches,
A method for training a convolutional neural network for non-destructive evaluation, the method comprising: (Para. [0008] teaches “In accordance with one or more broad aspects, there is provided methods and systems for non-destructive testing (NDT) using ultrasonic testing (UT). The NDT method and systems may serve to determine whether or not a given component satisfies an acceptability criterion with respect to the presence of flaws within the component. The methods and systems for performing NDT may make use of neural networks of various types, for instance Faster Region-based Convolutional Neural Networks (R-CNN).”
generating a plurality of training samples, having selected characteristics, in an article, wherein each of a subset of the plurality of training samples represents an article with a defect and generating a given training sample comprises; (Para. [0044] teaches “In some embodiments, training of the Faster R-CNN 109 is performed using datasets of simulated ultrasonic tests, that is to say, using simulated data. The simulated data may be produced by graphical processing unit (GPU)-accelerated finite element (FE) simulations, which replicate the empirical acquisition process described above to generate simulated TTMs.” Para. [0045] teaches “For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present.”)
generating a virtual model of the article on a computer system; (Para. [0147] teaches “At step 152, the method 150 includes modeling a plurality of simulated ultrasonic testing scenarios. For example, a plurality of different models 120 may be generated, which may have differing dimensions for the blocks 122, different positioning and sizing for the defects 124, differing levels for the fluid 128, and the like.” Para. [0078] teaches “The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks.”)
generating a simulated signal representing an output of a non-destructive evaluation system scanning the article based on the virtual model at the computer system; (Para. [0148] teaches “The first training scenarios may be generated by simulating the response of the models 120 generated at step 152 to a simulated insonification using simulated acoustic waves. In some cases, different simulated insonifications may be performed for a given model.”
and associating a representation of the simulated signal with a parameter representing a characteristic of the virtual model of the article to generate the given training sample; (Para. [0045] teaches “FE model 120 representing the component 110 within a simulated version of the experimental setup 100 of FIG. 1B is illustrated. The component 110 is simulated as an aluminium block 122 which is immersed in a fluid 128. For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.” Para. [0054] teaches “The sizes and positions of defects 112 were randomly generated in order to diversify the training data. For simulation purposes, the defects (FBH and SDH) were filled with water, and all the positions and sizes of the reflectors, as well as the heights of the aluminum block, were stored in memory in order to be used during the supervised learning of the neural network.”)
and training the convolutional neural network on the plurality of training samples to provide a trained convolutional neural network. (Para. [0049] teaches “At step 156, the method 150 includes training the Faster R-CNN 109 using the first training dataset. Training of the Faster R-CNN 109 may be performed in any suitable fashion.”)
With respect to claim 17,
Belanger further teaches,
The method of claim 16, wherein the trained machine learning model is implemented as an integrated circuit chip onboard a non-destructive evaluation system. (Para. [0028] teaches “The UT tool 101 may perform part or all of the evaluation of the component 110 itself, or may provide information regarding the recorded acoustic waves (termed “acoustic information”) to the processing system 107. The processing system 107 analyzes the acoustic information in any suitable fashion, and can produce various results or conclusions regarding the characteristics of the component 110.” Para. [0056] teaches “Each processor 252 may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof.”)
With respect to claim 18,
Belanger teaches,
The method of claim 16, wherein the parameter represents one of a plurality of classes, the plurality of classes including a first class representing an absence of a defect in the virtual model of the article, a second class representing the presence of a first type of defect in the virtual model of the article, a third class representing a second type of defect in the virtual model of the article, and a fourth class representing multiple defects in the virtual model of the article. (Para. [0045] teaches “For the purposes of training the Faster R-CNN 109, the block 122 is provided a defect 124 for detection. The defect 124 may be any suitable type of defect, for example a side-drilled hole (SDH) or a flat bottom hole (FBH), though other types of defects may also be present. In the example of FIG. 3A, the model 120 has a FBH defect. In some instances, each simulated FE model 120 includes a single defect, though training sets including one or more FE models 120 with multiple defects in a single block 122 are also considered.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Belanger (US 20220276207 A1) as applied to claims 1 and 16 above, and further in view of Li (CN 112348034 A).
With respect to claim 4,
Belanger does not explicitly teach,
The method of claim 1, wherein a defect associated with a first training sample of the plurality of training samples represents corrosion affecting the article associated with the first training sample.
Li teaches,
wherein a defect associated with a first training sample of the plurality of training samples represents corrosion affecting the article associated with the first training sample. (Para. [0078] teaches “Images of crane metal structures are collected through the UAV vision system, and four types of crane metal structure image libraries are established: normal, cracked, corroded, and worn. Feature vectors are obtained using the feature extraction method based on multi-scale geometric analysis mentioned above, and then input into the support vector machine for training.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Belanger wherein a defect associated with a first training sample of the plurality of training samples represents corrosion affecting the article associated with the first training sample such as that of Li.
One of ordinary skill would have been motivated to modify Belanger, because it would allow the method to detect corrosion which could lead to collapse or degradation of the article. Therefore, detecting corrosion early would prevent such failures from occurring.
With respect to claim 20,
Belanger does not explicitly teach,
The method of claim 16, wherein the parameter represents one of a length of a patch of corrosion in the virtual model of the article, a width of the patch of corrosion in the virtual model of the article, and a location of the patch of corrosion in the virtual model of the article.
Li teaches,
wherein the parameter represents one of a length of a patch of corrosion in the virtual model of the article, a width of the patch of corrosion in the virtual model of the article, and a location of the patch of corrosion in the virtual model of the article. ((Para. [0078] teaches “Intelligent detection of defects such as cracks, corrosion, and wear is performed.” Para. [0082] teaches “The proportion of the defect area in the image, and the position of the defect area in the image.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Belanger wherein a defect associated with a first training sample of the plurality of training samples represents corrosion affecting the article associated with the first training sample such as that of Li.
One of ordinary skill would have been motivated to modify Belanger, because it would allow the method to detect corrosion which could lead to collapse or degradation of the article. Therefore, detecting corrosion early would prevent such failures from occurring.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Belanger (US 20220276207 A1) as applied to claim 1 above, and further in view of Wrenninge (US 20190156151 A1).
With respect to claim 8,
Belanger further teaches,
wherein the characteristic is a first characteristic of a plurality of characteristics, and the parameter is a first parameter of a plurality of parameters corresponding to the plurality of characteristics, (Para. [0046] teaches “Faster R-CNN suitable for use in detecting defects in a component, for example the Faster R-CNN 109. As presented herein, the production of the Faster R-CNN 109 involves training a neural network to detect the presence of defects. In some embodiments, the neural network in question may be subjected to various preliminary training procedures prior to execution of the method 150.”(i.e. defects are viewed as characteristics.) Para. [0060] teaches “The parameters of the network were defined following multiple tests in order to obtain a p level of efficiency. In order to provide improved context for pattern recognition to the Region Proposal Network (RPN), maximum pooling operations are performed to decrease the data inputs time trace dimension while preserving the 64 A-Scan columns. Following the final convolutional layers, a dropout layer with a rate of 0.2 was set. In some cases, the use of the dropout layer may avoid overfitting or specialization of the CNN to the simulated dataset.”)
Belanger does not explicitly teach,
wherein a first proper subset of the plurality of training samples is assigned predetermined values for the plurality of parameters associated with each training sample of the first proper subset, and a second proper subset of the plurality of training samples is assigned random values for the plurality of parameters associated with each training sample of the second proper subset.
Wrenninge teaches,
wherein a first proper subset of the plurality of training samples is assigned predetermined values for the plurality of parameters associated with each training sample of the first proper subset, and a second proper subset of the plurality of training samples is assigned random values for the plurality of parameters associated with each training sample of the second proper subset.
wherein a first proper subset of the plurality of training samples is assigned predetermined values for the plurality of parameters associated with each training sample of the first proper subset, (Para. [0031] teaches “Block S100 includes determining a set of parameter values associated with at least one of a set of geometric parameters, a set of rendering parameters, and a set of augmentation parameters. Block S100 functions to define the parametric rule set for 3D scene generation, image rendering, and image augmentation. Block S100 can also function to maximize coverage of a parameter space associated with a parameter (e.g., by sampling the value of the parameter from an LDS) given a finite number of samples (e.g., synthetic images, data samples, etc.)”and a second proper subset of the plurality of training samples is assigned random values for the plurality of parameters associated with each training sample of the second proper subset.” (i.e. set of parameter values is a first proper subset
and a second proper subset of the plurality of training samples is assigned random values for the plurality of parameters associated with each training sample of the second proper subset. (Para. [0049] teaches “The grouped parameters can be: related, unrelated, the varied parameters (e.g., randomized parameters, parameters for which random values are determined), a varied parameter with a set of static parameters, parameters selected by a user, randomly selected parameters, or be any suitable set of parameters. In a first example,”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Belanger wherein a first proper subset of the plurality of training samples is assigned predetermined values for the plurality of parameters associated with each training sample of the first proper subset, and a second proper subset of the plurality of training samples is assigned random values for the plurality of parameters associated with each training sample of the second proper subset such as that of Wrenninge.
One of ordinary skill would have been motivated to modify Belanger, because it would allow the method to evaluate the training model based on the synthetic image dataset and ensure the model is accurate as seen in Para. [0013] of Wrenninge.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Belanger (US 20220276207 A1) as applied to claim 9 above, and further in view of Luo (CN 112150407 A).
With respect to claim 13,
Belanger teaches,
The system of claim 9, wherein the machine learning model is a convolutional neural network having a plurality of convolutional layers, a pooling layer,(Para. [0060] teaches “With reference to FIG. 6, an example architecture for implementing the Faster R-CNN is illustrated at 600.” Fig. 6 shows a plurality of convolutional layers and a several pooling layers.
the training component providing a set of sample values representing the simulated signal directly to a first convolutional layer of the plurality of convolutional layers. (Para. [0066] teaches “training was performed using the FE simulated dataset 710 which contained data from 500 FBH-having versions models 120, 500 SDH-having versions of the model 120 and 1000 versions of the model 120 where the block 122 had no defects. The amount of training performed may be selected based on the desired requisite time for training and/or available computation power. In one example, the Faster R-CNN went through 550 epochs of training” Fig. 6 shows that the convolutional layer is the first layer.)
Belanger does not explicitly teach,
and a plurality of fully connected layers,
Luo teaches,
and a plurality of fully connected layers, (Para. [00] teaches “The feature extraction network consists of an input layer, three convolutional neural network layers, two fully connected layers, and corresponding activation functions and pooling layers.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Belanger with a plurality of fully connected layers such as that of Luo.
One of ordinary skill would have been motivated to modify Belanger, because it would allow the system to more accurately detect defects as seen in Para. [0052] of Luo. Furthermore, fully connected layers allow the system to capture relationships across the entire dataset and are ideal for being used as a classifier in convolutional neural networks.
Claims 15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Belanger (US 20220276207 A1) as applied to claims 9 and 16 above, and further in view of Giurgiutiu (US 20200408720 A1).
With respect to claim 15,
Belanger does not explicitly teach,
The system of claim 9, wherein the parameter represents one of a length of a crack in the virtual model of the article, an orientation of the crack in the virtual model of the article, and a location of the crack in the virtual model of the article.
Giurgiutiu teaches,
wherein the parameter represents one of a length of a crack of the article, an orientation of the crack of the article, and a location of the crack of the article. (Para. [0015] teaches “Still further, the method may include identifying fatigue crack length and crack tip locations” Abstract teaches “determines if structural faults exist and extracts geometric features of the structural faults from acoustic emission waveforms, such as crack length and orientation.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Giurgiutiu wherein the parameter represents one of a length of a crack in the virtual model of the article, an orientation of the crack in the virtual model of the article, and a location of the crack in the virtual model of the article.
One of ordinary skill would have been motivated to modify Belanger, because training the neural network to detect cracks would allow the system to recognize them so they can be fixed or addressed before they get worse. Furthermore, Belanger in paragraph 3 teaches detecting cracks using ultrasonic testing where virtual models are used to train the neural networks.
With respect to claim 19,
Belanger does not explicitly teach,
The method of claim 16, wherein the parameter represents one of a length of a crack in the virtual model of the article, an orientation of the crack in the virtual model of the article, and a location of the crack in the virtual model of the article.
Giurgiutiu teaches,
wherein the parameter represents one of a length of a crack of the article, an orientation of the crack of the article, and a location of the crack of the article. (Para. [0015] teaches “Still further, the method may include identifying fatigue crack length and crack tip locations” Abstract teaches “determines if structural faults exist and extracts geometric features of the structural faults from acoustic emission waveforms, such as crack length and orientation.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Belanger wherein the parameter represents one of a length of a crack in the virtual model of the article, an orientation of the crack in the virtual model of the article, and a location of the crack in the virtual model of the article.
One of ordinary skill would have been motivated to modify Belanger, because training the neural network to detect cracks would allow the system to recognize them so they can be fixed or addressed before they get worse. Furthermore, Belanger in paragraph 3 teaches detecting cracks using ultrasonic testing where virtual models are used to train the neural networks.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to JOSHUA L FORRISTALL whose telephone number is 703-756-4554. The examiner
can normally be reached Monday-Friday 8:30 AM- 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Schechter can be reached on 571-272-2302. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/JOSHUA L FORRISTALL/Examiner, Art Unit 2857
/ANDREW SCHECHTER/Supervisory Patent Examiner, Art Unit 2857
Prosecution Insights