DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 2024/04/25. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (an abstract idea) without reciting significantly more.
Regarding independent claims 1, 13, and 14
Step 1 -- whether the claim falls within any statutory category. See MPEP 2106.03
Claim 1 is drawn to a method claim. Therefore, claim 1 falls within one of the four categories of statutory subject matter, namely a process.
Claim 13 is drawn to a method claim. Therefore, claim 13 falls within one of the four categories of statutory subject matter, namely a process.
Claim 14 is drawn to a system claim reciting a computer having a processor and memory. Therefore, claim 14 falls within one of the four categories of statutory subject matter, namely a machine.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II
Regarding independent claim 1, the claim is directed to a method for reconstruction-based time-series anomaly detection comprising performing a transform on data samples to provide a matrix, computing a reconstructed matrix, multiplying a weight vector by a difference between the two matrices, and computing a score from the resulting matrix.
The limitation of "performing a short time Fourier transform (STFT) on time-series data samples to provide an input STFT matrix" recites an abstract idea because a short time Fourier transform is a mathematical transformation that converts a sequence of sampled values into a matrix of frequency-domain coefficients by means of mathematical calculation. This limitation falls within the mathematical concepts grouping of abstract ideas, specifically a mathematical calculation and a mathematical relationship. See MPEP § 2106.04(a)(2), subsection I.
The limitation of "computes a reconstructed output STFT matrix" recites an abstract idea because, under the broadest reasonable interpretation, the claim recites no details regarding how the reconstructed matrix is computed. The limitation describes producing one set of numerical values from another set of numerical values by calculation, and therefore falls within the mathematical concepts grouping of abstract ideas. See MPEP § 2106.04(a)(2), subsection I.
The limitation of "multiplying a frequency weight vector by a difference between the input STFT matrix and the output STFT matrix to produce a weighted difference matrix" recites an abstract idea because it describes performing a subtraction operation between two matrices and a multiplication operation between a vector and the resulting matrix. Subtraction and multiplication are mathematical operations, and the limitation is expressed as a mathematical relationship between the recited matrices. This limitation falls within the mathematical concepts grouping of abstract ideas. See MPEP § 2106.04(a)(2), subsection I.
The limitation of "computing an anomaly score for the time-series data samples from the weighted difference matrix" recites an abstract idea because it describes deriving a numerical score from a matrix of values by calculation. Under the broadest reasonable interpretation, the claim does not recite how the anomaly score is computed. To the extent the limitation encompasses evaluating the values in the matrix and forming a judgment as to the degree of abnormality they represent, the limitation also falls within the mental processes grouping of abstract ideas, i.e., concepts performed in the human mind including observation, evaluation, judgment, and opinion. See MPEP § 2106.04(a)(2), subsections I and III.
Accordingly, independent claim 1 recites an abstract idea under Step 2A Prong One because the claim includes limitations falling within the mathematical concepts and mental processes groupings of abstract ideas. Therefore, the analysis proceeds to Step 2A Prong Two.
Independent claim 13 is a method claim reciting limitations similar to claim 1, and additionally recites that the recited matrices "each comprise a plurality of frequency component magnitudes for each of a plurality of time segments," that the network "is trained using supervised learning by computing the anomaly score for a plurality of pre-classified good time-series data samples and providing the anomaly score as feedback for neural network learning," and that "values in the frequency weight vector are determined by performing a gradient ascent optimization, including computing anomaly scores for a plurality of pre-classified time-series data samples comprising both good and bad samples, and iteratively adjusting the values in the frequency weight vector and re-computing the anomaly scores to maximize a difference between the anomaly scores of the good and bad samples." These additional limitations recite an abstract idea because organizing numerical magnitudes into rows and columns is a mathematical representation of information; because computing a score and adjusting parameters in response to that score is a mathematical calculation; and because a gradient ascent optimization is a mathematical optimization algorithm that iteratively adjusts values by means of mathematical calculation. See MPEP § 2106.04(a)(2), subsections I and III; see also 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, Example 47, in which the recited backpropagation and gradient descent algorithms were determined to be mathematical calculations. These limitations recite an abstract idea for similar reasons as claim 1.
Independent claim 14 is a system claim reciting limitations similar to claim 1, including performing a short time Fourier transform to provide an input matrix, computing a reconstructed output matrix, multiplying a frequency weight vector by a difference between the two matrices, and computing an anomaly score from the weighted difference matrix. These limitations recite an abstract idea for similar reasons as claim 1.
Accordingly, under MPEP § 2106.04, subsection II, and MPEP § 2106.04(a)(2), subsections I and III, independent claims 1, 13, and 14 each recite a judicial exception, namely an abstract idea in the form of mathematical concepts and mental processes.
Step 2A Prong 2 -- whether the claim as a whole integrates the recited judicial exception into a practical application of the exception, or whether the claim is "directed to" the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d)
Regarding independent claim 1, this claim recites the additional element of:
"an encoder/decoder neural network"
This limitation amounts to no more than mere instructions to apply the judicial exception using a generic component. The recited encoder/decoder neural network is set forth at a high level of generality, i.e., as a generic model that merely serves as a tool by which the recited mathematical operations are carried out. The claim recites no particular network architecture, no particular training technique, and no particular manner in which the network computes the reconstructed output matrix. See MPEP § 2106.05(f).
The claim does not recite a particular improvement to the functioning of a computer, to the encoder/decoder neural network, or to any other technology or technical field. See MPEP § 2106.05(a). The claim does not recite a particular machine that is integral to the claimed process. See MPEP § 2106.05(b). The claim does not recite a transformation of a particular article to a different state or thing; the recited transformation is a mathematical transformation of data values, which is not a transformation of an article. See MPEP § 2106.05(c). The claim recites no other meaningful limitation that applies the exception in a manner beyond merely implementing the abstract idea using a generic model. See MPEP § 2106.05(e).
It is further noted that claim 1 recites no step of acquiring, receiving, or measuring the recited "time-series data samples," and recites no step of taking any action in response to the computed anomaly score. The claim begins with a mathematical transform performed on data and ends with the computation of a number.
Accordingly, the additional element, individually and in combination with the recited limitations, does not integrate the recited judicial exception into a practical application. Therefore, independent claim 1 is directed to the abstract idea under Step 2A, Prong Two.
Regarding independent claim 13, this claim is drawn to a method claim reciting similar limitations to claim 1 and is rejected under the same rationale. Claim 13 also recites the additional elements of:
"an encoder/decoder neural network" and "supervised learning"
These limitations amount to no more than mere instructions to apply the judicial exception using a generic component and generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.05(f) and 2106.05(h). The recited neural network and the recited supervised learning are set forth at a high level of generality, without any particular architecture, loss function, or training improvement, and merely serve as tools on which the recited mathematical operations are performed. The claim does not recite a particular improvement to the functioning of the neural network or to machine learning technology itself. See MPEP § 2106.05(a).
Accordingly, the additional elements, individually and in combination, do not integrate the judicial exception into a practical application. Therefore, independent claim 13 is directed to the abstract idea under Step 2A, Prong Two.
Regarding independent claim 14, this claim is drawn to a system claim reciting similar limitations to claim 1 and is rejected under the same rationale. Claim 14 also recites the additional elements of:
"a computer having a processor and memory configured with," "a reconstruction-based anomaly detection module," "a frequency weight database including a frequency weight vector," "an anomaly score computation algorithm," and "an encoder/decoder neural network"
These limitations amount to no more than mere instructions to apply the judicial exception using generic computer components and generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.05(f) and 2106.05(h). The recited computer, processor, and memory are generic computer components recited at a high level of generality that merely serve as a tool on which the abstract idea is performed. The recited "module" and "algorithm" are recited functionally, by the mathematical operations they perform, and do not impose any structural or technological limitation beyond the abstract idea itself. The recited "frequency weight database" amounts to no more than the generic storage of the recited weight values in memory and constitutes insignificant extra-solution activity. See MPEP § 2106.05(g).
Accordingly, the additional elements, individually and in combination, do not integrate the judicial exception into a practical application. Therefore, independent claim 14 is directed to the abstract idea under Step 2A, Prong Two.
Step 2B -- whether the claim amounts to significantly more than the judicial exception. See MPEP § 2106.05
Regarding independent claim 1, the claim recites the additional element of:
an encoder/decoder neural network.
This additional element, individually and in combination with the recited limitations, does not amount to significantly more than the judicial exception. The recited encoder/decoder neural network is invoked only as a tool to compute the reconstructed output matrix, without reciting any particular model architecture, training improvement, or technological improvement to the neural network itself. This amounts to no more than instructions to apply the abstract idea using a generic model. See MPEP § 2106.05(f). The use of a neural network to reconstruct an input representation and to derive an anomaly score from the reconstruction error is well-understood, routine, and conventional when recited at this level of generality, as evidenced by applicant's own specification at ¶ [0003], which states that "[i]t is known in the art to use neural network systems, including encoder/decoder neural networks, to perform anomaly detection," and by the references cited in the accompanying rejections under 35 U.S.C. § 103. See MPEP §§ 2106.05(d) and 2106.07(a)(III).
Additionally, limiting the abstract idea to computation performed by an encoder/decoder neural network merely links the abstract idea to a particular technological environment or field of use. See MPEP § 2106.05(h). Accordingly, claim 1 does not include additional elements that amount to significantly more than the judicial exception.
Regarding independent claim 13, the claim recites the additional elements of:
an encoder/decoder neural network; and
supervised learning of the encoder/decoder neural network.
These additional elements, individually and in combination, do not amount to significantly more than the judicial exception, for the reasons set forth above with respect to claim 1. The recited supervised learning is set forth without any particular training algorithm, loss function, or architectural improvement, and amounts to no more than instructions to apply the abstract idea using a generically recited machine learning process. See MPEP §§ 2106.05(f) and 2106.05(h). Accordingly, claim 13 does not include additional elements that amount to significantly more than the judicial exception.
Regarding independent claim 14, the claim recites the additional elements of:
a computer having a processor and memory;
a reconstruction-based anomaly detection module;
a frequency weight database;
an anomaly score computation algorithm; and
an encoder/decoder neural network.
These additional elements, individually and in combination, do not amount to significantly more than the judicial exception. The recited computer, processor, and memory are generic computer components performing generic computer functions, including storing data, processing data, and producing an output. Such generic computer implementation is well-understood, routine, and conventional when recited at this level of generality. See MPEP §§ 2106.05(d) and 2106.07(a)(III). Applicant's specification confirms the generic nature of these components at ¶ [0072], which states that the recited functions may be performed by "one or more computing devices having a processor and a memory module," without describing any particular or improved computer hardware.
The recited "frequency weight database" amounts to mere data storage, which does not provide an inventive concept. See MPEP § 2106.05(g). The recited "module" and "algorithm" are defined solely by the mathematical operations they perform and add nothing beyond the abstract idea itself. Accordingly, claim 14 does not include additional elements that amount to significantly more than the judicial exception.
Regarding dependent claims 2–12 and 15–24
Claims 2–12 and 15–24 merely narrow the previously identified abstract idea limitations recited in independent claims 1 and 14. For the reasons described above with respect to independent claims 1, 13, and 14, the judicial exceptions recited in these dependent claims are not meaningfully integrated into a practical application, nor do they amount to significantly more than the abstract ideas. The additional limitations introduced in the dependent claims further define the abstract idea using matrix organization, feature perturbation, masking, weighting, optimization, norm computation, selection and averaging of values, threshold comparison, and identification of sensor data types. These limitations constitute mathematical concepts and mental processes, including calculating, organizing, adjusting, comparing, selecting, and evaluating numerical data, which are practically capable of being performed in the human mind or with the assistance of pen and paper. Accordingly, dependent claims 2–12 and 15–24 also recite abstract ideas that do not integrate into a practical application and do not amount to significantly more than the judicial exception. Therefore, claims 2–12 and 15–24 are rejected under 35 U.S.C. § 101.
Step 1 -- whether the claim falls within any statutory category. See MPEP § 2106.03
Dependent claims 2–12 depend from independent claim 1, which is drawn to a method. Therefore, each of claims 2–12 falls under at least one of the four categories of statutory subject matter, namely a process.
Dependent claims 15–24 depend from independent claim 14, which is drawn to a system comprising a computer having a processor and memory. Therefore, each of claims 15–24 falls under at least one of the four categories of statutory subject matter, namely a machine.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP § 2106.04, subsection II; MPEP § 2106.04(a)(2), subsections I and III
Claims 2–12
Regarding claim 2, this claim recites the limitation of "the input STFT matrix, the output STFT matrix and the weighted difference matrix each comprise a plurality of frequency component magnitudes for each of a plurality of time segments." This limitation is directed toward the abstract idea of mathematical concepts because arranging numerical magnitude values into rows and columns indexed by frequency and time is a mathematical representation of information. The limitation also involves mental processes, including organizing and arranging information into tabular form, which may be practically performed in the human mind or with pen and paper. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 3, this claim recites the limitation of "a feature jittering operation is performed on the input STFT matrix before it is provided to the encoder/decoder neural network." This limitation is directed toward the abstract idea of mathematical concepts because, as described at applicant's ¶ [0036], the feature jittering operation adds a random variation to the value in each cell of the matrix, which is a mathematical operation performed on numerical values. See MPEP § 2106.04(a)(2), subsection I.
Regarding claim 4, this claim recites the limitations of "a random masking mechanism applied before an encoder module" and "layer-wise query embedding." These limitations are directed toward the abstract idea of mathematical concepts and mental processes because masking values in a matrix amounts to selecting and excluding data from consideration, and because an embedding is a mathematical mapping of values into a representation. Under the broadest reasonable interpretation, the claim recites no particular masking ratio, no particular embedding computation, and no particular structural relationship among the recited elements beyond their sequence. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 5, this claim recites the limitations of "the encoder/decoder neural network is trained using supervised learning by computing the anomaly score for a plurality of pre-classified good time-series data samples" and "providing the anomaly score as feedback for neural network learning." These limitations are directed toward the abstract idea of mathematical concepts because computing a score and using that score to adjust parameter values are mathematical calculations. The limitations also involve mental processes, including classifying samples as good and evaluating the resulting scores. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 6, this claim recites the limitation of "the supervised learning includes a penalty or reinforcement which causes the encoder/decoder neural network to produce low anomaly scores for the good time-series data samples." This limitation is directed toward the abstract idea of mental processes and mathematical concepts because imposing a penalty or reinforcement based on a computed score amounts to evaluating a result and adjusting values accordingly, which is an evaluation and judgment that may be performed in the human mind or with pen and paper. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 7, this claim recites the limitation of "the frequency weight vector has all values set equal to one during the supervised learning of the encoder/decoder neural network." This limitation is directed toward the abstract idea of mathematical concepts because assigning a numerical value of one to each element of a vector is a mathematical operation and defines a mathematical relationship. See MPEP § 2106.04(a)(2), subsection I.
Regarding claim 8, this claim recites the limitations of "values in the frequency weight vector are determined by performing a gradient ascent optimization," "computing anomaly scores for a plurality of pre-classified time-series data samples comprising both good and bad samples," and "iteratively adjusting the values in the frequency weight vector and re-computing the anomaly scores to maximize a difference between the anomaly scores of the good and bad samples." These limitations are directed toward the abstract idea of mathematical concepts because a gradient ascent optimization is a mathematical optimization algorithm that iteratively adjusts numerical values by means of mathematical calculation in order to maximize a function, and because computing scores and computing the difference between them are mathematical calculations. Applicant's specification confirms the mathematical character of the limitation at ¶ [0055], which defines the gradient ascent calculation by the equation w = w + α∇g. See MPEP § 2106.04(a)(2), subsection I; see also 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, Example 47.
Regarding claim 9, this claim recites the limitation of "the gradient ascent optimization of the frequency weight vector is performed after the encoder/decoder neural network is trained to produce low anomaly scores for good time-series data samples." This limitation is directed toward the abstract idea of mathematical concepts and mental processes because it merely specifies the order in which the previously recited mathematical operations are performed. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 10, this claim recites the limitations of "taking a norm of the weighted difference matrix to obtain a vector having a largest frequency component for each time segment," "selecting a quantity of elements of the vector having a greatest value," and "calculating the anomaly score as a mean of the quantity of elements having the greatest value." These limitations are directed toward the abstract idea of mathematical concepts because computing a norm, identifying largest values, and calculating a mean are mathematical calculations. The limitations also involve mental processes, including comparing values and selecting those having the greatest magnitude, which may be practically performed in the human mind or with pen and paper. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 11, this claim recites the limitations of "anomaly scores are computed for unclassified time-series data samples," "the values in the frequency weight vector are optimized to maximize a difference between the anomaly scores of the good and bad samples," and "the unclassified time-series data samples are classified as an anomaly when their anomaly score is above a predefined threshold." These limitations are directed toward the abstract idea of mental processes because comparing a computed score against a threshold and classifying a sample on the basis of that comparison involve observation, evaluation, and judgment. The limitations also involve mathematical concepts because the computation and optimization of the scores are mathematical calculations. See MPEP § 2106.04(a)(2), subsections I and III.
Regarding claim 12, this claim recites that the time-series data samples "include one or more of torque data from a spindle motor of a machine tool, speed data from the spindle motor, and/or data from one or more axial accelerometers mounted on the machine tool." This limitation merely defines the type and source of the information on which the previously identified abstract idea operates. To the extent the limitation requires observing and identifying the type of data being evaluated, it involves mental processes. See MPEP § 2106.04(a)(2), subsection III.
Claims 15–24
Regarding claim 15, this claim recites limitations corresponding to those of claim 2 and recites an abstract idea for the same reasons set forth above with respect to claim 2.
Regarding claim 16, this claim recites limitations corresponding to those of claims 3 and 4 and recites an abstract idea for the same reasons set forth above with respect to claims 3 and 4.
Regarding claim 17, this claim recites limitations corresponding to those of claims 5 and 6 and recites an abstract idea for the same reasons set forth above with respect to claims 5 and 6.
Regarding claim 18, this claim recites limitations corresponding to those of claim 7 and recites an abstract idea for the same reasons set forth above with respect to claim 7.
Regarding claim 19, this claim recites limitations corresponding to those of claim 8 and recites an abstract idea for the same reasons set forth above with respect to claim 8.
Regarding claim 20, this claim recites limitations corresponding to those of claim 9 and recites an abstract idea for the same reasons set forth above with respect to claim 9.
Regarding claim 21, this claim recites limitations corresponding to those of claim 10 and recites an abstract idea for the same reasons set forth above with respect to claim 10.
Regarding claim 22, this claim recites limitations corresponding to those of claim 11 and recites an abstract idea for the same reasons set forth above with respect to claim 11.
Regarding claim 23, this claim recites limitations corresponding to those of claim 12 and recites an abstract idea for the same reasons set forth above with respect to claim 12.
Regarding claim 24, this claim recites the limitations of "data from concurrent time-series data samples containing acceleration data measured in three principle directions on the machine tool are processed concurrently," "concatenating frequency component data from all of the concurrent time-series data samples into a combined weighted difference matrix," and "computing the anomaly score from the combined weighted difference matrix." These limitations are directed toward the abstract idea of mathematical concepts because concatenating sets of numerical values into a single matrix is a mathematical representation and organization of information, and computing a score from that matrix is a mathematical calculation. The limitations also involve mental processes, including organizing and arranging information. See MPEP § 2106.04(a)(2), subsections I and III.
Conclusion under Step 2A Prong One
Accordingly, dependent claims 2–12 and 15–24 recite additional limitations that further define the previously identified abstract ideas using mathematical concepts and mental processes, including transforming, arranging, perturbing, masking, weighting, optimizing, computing norms, selecting, averaging, comparing, and classifying numerical data. These limitations fall within the abstract idea groupings set forth in MPEP § 2106.04(a)(2), subsections I and III. Therefore, dependent claims 2–12 and 15–24 recite judicial exceptions under Step 2A, Prong One.
Step 2A Prong 2 -- whether the claim as a whole integrates the recited judicial exception into a practical application of the exception, or whether the claim is "directed to" the judicial exception. See MPEP 2106.04(d)
Regarding dependent claims 2–12, these claims recite the additional elements of an "encoder/decoder neural network," an "encoder module," and "supervised learning."
These limitations amount to no more than mere instructions to apply the judicial exception using generic components and generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.05(f) and 2106.05(h). The recited neural network, encoder module, and supervised learning are set forth at a high level of generality and merely serve as tools on which the recited mathematical operations are performed. The claims do not recite a particular improvement to the functioning of a computer, to the neural network, or to any other technology or technical field. See MPEP § 2106.05(a). The claims do not recite a particular machine integral to the claim, do not transform an article to a different state or thing, and do not include any other meaningful limitation that integrates the judicial exception into a practical application. See MPEP §§ 2106.05(b), 2106.05(c), and 2106.05(e).
Regarding dependent claims 12 and 23 in particular, the recited "torque data from a spindle motor of a machine tool," "speed data from the spindle motor," and "data from one or more axial accelerometers mounted on the machine tool" amount to insignificant extra-solution activity in the nature of data gathering, because the claims merely specify the source and type of the data upon which the recited mathematical operations are performed. See MPEP § 2106.05(g). To the extent these limitations confine the abstract idea to machine tool monitoring, they merely link the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Regarding dependent claims 15–24, these claims recite the additional elements of a "computer having a processor and memory," a "reconstruction-based anomaly detection module," a "frequency weight database," an "anomaly score computation algorithm," an "encoder/decoder neural network," and an "encoder module." These limitations amount to no more than generic computer components that merely serve as tools on which the abstract idea is performed, together with the generic storage and retrieval of data. See MPEP §§ 2106.05(f) and 2106.05(g). Limiting the abstract idea to execution by a computer system employing a neural network merely links the judicial exception to a technological environment or field of use. See MPEP § 2106.05(h).
Accordingly, the additional elements of dependent claims 2–12 and 15–24, individually and in combination, do not integrate the judicial exception into a practical application.
Step 2B -- whether the claim amounts to significantly more than the judicial exception. See MPEP § 2106.05
Regarding dependent claims 2–12 and 15–24, the additional elements recited in these claims, considered individually and in combination, do not amount to significantly more than the judicial exception, for the same reasons set forth above with respect to independent claims 1, 13, and 14.
The recited computer, processor, memory, and database are generic computer components performing generic computer functions, including storing data, retrieving data, processing data, and producing an output. Such generic computer implementation is well-understood, routine, and conventional when recited at this level of generality. See MPEP §§ 2106.05(d) and 2106.07(a)(III). The recited encoder/decoder neural network, encoder module, and supervised learning are invoked only as tools for performing the recited mathematical operations, without reciting any particular model architecture, training improvement, or improvement to machine learning technology itself. See MPEP § 2106.05(f).
The data-gathering limitations of claims 12 and 23 do not provide an inventive concept, because the acquisition of sensor data from a machine tool for purposes of condition monitoring is well-understood, routine, and conventional, as evidenced by applicant's specification at ¶ [0004], which states that "[t]echniques for anomaly detection using vibration and other machine tool data are known in the art." See MPEP §§ 2106.05(d) and 2106.05(g).
Accordingly, dependent claims 2–12 and 15–24 do not include additional elements that amount to significantly more than the judicial exception, and are rejected under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 5, 6, 7, 14, 15, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jun et al. (Jun), US 11,688,415 B2, in view of Nakanishi et al. (Nakanishi), Non-Patent Literature, "Anomaly Detection By Autoencoder Based On Weighted Frequency Domain Loss," arXiv:2105.10214v1 [eess.IV], published 21 May 2021, pages 1-6.
As to independent Claim 1, Jun teaches a method for reconstruction-based time-series anomaly detection, said method comprising (Jun, col. 7, lines 12-20: "The error between X(i) and Y(i) is named as reconstruction error (RE) which is also represented as a loss function Loss(x(i), y(i)) of the autoencoder"; FIG. 7, elements 702–710: obtain audio signal, generate spectrogram, extract spectrogram features, and "Identify anomalous machine operation based on an autoencoder"):
"performing a short time Fourier transform (STFT) on time-series data samples to provide an input STFT matrix" — Jun teaches this limitation at col. 5, limes 14-16: "The Feature extractor may transform the audio signals into spectrograms in frequency domain using short-term Fourier transform (STFT)," and at col. 6, lines 9-14: "Sound signal obtained at sampling frequency of f_sampling, which is high enough to cover the frequency range of interest [0, f_interest] in monitoring with satisfying Nyquist frequency, are converted into spectrograms using STFT with windowing at every second, therefore the frequency resolution of the spectrograms is 1 Hz." Jun further teaches at col. 5, lines 57-60 that "STFT, which is a Fourier-related transformation, analyzes the frequency and phase content of any segment of a signal from a time-varying system," establishing that the transform is performed on time-series data samples. Jun teaches that the resulting representation is a matrix at col. 5, lines 17-21: "The spectrogram, which is a sound signal magnitude versus frequency is a one-dimensional image (i.e. array), may be extended into a two-dimensional image by concatenating in order of time," and at col. 6, lines 17-22: "The feature extractor may extract spectrogram features from the generated spectrogram (906). For example, acquired signals may be converted into spectrogram at a regular time interval. The PSDs may be filtered up to a predetermined frequency providing a fixed length vector for each PSD."
"providing the input STFT matrix to an encoder/decoder neural network which computes a reconstructed output STFT matrix" — Jun teaches this limitation at col. 5, lines 21-24: "The features, i.e. the spectrogram image, are inputs of autoencoder, also they play the role of reference outputs for training the autoencoder." Jun teaches the encoder/decoder structure at col. 7, lines 7-13, Eq. (5): "(Hidden Layer) Z(i) = σ_enc(W_E X(i)); (Output Layer) Y(i) = σ_dec(W_D Z(i)))”, and at col. 7, lines 12-14: “where W_E and W_D are the weight arrays of encoder and decoder, σ is the activation function, and Z is the output of hidden layer, respectively." Jun teaches computation of the reconstructed output at col. 7, lines 3-5: "The autoencoder tried to adapt output Y(i) close to the original input X(i)," and at col. 7, lines 17-20: "the 'learning' yields minimizing the loss so that the reconstructed images resemble the original inputs (i.e. spectrogram images)."
"computing an anomaly score for the time-series data samples from the weighted difference matrix" — Jun teaches computing an anomaly score for the time-series data samples from a difference matrix at col. 7, lines 46-50: "The anomaly detector may generate a reconstruction error (RE) (912). The reconstruction error may include a measure between the input features and output features of the autoencoder model. The reconstruction error may be determined based on the loss function of the autoencoder." Jun quantifies the score at col 7, lines 23-31, Eq. (6): “(RE) Loss(x(i), y(i)) = (1/n) Σ_{k=1}^{n} (x_k(i) - y_k(i))2.” Jun further teaches at col. 5, lines 49-51 that “the model is used to detect the anomaly by comparing the RE with its corresponding threshold,” and at col. 8, lines 1-3: “The anomaly criteria may include a rule and/or logic that compares the reconstruction error with threshold(s).”
Jun teaches computing a reconstruction error as the difference between the reconstructed output spectrogram matrix and the corresponding input spectrogram matrix, and using that difference to determine an anomaly (Jun, col. 5, lines 45-51; col. 7, lines 46-50). However, Jun does not teach "multiplying a frequency weight vector by a difference between the input STFT matrix and the output STFT matrix to produce a weighted difference matrix."
In the same field of endeavor, Nakanishi teaches multiplying a frequency weight vector by a difference between the input matrix and the output matrix to produce a weighted difference matrix. Nakanishi teaches the difference between the input and reconstructed representations in the frequency domain at page 3, § III.B, Eq. (5): "d(F, F̂) = (1/MN) Σ_{u=0}^{M−1} Σ_{v=0}^{N−1} |F(u,v) − F̂(u,v)|," where F is the frequency representation of the input and F̂ is the frequency representation of the reconstructed output, and where per page 2 "The input image and the reconstructed image are transformed into the frequency domain by DFT, respectively." Nakanishi teaches the frequency weight vector at page 3, § III.B, Eq. (7): "When each frequency component corresponding to a two-dimensional Fourier representation is denoted by F(u,v), the corresponding weight w(u,v) is defined as follows: w(u,v) = √(u² + v²)," and explains at page 3 that "we consider weighting equation (5) according to the frequency value to improve the accuracy of the reconstruction of high-frequency components. The weights should be set so that the higher the frequency, the higher values are taken." Nakanishi teaches the multiplication producing the weighted difference matrix at page 3, § III.B, Eq. (8): "L_WFDL(F, F̂) = (1/MN) Σ_{u=0}^{M−1} Σ_{v=0}^{N−1} w(u,v)|F(u,v) − F̂(u,v)|," in which the frequency-dependent weight w(u,v) is multiplied by the corresponding element of the input-minus-output difference at each frequency coordinate (u,v). Nakanishi further teaches that the anomaly score is derived from the resulting weighted reconstruction at page 3, § III.C: "Autoencoders trained by WFDL are expected to be capable of sufficiently reconstructing high-frequency components. Therefore, by calculating the anomaly score based on the difference between the input image and the reconstructed image, only the anomalous regions can be considered anomalous while edges and fine textures are not."
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection in which an encoder/decoder neural network reconstructs an input representation and an anomaly score is derived from the difference between the input and the reconstruction. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the STFT spectrogram reconstruction and reconstruction-error scoring of Jun with the frequency-weighted difference of Nakanishi. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1, Abstract: "the accuracy of the reconstruction is insufficient in many of these methods, so it leads to degraded accuracy of anomaly detection… we introduce a new loss function named weighted frequency domain loss (WFDL). WFDL provides a sharper reconstructed image, which contributes to improving the accuracy of anomaly detection"), and further at page 1: "when using the Mean Squared Error (MSE) to train an Autoencoder, outputs tend to produce blurry output images that high-frequency components are lost. As a result, fine textures and edges of normal images are detected as defects, which reduces the accuracy of anomaly detection." Jun computes its reconstruction error using precisely such a squared-error loss (Jun, col. 7, Eq. (7)) and therefore exhibits the deficiency Nakanishi identifies, such that one of ordinary skill would have applied the frequency weighting of Nakanishi to the reconstruction error of Jun to obtain the improved anomaly detection accuracy Nakanishi reports (Nakanishi, page 4: "AUROC by WFDL is the best value").
As to Claim 2, the limitations of Claim 1 from which Claim 2 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitation of Claim 2, Jun teaches:
"the input STFT matrix, the output STFT matrix … each comprise a plurality of frequency component magnitudes for each of a plurality of time segments" — Jun, col. 5, lines 17–21: "The spectrogram, which is a sound signal magnitude versus frequency is a one-dimensional image (i.e. array), may be extended into a two-dimensional image by concatenating in order of time." Jun teaches the frequency component msagnitudes at col. 6, lines 12–14: "converted into spectrograms using STFT with windowing at every second, therefore the frequency resolution of the spectrograms is 1 Hz," and teaches that each time segment is represented by a fixed-length vector of such magnitudes at col. 6, lines 17–22: "The feature extractor may extract spectrogram features from the generated spectrogram (906). For example, acquired signals may be converted into spectrogram at a regular time interval. The PSDs may be filtered up to a predetermined frequency providing a fixed length vector for each PSD." Jun teaches that the plurality of such per-segment vectors is assembled into the matrix at col. 6, lines 40–47: "After normalization, successive PSDs are concatenated horizontally along the time axis… the number of n PSDs are combined into two-dimensional (2D) PSD sequences to construct a 2D input for a certain time interval for the autoencoder. The concatenation of 1D PSDs results in 2D images with a size of n." The output matrix has the same composition, as Jun teaches at col. 5, lines 21–24 that the spectrogram images "play the role of reference outputs for training the autoencoder," and at col. 7, lines 17–20 that "the reconstructed images resemble the original inputs (i.e. spectrogram images)."
Jun teaches computing a difference matrix between the input and output spectrogram matrices (Jun, col. 5, lines 45–49: "RE is the difference between reconstructed features, which are the output of autoencoder (i.e. the reconstructed spectrogram image) and its corresponding input"). However, Jun does not teach that "the weighted difference matrix … comprise[s] a plurality of frequency component magnitudes for each of a plurality of time segments."
In the same field of endeavor, Nakanishi teaches that "the weighted difference matrix … comprise[s] a plurality of frequency component magnitudes for each of a plurality of time segments." Nakanishi teaches at page 3, § III.B, Eq. (8): "L_WFDL(F, F̂) = (1/MN) Σ_{u=0}^{M−1} Σ_{v=0}^{N−1} w(u,v)|F(u,v) − F̂(u,v)|," in which the weighted difference is evaluated element by element at every coordinate (u,v) of the M × N frequency representation, such that the weighted difference matrix retains one magnitude value per frequency component per segment. Nakanishi teaches that each such element is a magnitude at page 3, Eq. (6): "|F(u,v)| = √(Re(F(u,v))² + Im(F(u,v))²)," and teaches at page 3 that the weight is indexed to the frequency component: "When each frequency component corresponding to a two-dimensional Fourier representation is denoted by F(u,v), the corresponding weight w(u,v) is defined as follows: w(u,v) = √(u² + v²)."
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection using an encoder/decoder neural network. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the STFT spectrogram matrices of Jun with the element-wise frequency weighting of Nakanishi. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1, Abstract: "the accuracy of the reconstruction is insufficient in many of these methods, so it leads to degraded accuracy of anomaly detection… WFDL provides a sharper reconstructed image, which contributes to improving the accuracy of anomaly detection").
As to Claim 5, the limitations of Claim 1 from which Claim 5 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 5, Jun teaches:
"the encoder/decoder neural network is trained using supervised learning" — Jun, col. 6, lines 60–64: "An autoencoder is one of semi-supervised learning based NN architectures, which is a popular approach in image reconstruction and denoising. The term 'semi-supervised' comes from the aspect that an autoencoder makes use of inputs as targets for reference." Jun further teaches at col. 5, lines 25–28 that the training set is a pre-classified set: "Training an autoencoder. Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder."
"by computing the anomaly score for a plurality of pre-classified good time-series data samples" — Jun, col. 5, lines 25–31: "Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder. To select the structure of autoencoder, changing hyperparameters are performed. Features are extracted from sound spectrograms generating 2D images, and then fed into the autoencoders." Jun teaches that the anomaly score is computed for those samples at col. 7, lines 14–17: "The error between X⁽ⁱ⁾ and Y⁽ⁱ⁾ is named as reconstruction error (RE) which is also represented as a loss function Loss (x⁽ⁱ⁾,y⁽ⁱ⁾) of the autoencoder. RE is computed after calculating output layer," and at col. 7, lines 23–31, Eq. (6): "(RE) Loss(x⁽ⁱ⁾, y⁽ⁱ⁾) = (1/n) Σ_{k=1}^{n} (x_k⁽ⁱ⁾ − y_k⁽ⁱ⁾)²."
"and providing the anomaly score as feedback for neural network learning" — Jun, col. 7, lines 17–20: "the 'learning' yields minimizing the loss so that the reconstructed images resemble the original inputs (i.e. spectrogram images)," and col. 7, lines 39–45: "Next, the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples. This framework uses Adaptive moment estimation (Adam) optimizer which is recommended for faster optimization than other methods such as Momentum optimization or Nesterov Accelerated Gradient." The reconstruction error, which Jun identifies as both the loss function (col. 7, lines 14–17) and the quantity compared against the anomaly threshold (col. 5, lines 45–51), is thereby fed back to update the encoder and decoder weight arrays.
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection using an encoder/decoder neural network trained on normal data. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to train the autoencoder of Jun on pre-classified good samples with the reconstruction error fed back as the learning objective, as modified by the frequency-weighted difference of Nakanishi. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1, Abstract: "WFDL provides a sharper reconstructed image, which contributes to improving the accuracy of anomaly detection"), Nakanishi further teaching at page 1 that "in unsupervised methods, only normal images are used for training, and the model learns their distributions," and at page 4, § IV.A: "For all images, we trained the Autoencoders using the loss function WFDL described in Chapter III."
As to Claim 6, the limitations of Claims 1 and 5 from which Claim 6 depends are rejected under the same rationale set forth above with respect to Claims 1 and 5.
Regarding the additional limitation of Claim 6, Jun teaches:
"the supervised learning includes a penalty … which causes the encoder/decoder neural network to produce low anomaly scores for the good time-series data samples" — Jun, col. 7, lines 14–17: "The error between X⁽ⁱ⁾ and Y⁽ⁱ⁾ is named as reconstruction error (RE) which is also represented as a loss function Loss (x⁽ⁱ⁾,y⁽ⁱ⁾) of the autoencoder," and col. 7, lines 23–31, Eq. (6): "(RE) Loss(x⁽ⁱ⁾, y⁽ⁱ⁾) = (1/n) Σ_{k=1}^{n} (x_k⁽ⁱ⁾ − y_k⁽ⁱ⁾)²." Jun teaches that this loss operates as a penalty imposed during training at col. 7, lines 17–20: "the 'learning' yields minimizing the loss so that the reconstructed images resemble the original inputs (i.e. spectrogram images)," and at col. 7, lines 39–42: "the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples." Jun teaches that the effect of the penalty is to produce low anomaly scores for the good samples and higher scores for anomalous samples at col. 7, lines 50–55: "To classify anomalous signals from normal signals by (REs), the autoencoder should be trained purely with normal signals. After training without abnormal signals, the autoencoder produces larger RE when 'unseen' data from abnormal status are fed in as input." Because Jun trains exclusively on samples pre-classified as good (col. 5, lines 25–28), and because the quantity so minimized is the same reconstruction error Jun uses as the anomaly score (col. 5, lines 45–51: "RE is the difference between reconstructed features… the model is used to detect the anomaly by comparing the RE with its corresponding threshold"), the penalty imposed by the loss function necessarily drives the encoder/decoder neural network to produce low anomaly scores for the good time-series data samples.
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection using an encoder/decoder neural network trained by minimizing a reconstruction loss over normal samples. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to train the autoencoder of Jun by minimizing the reconstruction loss over pre-classified good samples, as modified by the frequency-weighted loss of Nakanishi. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1: "when using the Mean Squared Error (MSE) to train an Autoencoder, outputs tend to produce blurry output images that high-frequency components are lost. As a result, fine textures and edges of normal images are detected as defects, which reduces the accuracy of anomaly detection"), such that one would adopt the weighted frequency domain loss of Nakanishi as the penalty term in Jun, which expressly assigns mean-squared error as its loss function (Jun, col. 7, lines 36–38: "For the loss function, mean-squared error is assigned").
As to Claim 7, the limitations of Claims 1 and 5 from which Claim 7 depends are rejected under the same rationale set forth above with respect to Claims 1 and 5.
Regarding the additional limitation of Claim 7, Jun teaches:
"during the supervised learning of the encoder/decoder neural network" — Jun, col. 5, lines 25–28: "Training an autoencoder. Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder"; col. 7, lines 39–42: "the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples."
Jun teaches that the objective minimized during training applies equal weight to every component of the difference, the loss being an unweighted mean of squared errors (Jun, col. 7, lines 23–31, Eq. (6): "(RE) Loss(x⁽ⁱ⁾, y⁽ⁱ⁾) = (1/n) Σ_{k=1}^{n} (x_k⁽ⁱ⁾ − y_k⁽ⁱ⁾)²"; col. 7, lines 36–38: "For the loss function, mean-squared error is assigned"). However, Jun does not teach that "the frequency weight vector has all values set equal to one during the supervised learning of the encoder/decoder neural network."
In the same field of endeavor, Nakanishi teaches that "the frequency weight vector has all values set equal to one during the supervised learning of the encoder/decoder neural network." Nakanishi teaches the frequency weight vector at page 3, § III.B, Eq. (7): "the corresponding weight w(u,v) is defined as follows: w(u,v) = √(u² + v²)." Nakanishi teaches the state in which that vector has all values equal to one, and the effect of that state, at page 2, § II, Eq. (1): "MSE(f, f̂) = Σ_{x=0}^{M−1} Σ_{y=0}^{N−1} (f(x,y) − f̂(x,y))²," in which no frequency-dependent weight is applied and every component is therefore weighted by unity; at page 3: "This is because the loss function of the Autoencoder is defined as the L2 norm of the error"; and at page 4, § IV.A: "We use the Autoencoder with L2 norm loss and the Autoencoder with SSIM loss for baselines." Nakanishi thereby teaches that the conventional autoencoder trained without frequency weighting is the uniform-unit-weight case of the same formulation, distinguished from the weighted case only by the value assigned to w(u,v) in Eq. (8).
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection using an encoder/decoder neural network. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to train the autoencoder of Jun with the frequency weight vector at unity, that is, with the unweighted mean-squared-error loss of Jun (col. 7, lines 23–31, Eq. (6)), and to apply the non-uniform frequency weight vector of Nakanishi to the difference used for anomaly scoring. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1, Abstract: "the accuracy of the reconstruction is insufficient in many of these methods, so it leads to degraded accuracy of anomaly detection… WFDL provides a sharper reconstructed image, which contributes to improving the accuracy of anomaly detection").
As to independent Claim 14, the limitations "performs a short time Fourier transform (STFT) on time-series data samples and provides an input STFT matrix to an encoder/decoder neural network which computes a reconstructed output STFT matrix"; "a frequency weight vector which is multiplied by a difference between the input STFT matrix and the output STFT matrix to produce a weighted difference matrix"; and "computes an anomaly score for the time-series data samples from the weighted difference matrix" are rejected under the same rationale set forth above with respect to Claims 1 and 13.
Regarding the remaining limitations of Claim 14, Jun teaches:
"A reconstruction-based time-series anomaly detection system, said system comprising:" — Jun, claim 11 (col. 14, lines 47–52): "A system comprising: an audio capture device configured to generate a signal in response to sound separately generated a first component and a second component of a mechanical apparatus."
"a computer having a processor and memory configured with[:]" — Jun, claim 11 (col. 14, lines 52–58): "a processor, the processor configured to: receive the signal generated by the microphone of the audio capture device… determine, based on an autoencoder neural network and the signal, an anomalous event." Jun teaches the memory at col. 11, lines 9–16: "The memory 820 may be any device for storing and retrieving data or any combination thereof. The memory 820 may include non-volatile and/or volatile memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or flash memory. Alternatively or in addition, the memory 820 may include an optical, magnetic (hard-drive), solid-state drive or any other form of data storage device." Jun teaches that the processor and memory are configured with the recited functional components at col. 11, lines 1–8: "in other memory that when executed by the processor 816, cause the processor 816 to perform the operations of the audio processing controller 106, input controller 602, the feature extractor 604, the anomaly detector 606, the autoencoder model 608, the classifier 610, the classification model 612, the output controller 614, and/or system 100. The computer code may include instructions executable with the processor 816," and at col. 11, lines 17–22: "The memory 820 may include at least one of the processor 816, cause the processor 816 to perform the operations of the audio processing controller 106, the input controller 602, the feature extractor 604, the anomaly detector 606, the autoencoder model 608, the classifier 610, the classification model 612, the output controller 614, and/or system 100."
"a reconstruction-based anomaly detection module which…" — Jun, claim 12 (col. 14, line 63 through col. 15, line 6): "wherein to determine, based on the autoencoder neural network and the signal, the anomalous event, the processor is further configured to: generate a spectrogram based on the signal; generate a feature set for the machine learning model based on the spectrogram, or a portion thereof; generate, based on the feature set and the machine learning model, a reconstruction error measurement; and determine the reconstruction error satisfies an anomaly criteria." Jun teaches the module structure at col. 11, lines 3–6, identifying "the feature extractor 604, the anomaly detector 606, the autoencoder model 608" as components whose operations are performed by the processor, and at col. 6, lines 56–59: "The anomaly detector may reconstruct spectrogram features based on the prepared spectrogram features (910). For example, anomaly detector may access the autoencoder model 608 for the reconstruction."
"an anomaly score computation algorithm which…" — Jun, col. 7, lines 46–50: "The anomaly detector may generate a reconstruction error (RE) (912). The reconstruction error may include a measure between the input features and output features of the autoencoder model. The reconstruction error may be determined based on the loss function of the autoencoder," and col. 8, lines 1–3: "The anomaly criteria may include a rule and/or logic that compares the reconstruction error with threshold(s)."
Jun teaches storing in the memory of the audio processing controller the parameters and models used by the anomaly detector, including the autoencoder model 608 (Jun, col. 11, lines 17–22). However, Jun does not teach "a frequency weight database including a frequency weight vector."
In the same field of endeavor, Nakanishi teaches "a frequency weight database including a frequency weight vector." Nakanishi teaches the frequency weight vector at page 3, § III.B, Eq. (7): "When each frequency component corresponding to a two-dimensional Fourier representation is denoted by F(u,v), the corresponding weight w(u,v) is defined as follows: w(u,v) = √(u² + v²)." Nakanishi thereby teaches a set of weight values, one for each frequency component (u,v) of the frequency representation, which set is maintained and accessed when evaluating the weighted difference at page 3, Eq. (8): "L_WFDL(F, F̂) = (1/MN) Σ_{u=0}^{M−1} Σ_{v=0}^{N−1} w(u,v)|F(u,v) − F̂(u,v)|." Nakanishi further teaches that the values so maintained are assigned according to frequency at page 3: "we consider weighting equation (5) according to the frequency value to improve the accuracy of the reconstruction of high-frequency components. The weights should be set so that the higher the frequency, the higher values are taken."
Jun and Nakanishi are analogous to the claimed invention as both are from the same field of endeavor of reconstruction-based anomaly detection using an encoder/decoder neural network. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to store the frequency weight values of Nakanishi in the memory 820 of the system of Jun and to apply them to the reconstruction error Jun computes. The motivation to combine Jun and Nakanishi is as recited by Nakanishi (page 1, Abstract: "the accuracy of the reconstruction is insufficient in many of these methods, so it leads to degraded accuracy of anomaly detection… WFDL provides a sharper reconstructed image, which contributes to improving the accuracy of anomaly detection").
As to Claim 15, Claim 15 recites limitations corresponding to those of Claim 2. Claim 15 is therefore rejected under the same rationale set forth above with respect to Claims 2 and 14.
As to Claim 17, Claim 17 recites limitations corresponding to those of Claims 5 and 6. Claim 17 is therefore rejected under the same rationale set forth above with respect to Claims 5, 6, and 14.
As to Claim 18, Claim 18 recites limitations corresponding to those of Claim 7. Claim 18 is therefore rejected under the same rationale set forth above with respect to Claims 7 and 17.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Jun, in view of Nakanishi, and further in view of You et al. (You), Non-Patent Literature, "A Unified Model for Multi-class Anomaly Detection," arXiv:2206.03687v3 [cs.CV], published 25 Oct 2022, pages 1-19, cited in the IDS filed 25 April 2024.
As to Claim 3, the limitations of Claim 1 from which Claim 3 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitation of Claim 3, Jun teaches:
"the input STFT matrix … is provided to the encoder/decoder neural network" — Jun, col. 5, lines 21–23: "The features, i.e. the spectrogram image, are inputs of autoencoder," and col. 6, lines 56–59: "The anomaly detector may reconstruct spectrogram features based on the prepared spectrogram features (910). For example, anomaly detector may access the autoencoder model 608 for the reconstruction."
Jun teaches preprocessing operations applied to the spectrogram features prior to input to the autoencoder, including filtering (Jun, col. 6, lines 19–22: "The PSDs may be filtered up to a predetermined frequency providing a fixed length vector for each PSD") and normalization (Jun, col. 6, lines 31–33: "the filtered PSDs may be normalized to have values within 0 and 1 using (eqn3)"). However, the combination of Jun and Nakanishi does not teach "wherein a feature jittering operation is performed on the input STFT matrix before it is provided to the encoder/decoder neural network."
In the same field of endeavor, You teaches "wherein a feature jittering operation is performed on the input STFT matrix before it is provided to the encoder/decoder neural network." You teaches the feature jittering operation at page 5, § 3.2: "we propose a Feature Jittering (FJ) strategy to add perturbations to the input features, leading the model to learn normal distribution from the denoising task," and at page 4: "We employ a Feature Jittering (FJ) strategy to disturb the input features, leading the model to learn normal distribution from denoising." You teaches the operation itself at page 6: "Inspired by Denoising Auto-Encoder (DAE), we add perturbations to feature tokens, guiding the model to learn knowledge of normal samples by the denoising task. Specifically, for a feature token, f_tok ∈ R^C, we sample the disturbance D from a Gaussian distribution, D ~ N(µ = 0, σ² = (α‖f_tok‖₂/C)²), where α is the jittering scale to control the noisy degree. Also, the sampled disturbance is added to f_tok with a fixed jittering probability, p." You teaches that the operation is performed on the input matrix before it is provided to the encoder/decoder neural network at page 5, FIG. 3, which places "Feature Jittering" upstream of the Neighbor Masked Encoder (NME) and the Layer-wise Query Decoder (LQD), and at page 5: "the feature tokens extracted by a fixed pre-trained backbone are further integrated by NME to derive the encoder embeddings."
Jun, Nakanishi, and You are analogous to the claimed invention as all are from the same field of endeavor of reconstruction-based anomaly detection in which an encoder/decoder neural network reconstructs an input representation and an anomaly score is derived from the difference between the input and the reconstruction. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply the feature jittering operation of You to the spectrogram matrix input to the autoencoder of Jun as modified by Nakanishi. The motivation to combine Jun, Nakanishi, and You is as recited by You (page 1, Abstract: "we propose a feature jittering strategy that urges the model to recover the correct message even with noisy inputs," addressing the problem that a reconstruction network "may fall into an 'identical shortcut', where both normal and anomalous samples can be well recovered, and hence fail to spot outliers"), such that one would apply feature jittering to the reconstruction network of Jun, which trains the autoencoder to "adapt output Y⁽ⁱ⁾ close to the original input X⁽ⁱ⁾" (Jun, col. 7, lines 3–5) and is therefore susceptible to the same identical-shortcut deficiency You identifies.
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Jun, in view of Nakanishi, further in view of Feichtenhofer et al. (Feichtenhofer), Non-Patent Literature, "Masked Autoencoders As Spatiotemporal Learners," arXiv:2205.09113v2 [cs.CV], published 21 October 2022, pages 1-18, cited in the IDS filed 25 April 2024, and further in view of You et al. (You), Non-Patent Literature, "A Unified Model for Multi-class Anomaly Detection.", arXiv:2206.03687v3 [cs.CV], published 25 October 2022, pages 1-19, cited in the IDS filed 25 April 2024.
As to Claim 4, the limitations of Claim 1 from which Claim 4 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 4, Jun teaches:
"the encoder/decoder neural network includes … an encoder module" — Jun, col. 7, lines 7–11, Eq. (5): "(Hidden Layer) Z⁽ⁱ⁾=σ_enc(W_E X⁽ⁱ⁾); (Output Layer) Y⁽ⁱ⁾=σ_dec(W_D Z⁽ⁱ⁾)," and col. 7, lines 12–14: "where, W_E and W_D are the weight arrays of encoder and decoder, σ is the activation function, and Z is the output of hidden layer, respectively." Jun further teaches at col. 7, lines 1–2: "example, the general type of stacked autoencoder is used," and at col. 7, lines 33–36: "the activation functions σT_enc(x) and σ_dec(x) are also known as the Rectified Linear Unit (ReLU) and the sigmoid function, respectively. In this study, σ_enc(x) is used for encoding (data compression)."
Jun teaches filtering and normalization operations applied to the spectrogram features upstream of the autoencoder (Jun, col. 6, lines 19–22: "The PSDs may be filtered up to a predetermined frequency providing a fixed length vector for each PSD"; col. 6, lines 31–33: "the filtered PSDs may be normalized to have values within 0 and 1 using (eqn3)"). However, the combination of Jun and Nakanishi does not teach "a random masking mechanism applied before an encoder module."
In the same field of endeavor, Feichtenhofer teaches "a random masking mechanism applied before an encoder module." Feichtenhofer teaches the random masking mechanism at page 1, Abstract: "We randomly mask out spacetime patches in videos and learn an autoencoder to reconstruct them in pixels," and at page 4, § "Masking": "We sample random patches without replacement from the set of embedded patches." Feichtenhofer teaches that the masking is applied before the encoder module at page 2, FIG. 1: "We mask a large subset (e.g., 90%) of random patches in spacetime. An encoder operates on the set of visible patches. A small decoder then processes the full set of encoded patches and mask tokens to reconstruct the input"; at page 4, § "Autoencoding": "Our encoder is a vanilla ViT applied only on the visible set of embedded patches"; and at page 2: "Following the MAE that applies the encoder only on visible tokens, a masking ratio of 90% reduces the encoder time." Feichtenhofer further teaches at page 1 that "the optimal masking ratio is as high as 90%."
Jun, Nakanishi, and Feichtenhofer are analogous to the claimed invention as all are from the same field of endeavor of encoder/decoder neural networks that reconstruct an input representation. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply the random masking mechanism of Feichtenhofer before the encoder module of the autoencoder of Jun as modified by Nakanishi. The motivation to combine Jun, Nakanishi, and Feichtenhofer is as recited by Feichtenhofer (page 1, Abstract: "A high masking ratio leads to a large speedup, e.g., > 4× in wall-clock time or even more," and page 2: "a masking ratio of 90% reduces the encoder time"), such that one would obtain the reduced encoder computation time Feichtenhofer reports while training the reconstruction network of Jun.
The combination of Jun, Nakanishi, and Feichtenhofer, however, does not teach "layer-wise query embedding."
In the same field of endeavor, You teaches "layer-wise query embedding." You teaches at page 6: "we design a Layer-wise Query Decoder (LQD) to intensify the use of query embedding, as shown in Fig. 3. Specifically, in each layer of LQD, a learnable query embedding is first fused with the encoder embeddings, then integrated with the outputs of the previous layer (self-integration for the first layer)." You teaches the same at page 5, FIG. 3: "Each layer in LQD employs a learnable query embedding to help model the complex training data distribution," and at page 2: "we propose a layer-wise query decoder to intensify the use of query embedding." You further teaches at page 4: "the query embedding is of vital significance," and "1) According to that the query embedding can prevent reconstructing anomalies, we design a Layer-wise Query Decoder (LQD) by adding the query embedding in each decoder layer rather than only [the first]."
Jun, Nakanishi, Feichtenhofer, and You are analogous to the claimed invention as all are from the same field of endeavor of encoder/decoder neural networks that reconstruct an input representation. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the layer-wise query embedding of You into the decoder of the autoencoder of Jun as modified by Nakanishi and Feichtenhofer. The motivation to combine Jun, Nakanishi, Feichtenhofer, and You is as recited by You (page 1, Abstract: the reconstruction network "may fall into an 'identical shortcut', where both normal and anomalous samples can be well recovered, and hence fail to spot outliers," and page 4: "the query embedding is of vital significance"), such that one would incorporate the layer-wise query embedding into the autoencoder of Jun, which trains the network to "adapt output Y⁽ⁱ⁾ close to the original input X⁽ⁱ⁾" (Jun, col. 7, lines 3–5) and is therefore susceptible to the identical-shortcut deficiency You identifies.
As to Claim 16, Claim 16 recites limitations corresponding to those of Claims 3 and 4. Claim 16 is therefore rejected under the same rationale set forth above with respect to Claims 3, 4, and 14.
Claims 8, 9, 11, 13, 19, 20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Jun, in view of Nakanishi, and further in view of Wichern et al. (Wichern), US 2024/0003737 A1.
As to Claim 8, the limitations of Claim 1 from which Claim 8 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 8, Jun teaches:
"computing anomaly scores for a plurality of pre-classified time-series data samples comprising both good and bad samples" — Jun, col. 5, lines 36–44: "Testing the trained model and making decision (i.e. detecting anomaly). The signals under 'normal' and 'anomalous' conditions are collected for testing. Again, the features are extracted in the same way, then feed forward into the trained autoencoder. Here, different weights are loaded on the end effector of robot. In normal conditions, the weight less than the allowable level of the robot is loaded, whereas heavier loads attached to the robot are regarded as anomalous state in this study." Jun teaches that the anomaly score is computed for each such sample at col. 5, lines 46–49: "RE is the difference between reconstructed features, which are the output of autoencoder (i.e. the reconstructed spectrogram image) and its corresponding input," and teaches that the scores of the two pre-classified groups are then compared against one another at col. 5, lines 44–46: "feasible thresholds are preliminarily set between 'normal' and 'abnormal' status in each axis." Jun further teaches at col. 7, lines 50–55: "To classify anomalous signals from normal signals by (REs), the autoencoder should be trained purely with normal signals. After training without abnormal signals, the autoencoder produces larger RE when 'unseen' data from abnormal status are fed in as input."
Jun teaches gradient-based iterative optimization at col. 7, lines 39–45: "Next, the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples. This framework uses Adaptive moment estimation (Adam) optimizer which is recommended for faster optimization than other methods such as Momentum optimization or Nesterov Accelerated Gradient." However, the combination of Jun and Nakanishi does not teach the following limitations of Claim 8.
In the same field of endeavor, Wichern teaches:
"values in the frequency weight vector are determined by performing a gradient ascent optimization" — Wichern, ¶ [0090], Eq. (5): "D_NN^wt(z_S, z_A, D) = w_S·D_NN(z_S, D) + w_A·D_NN(z_A, D)," and ¶ [0091]: "Where w_S and w_A are scalar weights, which are optimized after training is complete." Wichern teaches that the weights so optimized are indexed to the frequency-resolved representation of the sensed signal at ¶ [0086]: "The dataset D={(X⁽ⁿ⁾, y⁽ⁿ⁾)}_{n=1}^N, where X∈R^(F×T) is a magnitude spectrogram with F frequencies and T time frames."
"iteratively adjusting the values in the frequency weight vector" — Wichern, ¶ [0085]: "An additional approach for determining classifier weights of different dimensions could be after some known anomalies are observed, the anomalous sound detection system 102 may adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers." Wichern further teaches at ¶ [0085]: "the anomalous sound detection system 102 can determine the weights between the different classifiers, based on how accurate classifiers are trained (i.e., accurate classifiers have higher weights)."
"and re-computing the anomaly scores" — Wichern, ¶ [0085]: "By computing separate anomaly scores for each classifier at inference time, the anomalous sound detection system 102 can determine anomaly scores from embedding vectors that were trained by different classifiers (e.g. the first classifier 204a and the second classifier 204b) differently." Wichern teaches that the scores are recomputed under the revised weights at ¶ [0080]: "Such a weighted combination increases the accuracy and performance of the multi-head neural network 112 by putting less emphasis on redundant processing operations."
"to maximize a difference between the anomaly scores of the good and bad samples" — Wichern, ¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence." Wichern teaches that the separation so obtained is evaluated against a threshold dividing the two classes at ¶ [0078]: "the received audio signal 110 is determined to be anomalous when the anomaly score is above a pre-specified threshold value."
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a neural network processes a spectrogram of a sensed machine signal and an anomaly score is derived therefrom. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to determine the values of the frequency weight vector of Nakanishi by the post-training weight optimization of Wichern, iteratively adjusting those values against the pre-classified good and bad samples of Jun using the gradient-based optimization Jun already employs (Jun, col. 7, lines 39–45). The motivation to combine Jun, Nakanishi, and Wichern is as recited by Wichern (¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers") and by Nakanishi (page 3: increasing the frequency weights "can be expected to increase the gradient on the Autoencoder's reconstruction error of high-frequency components"), such that one would optimize the frequency weight values rather than fix them, thereby increasing the separation between the anomaly scores of the good and bad samples that Jun already computes and compares (Jun, col. 5, lines 36–46).
As to Claim 9, the limitations of Claims 1 and 8 from which Claim 9 depends are rejected under the same rationale set forth above with respect to Claims 1 and 8.
Regarding the additional limitations of Claim 9, Jun teaches:
"the encoder/decoder neural network is trained to produce low anomaly scores for good time-series data samples" — Jun, col. 5, lines 25–28: "Training an autoencoder. Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder." Jun teaches that the training objective drives the score downward on those samples at col. 7, lines 17–20: "the 'learning' yields minimizing the loss so that the reconstructed images resemble the original inputs (i.e. spectrogram images)," and at col. 7, lines 39–42: "the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples." Jun teaches the resulting effect at col. 7, lines 50–55: "To classify anomalous signals from normal signals by (REs), the autoencoder should be trained purely with normal signals. After training without abnormal signals, the autoencoder produces larger RE when 'unseen' data from abnormal status are fed in as input."
Jun teaches that the network is trained before anomalous samples are evaluated, stating at col. 5, lines 37–40: "The signals under 'normal' and 'anomalous' conditions are collected for testing. Again, the features are extracted in the same way, then feed forward into the trained autoencoder," and at col. 7, lines 56–59: "Accordingly, the autoencoder model 408 may be trained with features from normal (i.e. non-anomalous) operating conditions. After training, features derived in real-time may be fed into the autoencoder to measure RE values." However, the combination of Jun and Nakanishi does not teach the following limitation of Claim 9.
In the same field of endeavor, Wichern teaches:
"the gradient ascent optimization of the frequency weight vector is performed after the encoder/decoder neural network is trained" — Wichern, ¶ [0091]: "Where w_S and w_A are scalar weights, which are optimized after training is complete." Wichern further teaches that the weight optimization is a procedure separate from and subsequent to network training, and leaves the trained network unaltered, at ¶ [0085]: "An additional approach for determining classifier weights of different dimensions could be after some known anomalies are observed, the anomalous sound detection system 102 may adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers." Wichern teaches that the weight adjustment accordingly occurs at inference time on an already-trained model at ¶ [0085]: "By computing separate anomaly scores for each classifier at inference time, the anomalous sound detection system 102 can determine anomaly scores from embedding vectors that were trained by different classifiers… differently," and at ¶ [0084]: "once the classifier is trained, the anomalous sound detection system 102 may not know a specific environment of an individual drilling machine at inference time."
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a neural network processes a spectrogram of a sensed machine signal and an anomaly score is derived therefrom. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to perform the optimization of the frequency weight vector of Nakanishi after the autoencoder of Jun has been trained on good samples, in the manner taught by Wichern. The motivation to combine Jun, Nakanishi, and Wichern is as recited by Wichern (¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers"), such that one would obtain improved separation between good and bad anomaly scores without incurring the cost of retraining the encoder/decoder neural network of Jun.
As to Claim 11, the limitations of Claim 1 from which Claim 11 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 11, Jun teaches:
"anomaly scores are computed for unclassified time-series data samples" — Jun, col. 7, lines 56–60: "Accordingly, the autoencoder model 408 may be trained with features from normal (i.e. non-anomalous) operating conditions. After training, features derived in real-time may be fed into the autoencoder to measure RE values." Jun teaches computation of the score for such samples at col. 7, lines 46–50: "The anomaly detector may generate a reconstruction error (RE) (912). The reconstruction error may include a measure between the input features and output features of the autoencoder model. The reconstruction error may be determined based on the loss function of the autoencoder."
"after the encoder/decoder neural network is trained to produce low anomaly scores for good time-series data samples" — Jun, col. 5, lines 25–28: "Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder"; col. 7, lines 17–20: "the 'learning' yields minimizing the loss so that the reconstructed images resemble the original inputs (i.e. spectrogram images)"; col. 7, lines 50–55: "To classify anomalous signals from normal signals by (REs), the autoencoder should be trained purely with normal signals. After training without abnormal signals, the autoencoder produces larger RE when 'unseen' data from abnormal status are fed in as input."
"where the unclassified time-series data samples are classified as an anomaly when their anomaly score is above a predefined threshold" — Jun, col. 5, lines 44–51: "feasible thresholds are preliminarily set between 'normal' and 'abnormal' status in each axis… After the thresholds are set for each joint, the model is used to detect the anomaly by comparing the RE with its corresponding threshold." Jun further teaches at col. 8, lines 1–3: "criterion is satisfied (914). The anomaly criteria may include a rule and/or logic that compares the reconstruction error with threshold(s)," and at col. 8, lines 10–11: "In response to satisfaction of the anomaly criteria, the anomaly detector may generate an anomalous event (916)." Jun teaches the setting of the threshold at col. 7, lines 59–62: "During training time, by comparing the distributions of RE, a threshold can be set to distinguish the normal and the abnormal status."
Jun teaches that the thresholds are preliminarily set by reference to the reconstruction errors observed for pre-classified normal and abnormal samples (Jun, col. 5, lines 36–46; col. 7, lines 59–62). However, the combination of Jun and Nakanishi does not teach the following limitation of Claim 11.
In the same field of endeavor, Wichern teaches:
"the values in the frequency weight vector are optimized to maximize a difference between the anomaly scores of the good and bad samples" — Wichern, ¶ [0090], Eq. (5): "D_NN^wt(z_S, z_A, D) = w_S·D_NN(z_S, D) + w_A·D_NN(z_A, D)," and ¶ [0091]: "Where w_S and w_A are scalar weights, which are optimized after training is complete." Wichern teaches that the objective governing the optimization is increased separation of known anomalies from normal samples at ¶ [0085]: "An additional approach for determining classifier weights of different dimensions could be after some known anomalies are observed, the anomalous sound detection system 102 may adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers." Wichern teaches that the weights so optimized are indexed to the frequency-resolved representation of the sensed signal at ¶ [0086]: "The dataset D={(X⁽ⁿ⁾, y⁽ⁿ⁾)}_{n=1}^N, where X∈R^(F×T) is a magnitude spectrogram with F frequencies and T time frames." Wichern further teaches that the resulting score is compared against a threshold to classify an unclassified sample at ¶ [0078]: "the received audio signal 110 is determined to be anomalous when the anomaly score is above a pre-specified threshold value… may be 0.5."
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a neural network processes a spectrogram of a sensed machine signal and an anomaly score is derived therefrom. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to optimize the values of the frequency weight vector of Nakanishi in the manner taught by Wichern before applying the trained detector of Jun to unclassified samples and comparing the resulting anomaly score against the threshold Jun establishes. The motivation to combine Jun, Nakanishi, and Wichern is as recited by Wichern (¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers"), such that one would increase the separation between the good and bad anomaly scores that Jun already relies upon in setting its thresholds (Jun, col. 7, lines 59–62: "by comparing the distributions of RE, a threshold can be set to distinguish the normal and the abnormal status"), thereby improving the reliability of the threshold classification.
As to independent Claim 13, the preamble and the limitations "performing a short time Fourier transform (STFT) on time-series data samples to provide an input STFT matrix"; "providing the input STFT matrix to an encoder/decoder neural network which computes a reconstructed output STFT matrix"; "multiplying a frequency weight vector by a difference between the input STFT matrix and the output STFT matrix to produce a weighted difference matrix"; and "computing an anomaly score for the time-series data samples from the weighted difference matrix" are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the remaining limitations of Claim 13, Jun teaches:
"where the input STFT matrix, the output STFT matrix and the weighted difference matrix each comprise a plurality of frequency component magnitudes for each of a plurality of time segments" — this limitation is rejected under the same rationale set forth above with respect to Claim 2. Jun, col. 5, lines 17–21: "The spectrogram, which is a sound signal magnitude versus frequency is a one-dimensional image (i.e. array), may be extended into a two-dimensional image by concatenating in order of time"; col. 6, lines 12–14: "converted into spectrograms using STFT with windowing at every second, therefore the frequency resolution of the spectrograms is 1 Hz"; col. 6, lines 19–22: "The PSDs may be filtered up to a predetermined frequency providing a fixed length vector for each PSD"; col. 6, lines 40–47: "After normalization, successive PSDs are concatenated horizontally along the time axis… the number of n PSDs are combined into two-dimensional (2D) PSD sequences to construct a 2D input for a certain time interval for the autoencoder."
"where the encoder/decoder neural network is trained using supervised learning by computing the anomaly score for a plurality of pre-classified good time-series data samples and providing the anomaly score as feedback for neural network learning" — this limitation is rejected under the same rationale set forth above with respect to Claim 5. Jun, col. 5, lines 25–28: "Training an autoencoder. Audio signals, which are considered as 'normal' or 'acceptable', are collected to train the autoencoder"; col. 6, lines 60–64: "An autoencoder is one of semi-supervised learning based NN architectures, which is a popular approach in image reconstruction and denoising. The term 'semi-supervised' comes from the aspect that an autoencoder makes use of inputs as targets for reference"; col. 7, lines 14–17: "The error between X⁽ⁱ⁾ and Y⁽ⁱ⁾ is named as reconstruction error (RE) which is also represented as a loss function Loss (x⁽ⁱ⁾,y⁽ⁱ⁾) of the autoencoder. RE is computed after calculating output layer"; col. 7, lines 39–42: "the network parameters W_E and W_D are updated by back-propagation algorithm. The parameters are adjusted where the loss function defined in Eq. (6) is minimized for all training examples."
"including computing anomaly scores for a plurality of pre-classified time-series data samples comprising both good and bad samples" — this limitation is rejected under the same rationale set forth above with respect to Claim 8. Jun, col. 5, lines 36–44: "Testing the trained model and making decision (i.e. detecting anomaly). The signals under 'normal' and 'anomalous' conditions are collected for testing. Again, the features are extracted in the same way, then feed forward into the trained autoencoder. Here, different weights are loaded on the end effector of robot. In normal conditions, the weight less than the allowable level of the robot is loaded, whereas heavier loads attached to the robot are regarded as anomalous state in this study."
"and anomaly scores are computed for unclassified time-series data samples after the encoder/decoder neural network is trained" — Jun, col. 7, lines 56–59: "Accordingly, the autoencoder model 408 may be trained with features from normal (i.e. non-anomalous) operating conditions. After training, features derived in real-time may be fed into the autoencoder to measure RE values"; col. 7, lines 46–50: "The anomaly detector may generate a reconstruction error (RE) (912). The reconstruction error may include a measure between the input features and output features of the autoencoder model."
Jun teaches gradient-based iterative optimization at col. 7, lines 39–45: "This framework uses Adaptive moment estimation (Adam) optimizer which is recommended for faster optimization than other methods such as Momentum optimization or Nesterov Accelerated Gradient." However, the combination of Jun and Nakanishi does not teach the following limitations of Claim 13.
In the same field of endeavor, Wichern teaches:
"and where values in the frequency weight vector are determined by performing a gradient ascent optimization" — Wichern, ¶ [0090], Eq. (5): "D_NN^wt(z_S, z_A, D) = w_S·D_NN(z_S, D) + w_A·D_NN(z_A, D)," and ¶ [0091]: "Where w_S and w_A are scalar weights, which are optimized after training is complete." Wichern teaches that the weights so optimized are indexed to the frequency-resolved representation of the sensed signal at ¶ [0086]: "The dataset D={(X⁽ⁿ⁾, y⁽ⁿ⁾)}_{n=1}^N, where X∈R^(F×T) is a magnitude spectrogram with F frequencies and T time frames."
"and iteratively adjusting the values in the frequency weight vector and re-computing the anomaly scores" — Wichern, ¶ [0085]: "An additional approach for determining classifier weights of different dimensions could be after some known anomalies are observed, the anomalous sound detection system 102 may adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers," and ¶ [0085]: "By computing separate anomaly scores for each classifier at inference time, the anomalous sound detection system 102 can determine anomaly scores from embedding vectors that were trained by different classifiers… differently."
"to maximize a difference between the anomaly scores of the good and bad samples" — Wichern, ¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence." Wichern teaches that the separation so obtained is evaluated against a threshold dividing the two classes at ¶ [0078]: "the received audio signal 110 is determined to be anomalous when the anomaly score is above a pre-specified threshold value."
"and the values in the frequency weight vector are optimized" — Wichern, ¶ [0091]: "which are optimized after training is complete," and ¶ [0084]: "once the classifier is trained, the anomalous sound detection system 102 may not know a specific environment of an individual drilling machine at inference time."
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a neural network processes a spectrogram of a sensed machine signal and an anomaly score is derived therefrom. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to determine the values of the frequency weight vector of Nakanishi by the post-training weight optimization of Wichern, iteratively adjusting those values against the pre-classified good and bad samples of Jun, and thereafter to evaluate unclassified samples with the trained network and the optimized weights. The motivation to combine Jun, Nakanishi, and Wichern is as recited by Wichern (¶ [0085]: "adjust the weights of the different dimensions such that the observed anomalies will be detected in the future with high confidence, without requiring any re-training of the classifiers") and by Nakanishi (page 3: increasing the frequency weights "can be expected to increase the gradient on the Autoencoder's reconstruction error of high-frequency components"), such that one would optimize the frequency weight values rather than fix them, thereby increasing the separation between the anomaly scores of the good and bad samples that Jun already computes and compares (Jun, col. 5, lines 36–46).
As to Claim 19, Claim 19 recites limitations corresponding to those of Claim 8. Claim 19 is therefore rejected under the same rationale set forth above with respect to Claims 8 and 14.
As to Claim 20, Claim 20 recites limitations corresponding to those of Claim 9. Claim 20 is therefore rejected under the same rationale set forth above with respect to Claims 9 and 19.
As to Claim 22, Claim 22 recites limitations corresponding to those of Claim 11. Claim 22 is therefore rejected under the same rationale set forth above with respect to Claims 11 and 14.
Claims 10 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Jun, in view of Nakanishi, and further in view of Guan et al. (Guan), Non-Patent Literature, "Transformer-based Autoencoder with ID Constraint for Unsupervised Anomalous Sound Detection," arXiv:2310.08950v1 [cs.SD], published 13 October 2023, pages 1-26.
As to Claim 10, the limitations of Claim 1 from which Claim 10 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 10, Jun teaches:
"the anomaly score is computed by taking a norm of the weighted difference matrix" — Jun, col. 7, lines 23–31, Eq. (6): "(RE) Loss(x⁽ⁱ⁾, y⁽ⁱ⁾) = (1/n) Σ_{k=1}^{n} (x_k⁽ⁱ⁾ − y_k⁽ⁱ⁾)²," which is the squared Euclidean norm of the element-wise difference between the input and output matrices. Jun teaches that this quantity is the anomaly score at col. 5, lines 46–51: "RE is the difference between reconstructed features, which are the output of autoencoder (i.e. the reconstructed spectrogram image) and its corresponding input. After the thresholds are set for each joint, the model is used to detect the anomaly by comparing the RE with its corresponding threshold," and at col. 7, lines 46–50: "The anomaly detector may generate a reconstruction error (RE) (912). The reconstruction error may include a measure between the input features and output features of the autoencoder model."
Jun teaches that the difference matrix is organized into a plurality of time segments, the spectrogram being formed by concatenating successive per-segment vectors (Jun, col. 6, lines 40–47: "After normalization, successive PSDs are concatenated horizontally along the time axis… the number of n PSDs are combined into two-dimensional (2D) PSD sequences to construct a 2D input for a certain time interval for the autoencoder"). However, the combination of Jun and Nakanishi does not teach the following limitations of Claim 10.
In the same field of endeavor, Guan teaches:
"to obtain a vector having a largest frequency component for each time segment" — Guan, page 4: "Suppose X ∈ R^(N×M) is the log-Mel spectrogram of the sound signal, where N is the number of frames and M is the feature dimension of each frame of X." Guan teaches reducing the difference matrix by a norm to a vector having one element per time segment at page 5, Eq. (2): "The reconstruction error e_i for Y_i is [e_i = ‖Y_i − Ȳ_i‖_F], where Ȳ_i = D(E(Y_i)) is the corresponding output frames, and ‖·‖F denotes Frobenius norm. It results in a reconstruction error sequence e = {e_i}{i=1}^I for Y." Guan further teaches selection of the largest element as an operative reduction at page 9: "One solution is to use the maximal reconstruction error as the anomaly score i.e., max anomaly score A(e)_max = max(e), to highlight the anomalies of these audio clips."
"selecting a quantity of elements of the vector having a greatest value" — Guan, page 9: "let ê = {ê_1, …, ê_I} be sorted by descending order of e, the GWRP anomaly score can be calculated as [Eq. (11)] A(ê)gwrp = (1/Z(r)) Σ{i=1}^I r^(i−1) ê_i," and page 9: "It intends to assign larger weights to anomalous audio clips and lower weights to normal audio clips, to generate high anomaly scores for the anomalous events of short duration." Guan thereby teaches sorting the vector in descending order and preferentially selecting the elements having the greatest values.
"and calculating the anomaly score as a mean of the quantity of elements having the greatest value" — Guan, page 5, Eq. (3): "the mean reconstruction error of e can be used as the anomaly score." Guan teaches that the mean and the maximum are the two limiting cases of the same selection-and-averaging operation, and that the disclosed computation spans the range between them, at page 9: "GWRP is a generalization of max and mean, which can highlight the anomaly score by setting different weights to reconstruction error sequence e," and "When r = 0, A(ê)_gwrp degenerates to A(e)_max, and when r = 1, A(ê)_gwrp becomes A(e)_mean." Averaging a selected subset of the greatest-valued elements is accordingly an intermediate case of the operation Guan discloses.
Jun, Nakanishi, and Guan are analogous to the claimed invention as all are from the same field of endeavor of reconstruction-based machine anomaly detection in which an encoder/decoder neural network reconstructs a spectrogram and an anomaly score is computed from the input-versus-reconstruction difference. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to compute the anomaly score of Jun, as modified by the frequency weighting of Nakanishi, by reducing the weighted difference matrix to a per-time-segment vector and averaging its greatest-valued elements as taught by Guan. The motivation to combine Jun, Nakanishi, and Guan is as recited by Guan (page 9: the mean "usually underestimates the anomaly scores of anomalous audio clips when the anomalous events only appear for a short time," while "it is not robust to use the maximum value of e as the anomaly score of the whole audio clip, as it may overestimate the anomaly scores of some normal audio clips," so that a reduction intermediate between the two is employed "to improve the reliability of the calculated anomaly score"), such that one would average a selected subset of the largest per-segment values rather than take either the full mean or the single maximum of the difference matrix of Jun (Jun, col. 7, lines 23–31, Eq. (6)).
As to Claim 21, Claim 21 recites limitations corresponding to those of Claim 10. Claim 21 is therefore rejected under the same rationale set forth above with respect to Claims 10 and 14.
Claims 12, 23, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Jun, in view of Nakanishi, further in view of Wichern, and further in view of Tnani et al. (Tnani), Non-Patent Literature, "Smart Data Collection System for Brownfield CNC Milling Machines: A New Benchmark Dataset for Data-Driven Machine Monitoring," Procedia CIRP, Volume 107, Elsevier B.V., 2002, pages 131-136, cited in the IDS filed 25 April 2024.
As to Claim 12, the limitations of Claim 1 from which Claim 12 depends are rejected under the same rationale set forth above with respect to Claim 1.
Regarding the additional limitations of Claim 12, Jun teaches:
"the time-series data samples include one or more of … data … of a machine tool" — Jun, col. 1, lines 30–32: "both methods typically utilize various sensor signals such as torque, vibration, emission, and currents," and col. 2, lines 25–28: "For monitoring machine conditions, sound sensing may be more affordable solution for machine monitoring than force or vibration measurement since the total costs for sensors and signal conditioning devices are not required." Jun teaches that the monitored apparatus is a machine tool at col. 10, lines 34–38: "The components may include, for example, motors, servos, gears, or other parts of the machine that generate audible sound," and identifies the field of application at col. 1, lines 23–25: "In the new era of manufacturing with the Industry 4.0, effective machine monitoring systems are an important aspect of smart manufacturing."
Jun teaches that the monitored machine includes motors whose operating condition is sensed and evaluated by the autoencoder (Jun, col. 10, lines 34–38). However, the combination of Jun and Nakanishi does not teach "torque data from a spindle motor of a machine tool" or "speed data from the spindle motor."
In the same field of endeavor, Wichern teaches:
"torque data from a spindle motor of a machine tool" — Wichern, ¶ [0043]: the monitored state of operation includes "a characteristic of an input provided to the motor, for example… voltage powering the motor" and "a characteristic of an output of the motor, for example, torque generated by the motor." Wichern teaches that the motor is that of a machine tool at ¶ [0096]: "in case of a motor operating in an industrial automation environment… the attributes pertaining to the states of operation of the motor, like operating voltages, operating torque etc., are dependent on the type of industrial automation set-up," and at ¶ [0096]: "The sound source 108 may include a drilling machine, a grinding machine, a packaging machine, and the like."
"speed data from the spindle motor" — Wichern, ¶ [0083]: "Example parameters of operation could be the speed at which motor of the drilling machine rotates, the type of material that is being drilled, and the like." Wichern further teaches at ¶ [0083] that a classifier is trained "to predict a specific operating parameter from the sound signal produced by the drilling machines 602-606 (for example, predicting the speed of rotation)."
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a sensed machine signal is evaluated to determine an anomalous condition. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply the reconstruction-based anomaly detection of Jun, as modified by Nakanishi, to the spindle motor torque data and spindle motor speed data taught by Wichern. The motivation to combine Jun, Nakanishi, and Wichern is as recited by Jun (col. 1, lines 30–32: "both methods typically utilize various sensor signals such as torque, vibration, emission, and currents") and by Wichern (¶ [0096]: "the attributes pertaining to the states of operation of the motor, like operating voltages, operating torque etc., are dependent on the type of industrial automation set-up"), such that one would select among the motor operating parameters Jun itself enumerates as available for machine monitoring and which Wichern identifies as characterizing the operating state of the monitored motor.
The combination of Jun, Nakanishi, and Wichern, however, does not teach "data from one or more axial accelerometers mounted on the machine tool."
In the same field of endeavor, Tnani teaches "data from one or more axial accelerometers mounted on the machine tool." Tnani teaches at page 2, § 3.1: the data acquisition system "collects accelerometer data from Bosch CISS sensors mounted to the rear end of the spindle housing," and "the axes of the accelerometer are in alignment with the linear motion axis of the machine." Tnani teaches that the accelerometers are mounted on a machine tool at page 1, Abstract: "a new benchmark dataset for data-driven machine monitoring… acquired from three CNC milling machines," and at page 2: "the data was collected with a sampling rate of 2 kHz." Tnani teaches that the acquired data comprises a plurality of axial channels at page 4, FIG. 3: "the X, Y, Z acceleration axes of 4 sequential tool operations are illustrated." Tnani further teaches that the acquired accelerometer data is referenced to the spindle motor speed at page 2: "the frequencies of interest of the machining processes are low integer multiples (1..4) of the spindle speed."
Jun, Nakanishi, Wichern, and Tnani are analogous to the claimed invention as all are from the same field of endeavor of data-driven condition monitoring of manufacturing machinery from time-series sensor signals. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to supply the axial accelerometer data of Tnani as the time-series data samples evaluated by the combination of Jun, Nakanishi, and Wichern. The motivation to combine Jun, Nakanishi, Wichern, and Tnani is as recited by Tnani (page 2: "the frequencies of interest of the machining processes are low integer multiples (1..4) of the spindle speed," and page 1, Abstract: the dataset is provided "to enable the development and benchmarking of data-driven machine monitoring approaches"), such that one would apply the anomaly detection of Jun to the axial accelerometer signals Tnani identifies as carrying the frequencies of interest for machine tool monitoring.
As to Claim 23, Claim 23 recites limitations corresponding to those of Claim 12. Claim 23 is therefore rejected under the same rationale set forth above with respect to Claims 12 and 14.
As to Claim 24, the limitations of Claims 14 and 23 from which Claim 24 depends are rejected under the same rationale set forth above with respect to Claims 14 and 23.
Regarding the additional limitations of Claim 24, Jun teaches:
"concatenating frequency component data from all of the concurrent time-series data samples into a combined weighted difference matrix" — Jun, col. 6, lines 40–47: "After normalization, successive PSDs are concatenated horizontally along the time axis. Since training autoencoder with one-dimensional (1D) PSDs may lead to confusion between normal and abnormal conditions, the number of n PSDs are combined into two-dimensional (2D) PSD sequences to construct a 2D input for a certain time interval for the autoencoder. The concatenation of 1D PSDs results in 2D images with a size of n." Jun teaches the same operation at col. 5, lines 17–21: "The spectrogram, which is a sound signal magnitude versus frequency is a one-dimensional image (i.e. array), may be extended into a two-dimensional image by concatenating in order of time." Jun further teaches that the matrix so formed is the one from which the difference is computed at col. 5, lines 46–49: "RE is the difference between reconstructed features, which are the output of autoencoder (i.e. the reconstructed spectrogram image) and its corresponding input."
"and computing the anomaly score from the combined weighted difference matrix" — Jun, col. 7, lines 23–31, Eq. (6): "(RE) Loss(x⁽ⁱ⁾, y⁽ⁱ⁾) = (1/n) Σ_{k=1}^{n} (x_k⁽ⁱ⁾ − y_k⁽ⁱ⁾)²"; col. 5, lines 49–51: "After the thresholds are set for each joint, the model is used to detect the anomaly by comparing the RE with its corresponding threshold."
Jun teaches monitoring a plurality of concurrently sensed channels of a machine and establishing a separate evaluation for each, stating at col. 5, lines 44–46: "feasible thresholds are preliminarily set between 'normal' and 'abnormal' status in each axis," and at col. 7, lines 63–66: "In some examples, the system may include multiple autoencoders. Each auto-encoder may correspond to a particular component or group of component of a machine." However, the combination of Jun and Nakanishi does not teach "data from concurrent time-series data samples … are processed concurrently."
In the same field of endeavor, Wichern teaches "data from concurrent time-series data samples … are processed concurrently." Wichern teaches at ¶ [0077] that the system determines "a combined anomaly score 406, by concatenating the first embedding vector 202a and the second embedding vector 202b to generate a concatenated embedding vector 502," and at ¶ [0089] gives the corresponding expression: "For disentangled concatenation, concatenated embedding is used, z_c = [z_S^T, z_A^T]^T in (4)." Wichern teaches that a single anomaly score is computed from the combined representation at ¶ [0078]: the system is configured to "compare the generated concatenated embedding vector 502 with each of the embedding vectors," and "the received audio signal 110 is determined to be anomalous when the anomaly score is above a pre-specified threshold value." Wichern further teaches at ¶ [0080] that the combined score is produced by "combining them with a known combination technique, such summation, averaging, and the like," in which "the weight of the first anomaly score 504a… is less than a weight of the second anomaly score 504b." Wichern thereby teaches that a plurality of concurrently derived representations of a single sensed event are combined into one representation and evaluated together rather than separately.
Jun, Nakanishi, and Wichern are analogous to the claimed invention as all are from the same field of endeavor of machine anomaly detection in which a neural network processes a spectrogram of a sensed machine signal and an anomaly score is derived therefrom. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to process the concurrently acquired channels of Jun together, in the concatenated manner taught by Wichern, and to compute a single anomaly score from the resulting combined weighted difference matrix produced under Nakanishi. The motivation to combine Jun, Nakanishi, and Wichern is as recited by Wichern (¶ [0080]: "a weighted combination increases the accuracy and performance of the multi-head neural network 112 by putting less emphasis on redundant processing operations"), such that one would evaluate the concurrent channels jointly rather than independently, thereby avoiding the redundant per-channel processing Jun would otherwise require through "multiple autoencoders" (Jun, col. 7, lines 63–64).
The combination of Jun, Nakanishi, and Wichern, however, does not teach "data from concurrent time-series data samples containing acceleration data measured in three principle directions on the machine tool."
In the same field of endeavor, Tnani teaches "data from concurrent time-series data samples containing acceleration data measured in three principle directions on the machine tool." Tnani teaches at page 2, § 3.1 that the data acquisition system "collects accelerometer data from Bosch CISS sensors mounted to the rear end of the spindle housing," and that "the axes of the accelerometer are in alignment with the linear motion axis of the machine." Tnani teaches that the acceleration data is measured in three principle directions at page 4, FIG. 3: "the X, Y, Z acceleration axes of 4 sequential tool operations are illustrated." Tnani teaches that the three channels are acquired concurrently from the same machine tool at page 2: "the data was collected with a sampling rate of 2 kHz," and at page 1, Abstract, that the dataset was "acquired from three CNC milling machines" in "real production."
Jun, Nakanishi, Wichern, and Tnani are analogous to the claimed invention as all are from the same field of endeavor of data-driven condition monitoring of manufacturing machinery from time-series sensor signals. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to supply the three concurrent axial acceleration channels of Tnani as the concurrent time-series data samples processed by the combination of Jun, Nakanishi, and Wichern. The motivation to combine Jun, Nakanishi, Wichern, and Tnani is as recited by Tnani (page 2: "the frequencies of interest of the machining processes are low integer multiples (1..4) of the spindle speed," and page 1, Abstract: the dataset is provided "to enable the development and benchmarking of data-driven machine monitoring approaches"), such that one would apply the anomaly detection of Jun to the tri-axial accelerometer signals Tnani identifies as carrying the frequencies of interest for machine tool monitoring.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG VAN LE whose telephone number is (571)270-0164. The examiner can normally be reached 8 a.m. - 5 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNG VAN LE/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights