Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner notes the entry of the following papers:
Amended claims filed 8/4/2026.
Applicant arguments/remarks made in amendment filed 8/4/2026.
Claim 8 is cancelled. Claims 1-2, and 16 are amended. Claims 1-7, and 9-20 are presented for examination.
Response to Arguments
Applicant presents arguments/remarks. Each is addressed.
Applicant remarks “Applicant has accordingly incorporated dependent claim 8 into the respective independent claims to overcome this rejection.” (Remarks, page 8, paragraph 3, line 1.) Examiner notes the amendment. The rejections under 35 U.S.C. § 101 are withdrawn.
Applicant remarks “As amended, the claims recite determining a number of hidden layers for the neural network based on a second feature number (from previous claim 8).” (Remarks, page 8, paragraph 5, line 1.) However, Examiner notes that the limitations of original claim 8 are not the same as the limitations incorporated into newly amended claim 1. The original limitations of claim 8 recite
“The method of claim 2, wherein determining the second model based on the first feature number further comprises:
receiving a second feature number; and
determining a number of hidden layers for the neural network based on the second feature number.”
whereas the limitations incorporated into claim 1 recite
“wherein the second model is based on:
receiving, via a user interface, a second feature number; and
determining a number of hidden layers for the neural network based on the second feature number”
Therefore, since the amendment is more than just the limitations of claim 8, new grounds of rejection can be used to address the limitations. The argument that the prior art does not teach the amended claim is moot in view of new grounds of rejection necessitated by amendment.
Applicant argues “Accordingly, claims 1, 2, and 16 are not obvious.” (Remarks, page 9, paragraph 2, line 1.) The argument is moot in view of new grounds of rejection necessitated by amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-6, 9, 14, 16-17, and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over Yoon, et al (Time-series Generative Adversarial Networks, herein Yoon), Shu, et al (Dazzle: Using Optimized Generative Adversarial Networks to Address Security Data Class Imbalance Issue, herein Shu), Rouesnel, L. (US 11,593,705 B1, Feature Engineering Pipeline Generation for Machine Learning Using Decoupled Dataset Analysis and Interpretation, herein Rouesnel), and Tang, H. (Artificial Neural Network Based on Transfer Learning for Robot Kinematic Modelling, herein Tang).
Regarding claim 1,
Yoon teaches a system for minimizing development time in artificial intelligence models by [automating hyperparameter selection] based on dataset fittings of time-series data (Yoon, abstract line 7 “We propose a novel framework for generating realistic time-series data that combines the flexibility of the unsupervised paradigm with the control afforded by supervised training.” And, abstract, line 11 “Empirically, we evaluate the ability of our method to generate realistic samples using a variety of real and synthetic time-series datasets. Qualitatively and quantitatively, we find that the proposed framework consistently and significantly outperforms state-of-the-art benchmarks with respect to measures of similarity and predictive ability.” And, page 2, paragraph 1, line 1 “TimeGAN is a generative time-series model, trained adversarially and jointly via a learned embedding space with both supervised and unsupervised losses.” In other words, training with both supervised and unsupervised losses that outperforms state-of-the-art benchmarks is minimizing development time, TimeGAN is artificial intelligence model, and time-series data is time series data.) ,
[the system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations] comprising:
receiving a first dataset, wherein the first dataset comprises a first feature; processing the first dataset with a first model to generate a second dataset, wherein the second dataset comprises a second feature (Yoon, Figures 1 and 2, and, page 3, paragraph 4, line 1 “Consider the general data setting where each instance consists of two elements: static features (that
do not change over time, e.g. gender), and temporal features (that occur over time, e.g. vital signs).” And, page 3, paragraph 5, line 1 “Our goal is to use training data D to learn a density
PNG
media_image1.png
22
14
media_image1.png
Greyscale
(S; X1:T ) that best approximates p(S;X1:T ).This is a high-level objective, and—depending on the lengths, dimensionality, and distribution of the data—may be difficult to optimize in the standard GAN framework.” And page 3, paragraph 7, line 1 “TimeGAN consists of four network components: an embedding function, recovery function, sequence generator, and sequence discriminator.” And, page 4, paragraph 3, line 2 “Let ZS; ZX denote vector spaces over which known distributions are defined, and from which random vectors are drawn as input for generating into HS; HX .”
PNG
media_image2.png
434
918
media_image2.png
Greyscale
PNG
media_image3.png
526
922
media_image3.png
Greyscale
Examiner notes that it is unclear what the second model refers to. There could be two separate neural network models, or there could be one system where the first model is untrained, and after it is trained, it is referred to as the “determined” second model. The specification recites “The system may determine a second model (e.g., model 110) based on the first feature number, wherein the second model comprises the neural network.” (Specification, paragraph [0016], line 1.) Based on this, Examiner is interpreting the second model as meaning there are two separate models. In other words, dataset D is a first dataset, static features is a first feature, embedding/recovery model is a first model, s, x1:T is the first dataset, and
PNG
media_image4.png
26
64
media_image4.png
Greyscale
is the second dataset comprising a second feature.), wherein
the second feature is a synthetic feature, and wherein the second feature is not included in the first dataset (Yoon, see above mapping. In other words,
PNG
media_image4.png
26
64
media_image4.png
Greyscale
is the second feature is a synthetic feature that is not included in the first dataset.);
receiving a first feature number, wherein the first feature number indicates an embedding dimension for a neural network; determining a second model based on the first feature number (Yoon, page 3, paragraph 4, line 1 “Consider the general data setting where each instance consists of two elements: static features (that do not change over time, e.g. gender), and temporal features (that occur over time, e.g. vital signs). Let S be a vector space of static features, X of temporal features, and let S Є S, X Є X be random vectors that can be instantiated with specific values denoted s and x.” Examiner notes that the claim recites a “first feature” and a “second feature” as if there is only one feature identified in a dataset at a time. But then the claim recites a “feature number” that “indicates an embedding dimension” which implies that the dataset comprises multiple features for each item. Based on this, examiner is interpreting that there are multiple features for each data item. In other words, s is a first feature number that indicates an embedding dimension for a neural network, and the generate/discriminate model is a second model based on the feature number wherein the second model comprises a neural network.), wherein the second model comprises the neural network, and wherein the second model is based on:
[receiving, via a user interface, a second feature number; and determining a number of hidden layers for the neural network based on the second feature number];
fitting the second model on the second dataset; determining a number of neural network embedding features equal to the first feature number (Yoon, Figure 2. In other words, the generate/discriminate model is the second model,
PNG
media_image5.png
29
72
media_image5.png
Greyscale
is the second dataset, and s is a first feature number.);
tuning a first untuned hyperparameter to a specific value to generate a tuned model based on the number of neural network embedding features (Yoon, page 6, paragraph 2, line 1 “Figure 1(b) illustrates the mechanics of our approach at training. Let Θe, Θr, Θg, Θd respectively denote the parameters of the embedding, recovery, generator, and discriminator networks. The first two components are trained on both the reconstruction and supervised losses,
PNG
media_image6.png
42
532
media_image6.png
Greyscale
where
PNG
media_image7.png
24
58
media_image7.png
Greyscale
is a hyperparameter that balances the two losses. Importantly, λS is included such that the embedding process not only serves to reduce the dimensions of the adversarial learning space—it is actively conditioned to facilitate the generator in learning temporal relationships from the data. Next, the generator and discriminator networks are trained adversarially as follows,
PNG
media_image8.png
44
552
media_image8.png
Greyscale
where n is another hyperparameter that balances the two losses… In practice, we find that TimeGAN is not sensitive to λ and n; for all experiments in Section 5, we set λ = 1, and n = 10.” In other words, λ is first hyperparameter, and set λ to 1 is a specific value, balancing losses is generate a tuned model, and equation (11) depicts based on the number of features s.); and
[generating for display, on the user interface, a recommendation for using the tuned model for time-series forecasting.]
Thus far, Yoon does not explicitly teach automating hyperparameter selection.
Shu teaches automating hyperparameter selection (Shu, Algorithm 3, and, page 149, column 1, paragraph 3, line 1 “Since training a new GANs model can be difficult, this work checks if that process can be automated with hyperparameter optimization.”
PNG
media_image9.png
376
526
media_image9.png
Greyscale
In other words, from Algorithm 3, select hyperparameter set is automating hyperparameter selection.)
Both Yoon and Shu are directed to generative adversarial networks, among other things. Yoon teaches a system for minimizing development time in artificial intelligence models dataset comprises a second feature; but does not explicitly teach automating hyperparameter selection. Shu teaches automating hyperparameter selection.
In view of the teaching of Yoon, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Shu into Yoon. This would result in a system for minimizing development time in artificial intelligence models by automating hyperparameter selection based on dataset fittings of time-series data, the system comprising receiving a first dataset, wherein the first dataset comprises a first feature; processing the first dataset with a first model to generate a second dataset, wherein the second dataset comprises a second feature.
One of ordinary skill in the art would be motivated to do this in order to speed up training of adversarial neural networks. (Shu, page 149, column 1, paragraph 3, line 1 “Since training a new GANs model can be difficulty, this work checks if that process can be automated with hyperparameter optimization. Typically, a hyperparameter has a known effect on a model in the general sense, but it is not clear how to best set a hyperparameter for a given dataset. Hyperparameter optimization or hyperparameter tuning is a technique that explores a range of
hyperparameters and search for the optimal solution for a task.”)
Thus far the combination of Yoon and Shu does not explicitly teach the system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations.
Rouesnel teaches the system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations (Rouesnel, column 10, line 32 “The code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising instructions executable by one or more processors. The computer-readable storage medium is non-transitory.” In other words, one or more processors is one or more processors, computer-readable storage medium is non-transitory is non-transitory, computer readable mediums, and code is instructions.)
Rouesnel teaches generating for display, on the user interface, a recommendation for using the tuned model for time-series forecasting (Rouesnel, column 25, line 7 “In some embodiments, the model training system 126 and/or the model hosting system 214 provides the user devices 602 with one or more user interfaces, command-line interfaces (CLI), application programing interfaces (API), and/or other programmatic interfaces for submitting training requests, deployment request, and/or execution requests.” And, column 20, line 7 “Execution of the code 656 results in the generation of outputs ( e.g., predicted or "inferred" results), as described in greater detail below.” In other words, provides user devices is generating for display, user interface is on a user interface, execution request is a recommendation for using the tuned model, and predicted is forecast. Examiner notes that time-series is previously mapped to Yoon.)
Both Rouesnel and the combination of Yoon and Shu are directed to machine learning models, feature engineering, and prediction, among other things. The combination of Yoon and Shu teaches a system minimizing development time in artificial intelligence models by automating hyperparameter selection based on dataset fittings of time-series data, or generating for display, on a user interface, a recommendation for using the tuned model for time-series forecasting. Rouesnel teaches a system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations, and generating for display, on a user interface, a recommendation for using the tuned model for time-series forecasting.
In view of the teaching of the combination of Yoon and Shu, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Rouesnel into the combination of Yoon and Shu. This would result in a system minimizing development time in artificial intelligence models by automating hyperparameter selection based on dataset fittings of time-series data, the system comprising one or more processors; and one or more non-transitory, computer readable mediums comprising instructions that when executed by the one or more processors cause operations comprising: receiving a first dataset, wherein the first dataset comprises a first feature; processing the first dataset with a first model to generate a second dataset, wherein the second dataset comprises a second feature, wherein the second feature is a synthetic feature, and wherein the second feature is not included in the first dataset; receiving a first feature number, wherein the first feature number indicates an embedding dimension for a neural network; determining a second model based on the first feature number, wherein the second model comprises the neural network; fitting the second model on the second dataset; determining a number of neural network embedding features equal to the first feature number; tuning a first untuned hyperparameter to a specific value to generate a tuned model based on the number of neural network embedding features; and generating for display, on a user interface, a recommendation for using the tuned model for time-series forecasting.
One of ordinary skill in the art would be motivated to do this because implementing machine learning techniques can be difficult, and implementing the techniques on a computer system that has a user interface can make the implementation easier for the user, saving time and money. (Rouesnel, column 1, paragraph 1, line 13 “While the high-level view of machine learning sounds simple---e.g., provide training data to a computer, to allow the computer to automatically learn from the training data to generate a model that can make predictions for other data-implementing machine learning techniques in practice can be tremendously difficult.”)
Thus far, the combination of Yoon, Shu, and Rouesnel does not explicitly teach receiving, via a user interface, a second feature number; and determining a number of hidden layers for the neural network based on the second feature number.
Tang teaches receiving, via a user interface, a second feature number; and determining a number of hidden layers for the neural network based on the second feature number (Tang, Figure 3.5, and page 34, paragraph 1, line 4 “The ANN configuration can be customized by inputting the number of hidden layers and corresponding neural nodes.”
PNG
media_image10.png
604
1078
media_image10.png
Greyscale
In other words, from Figure 3.5, GUI is user interface, and inputting the number of hidden layers and corresponding neural nodes is receiving, via a user interface, a second feature number, and determining the number of hidden layers for the neural network based on the second feature number.)
Both Tang and the combination of Yoon, Shu, and Rouesnel are directed to machine learning, and in particular, configuring neural networks, among other things. The combination of Yoon, Shu, and Rouesnel teaches a system for minimizing development time in artificial intelligence models by automating hyperparameter selection based on dataset fittings of time- series data, the system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations comprising: receiving a first dataset, wherein the first dataset comprises a first feature; processing the first dataset with a first model to generate a second dataset, wherein the second dataset comprises a second feature, wherein the second feature is a synthetic feature, and wherein the second feature is not included in the first dataset; receiving a first feature number, wherein the first feature number indicates an embedding dimension for a neural network; determining a second model based on the first feature number, wherein the second model comprises the neural network, and determining a number of hidden layers for the neural network based on the second feature number.
In view of the teaching of the combination of Yoon, Shu, and Rouesnel, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Tang into the combination of Yoon, Shu, and Rouesnel. This would result in a system for minimizing development time in artificial intelligence models by automating hyperparameter selection based on dataset fittings of time- series data, the system comprising: one or more processors; and one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors cause operations comprising: receiving a first dataset, wherein the first dataset comprises a first feature; processing the first dataset with a first model to generate a second dataset, wherein the second dataset comprises a second feature, wherein the second feature is a synthetic feature, and wherein the second feature is not included in the first dataset; receiving a first feature number, wherein the first feature number indicates an embedding dimension for a neural network; determining a second model based on the first feature number, wherein the second model comprises the neural network, and wherein the second model is based on:
receiving, via a user interface, a second feature number; and determining a number of hidden layers for the neural network based on the second feature number; fitting the second model on the second dataset; determining a number of neural network embedding features equal to the first feature number; tuning a first untuned hyperparameter to a specific value to generate a tuned model based on the number of neural network embedding features; and generating for display, on the user interface, a recommendation for using the tuned model for time-series forecasting.
One of ordinary skill in the art would be motivated to do this because using a graphical user interface for configuration provides a fast and effective way to customize the neural network such as inputting the number of hidden layers, making it easier for the user thereby saving time and cost. (Tang, page 33, paragraph 1, line 1 “To customize the ANN configuration, a graphical user interface (GUI) is developed in this thesis, which incorporates data generation, training settings and plotting. The GUI consists of two parts: training part and evaluation part. The training GUI is shown in Figure 3.5.”)
Claim 2 is a method claim corresponding to a subset of the relevant limitations of system claim 1. Otherwise, they are not patentably distinct. The combination of Yoon, Shu, Rouesnel, and Tang teaches a method (Yoon, abstract, line 7 “We propose a novel framework for generating realistic time-series data that combines the flexibility of the unsupervised paradigm with the control afforded by supervised training.” In other words, novel framework is a method.) Therefore, claim 2 is rejected for the same reasons as claim 1.
Regarding claim 3,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein processing the first dataset with the first model to generate the second dataset further comprises:
retrieving the first feature from the first dataset ; and performing a first function on the first feature to generate the second feature (Yoon, Figure 1b, page 4, paragraph 2, line 8 “eS : S -> HS is an embedding network for static features…” and, page 4, paragraph 2, line 8 “In the opposite direction, the recovery function
PNG
media_image11.png
22
294
media_image11.png
Greyscale
takes static and temporal codes back to their feature representations
PNG
media_image12.png
24
208
media_image12.png
Greyscale
In other words, eS is applying the first function to the first feature from the first dataset, and
PNG
media_image13.png
18
16
media_image13.png
Greyscale
is the second feature.)
Regarding claim 5,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein processing the first dataset with the first model to generate the second dataset further comprises:
determining a first temporal pattern in the first dataset; and generating the second feature based on the first temporal pattern (Yoon, page 4, paragraph 2, line 1 “The embedding and recovery functions provide mappings between feature and latent space, allowing the adversarial network to learn the underlying temporal dynamics of the data via lower-dimensional representations. Let HS; HX denote the latent vector spaces corresponding to feature spaces S;X. Then the embedding function
PNG
media_image14.png
28
290
media_image14.png
Greyscale
takes static and temporal features to their latent codes hs, h1:T = e(s, X1:T). ” In other words, underlying temporal dynamics is a first temporal pattern of the first dataset, and the latent codes is generating the second feature based on the first temporal pattern.) .
Regarding claim 6,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein processing the first dataset with the first model to generate the second dataset further comprises:
determining a first relationship in the first dataset; and performing a first function based on the first relationship to generate the second feature (Yoon, Equation (11), and page 6, paragraph 2, line 5 “Importantly, LS is included such that the embedding process not only serves to reduce the dimensions of the adversarial learning space—it is actively conditioned to facilitate the generator in learning temporal relationships from the data.” And, page 6, paragraph 2, line 9 “That is, in addition to the unsupervised minimax game played over classification accuracy, the generator additionally minimizes the supervised loss. By combining the objectives in this manner, TimeGAN is simultaneously trained to encode (feature vectors), generate (latent representations), and iterate (across time).”
PNG
media_image15.png
38
554
media_image15.png
Greyscale
In other words, temporal relationships is a first relationship based on the first dataset, and minimizing the supervised loss is performing a first function based on the first relationship.) .
Regarding claim 9,
The combination of Yoon, Shu, and Rouesnel teaches the method of claim 2, wherein determining a second model based on the first feature number further comprises:
receiving a third feature number, wherein the third feature number indicates an additional transformation to be applied to the first dataset; and generating the second dataset based on the additional transformation (Rouesnel, column 2, line 61 “Accordingly, in many scenarios to deploy ML techniques data scientists are utilized to do feature engineering-e.g., take raw, dirty data fields (product descriptions, product IDs, numbers such as price or part numbers or phone numbers, 65 etc.) from datasets represented as spreadsheets or text files and apply various feature transformations to the data to make it something that is usable for the ML process. Feature engineering is a process of using domain knowledge of the data to create features ( or feature sets) that make ML algorithms work.” In other words, apply various feature transformations to the data is based on the data applying an additional transformation and generating the second dataset based on the additional transformation.).
It would be obvious to combine the teaching of Rouesnel into the combination of Yoon, Shu, Rouesnel, and Tang in order to be able to apply additional transformations to the data as needed.
Regarding claim 14,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein the second feature is based on:
comparing the first feature number to a threshold value; and determining a difference between the first feature number and the threshold value (Rouesnel, column 7, line 1 “For example, a processing strategy may be implemented as logic with various fallback positions. As one specific use case, a processing strategy 120 may seek to treat certain data values in special way, such as phone numbers. Thus, the processing strategy 120 may use a number of heuristics, such as determining whether a first threshold number (e.g., 80%) of values appear to be phone numbers (while a second threshold number (e.g., 95%) of the values appear to be numbers), the column may be treated as phone numbers, though if the first threshold is not met while the second threshold continues to be met, the column 10 may be treated as numeric.” In other words, 80% is a first feature number threshold value, and if the first threshold is not met, is comparing the difference between the first feature number and the threshold value.)
It would be obvious to combine the teaching of Rouesnel into the combination of Yoon, Shu, Rouesnel, and Tang to be able to compare a feature number with a threshold to know the status of the computation.
Claim 16 is a non-transitory, computer-readable medium claim that is a subset of the relevant limitations of system claim 1. Otherwise, they are not patentably distinct. The combination of Yoon, Shu, Rouesnel, and Tang teaches a non-transitory, computer-readable medium (Rouesnel, column 10, line 32 “The code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising instructions executable by one or more processors. The computer-readable storage medium is non-transitory.” In other words, computer-readable storage medium is non-transitory is non-transitory, computer-readable mediums.) Therefore, claim 16 is rejected for the same reasons as claim 1.
Claims 17 and 19 are non-transitory, computer-readable medium claims corresponding to method claims 3 and 5, respectively. Otherwise, they are not patentably distinct. Therefore, claims 17 and 19 are rejected for the same reasons as claims 3 and 5, respectively.
Claims 4 and 18 are rejected under 35 U.S.C. § as being unpatentable over Yoon, Shu, Rouesnel, Tang, and Shi, et al (A dual-LSTM framework combining change point detection and remaining useful life prediction, herein Shi).
Regarding claim 4,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein processing the first dataset with the first model to generate the second dataset further comprises:
Thus far, the combination of Yoon, Shu, Rouesnel, and Tang does not explicitly teach recursively performing a first function on the first feature; determining a stopping condition for the first function is met; ending a recursive performance of the first function based on the stopping condition being met; and determining the second feature based on a result of the recursive performance of the first function.
Shi teaches recursively performing a first function on the first feature; determining a stopping condition for the first function is met; ending a recursive performance of the first function based on the stopping condition being met; and determining the second feature based on a result of the recursive performance of the first function (Shi, page 2, column 2, paragraph 3, line 1 “RNN is a neural network with a feedback loop as shown in Fig. 1. In Fig. 1, xt and ht are the input and hidden state at time step t respectively. At the hidden layer, ht is an activation function of a linear combination of ht-1 and xt, as shown in Eq. (1). The activation function represented as σ here is usually a sigmoid function. At the output layer, the output yt is calculated as Eq. (2) shows. The Wh, Vh, and bh in Eq. (1) as well as the Wy, by in Eq. (2) are parameters of RNN that will be optimized during the training process.
PNG
media_image16.png
80
652
media_image16.png
Greyscale
The recursive function of Eq. (1) is exactly the feedback loop of the RNN in Fig. 1.” And, page 2, column 1, paragraph 2, line 9 “Then they continuously predicted one-cycle ahead HI (health index) by feeding the sequence of previous His into a Bi-directional LSTM. The RUL (remaining useful life) was finally determined as the length from the current time to the time when the predicted HI went beyond a predefined threshold.” In other words, HI is first feature, recursive function is recursively performing a first function on the first feature, and predefined threshold is ending a recursive performance of the first function based on a stopping condition.)
Both Shi and the combination of Yoon, Shu, Rouesnel, and Tang are directed to neural networks and time series data, among other things. The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, but does not explicitly teach recursively performing a first function on the first feature; determining a stopping condition for the first function is met; ending a recursive performance of the first function based on the stopping condition being met; and determining the second feature based on a result of the recursive performance of the first function. Shi teaches recursively performing a first function on the first feature; determining a stopping condition for the first function is met; ending a recursive performance of the first function based on the stopping condition being met; and determining the second feature based on a result of the recursive performance of the first function.
In view of the teaching of the combination of Yoon, Shu, Rouesnel, and Tang, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Shi into the combination of Yoon, Shu, Rouesnel, and Tang. This would result in the method of claim 2, and recursively performing a first function on the first feature; determining a stopping condition for the first function is met; ending a recursive performance of the first function based on the stopping condition being met; and determining the second feature based on a result of the recursive performance of the first function.
One of ordinary skill in the art would be motivated to do this because predicting remaining useful life (RUL) using time series data is an important task in condition based maintenance. Being able to take advantage of multiple sensors to effectively predict RUL would save time and money from unexpected failure. (Shi, abstract, line 1 “Remaining Useful Life (RUL) prediction is a key task of Condition-based Maintenance (CBM). The massive data collected from multiple sensors enables monitoring the complex systems in near real-time. However, such multiple sensors data environments pose a challenging task of combining the sensor data to infer the quality and RUL of the system.”)
Claim 18 is a non-transitory, computer-readable medium claim that corresponds to method claim 4. Otherwise, they are not patentably distinct. Therefore, claim 18 is rejected for the same reasons as claim 4.
Claims 7, and 10-13 are rejected under 35 U.S.C. § 103 as being unpatentable over Yoon, Shu, Rouesnel, Tang, and Mahmoud, R. (Machine Learning from Limited Time-Series Data, herein Mahmoud).
Regarding claim 7
The combination of teaches the method of claim 2, wherein processing the first dataset with the first model to generate the second dataset further comprises:
Thus far, the combination of Yoon, Shu, Rouesnel, and Tang does not explicitly teach generating a plurality of features based the first dataset; filtering the plurality of features based on respective importance metrics; and selecting the second feature for including in the second dataset based on filtering the plurality of features based on the respective importance metrics.
Mahmoud teaches generating a plurality of features based the first dataset; filtering the plurality of features based on respective importance metrics; and selecting the second feature for including in the second dataset based on filtering the plurality of features based on the respective importance metrics (Mahmoud, Figures 4.4, and 5.1, and, page 36, paragraph 2, line 1 “To allow for the extraction of dynamical informative features from the input sequences, we aggregate a CNN with a GRU network. The output of the last convolutional layer is of the shape (B × d × filterlast), with filterlast being the number of output channels of the last convolutional layer which is defined by the number of filters in a given layer.” And, page 45, paragraph 4, line 1 “We propose a lifelong learning multitask autoencoder (LOMA) method to address the problem of fine-tuning a network previously trained on a rich, source task domain.” And page 54, paragraph 4, line 1 “To test the performance of our proposed LOMA method with tasks of varying importance, we varied λ0 between 0.05 and 0.95 in Eq. 5.1 to increase the weight of the old task distillation output labels compared to new tasks being introduced into the network.”
PNG
media_image17.png
500
656
media_image17.png
Greyscale
PNG
media_image18.png
314
652
media_image18.png
Greyscale
In other words, features from input sequences is generating a plurality of features based on the first dataset, filters is filters, and varied importance by varying λ0 between 0.05 and 0.95 is based on respective importance metrics.)
Both Mahmoud and the combination of Yoon, Shu, Rouesnel, and Tang are directed to using neural networks for time series data, among other things. The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, but does not explicitly teach generating a plurality of features based the first dataset; filtering the plurality of features based on respective importance metrics; and selecting the second feature for including in the second dataset based on filtering the plurality of features based on the respective importance metrics. Mahmoud teaches generating a plurality of features based the first dataset; filtering the plurality of features based on respective importance metrics; and selecting the second feature for including in the second dataset based on filtering the plurality of features based on the respective importance metrics.
In view of the teaching of the combination of Yoon, Shu, Rouesnel, and Tang, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Mahmoud into the combination of Yoon, Shu, Rouesnel, and Tang. This would result in the method of claim 2 and generating a plurality of features based the first dataset; filtering the plurality of features based on respective importance metrics; and selecting the second feature for including in the second dataset based on filtering the plurality of features based on the respective importance metrics.
One of ordinary skill in the art would be motivated to do this because time-series data is becoming more prevalent because of sensors and IoT devices, but requires processing a vast amount of data. New methods for handling this data need to be developed to save money, labor and time. (Mahmoud, abstract, page 2, paragraph 1, line 1 “Time-series presents an important class of data in our everyday life and is becoming predominant with the abundance of sensors and IoT devices, which as a result has create opportunities for new machine learning (ML) applications. Unfortunately, the vast amounts of data collected are unlabeled and annotation of such data for ML becomes a challenge as it demands high monetary cost, labor, and time.”)
Regarding claim 10,
The combination of Yoon, Shu, Rouesnel, Tang, and Mahmoud teaches the method of claim 2, wherein determining the second model based on the first feature number further comprises:
receiving a fourth feature number, wherein the fourth feature number indicates a K value for a k-fold cross-validation process; and training the neural network using the k-fold cross-validation process (Mahmoud, page 38, paragraph 2, line 1 “The dataset was randomly split into an 80% training set, 10% validation set and 10% testing set. The validation set was used for hyperparameter tuning through k-fold cross validation (k=10). After hyperparameters are selected, we retrain the final model on the training and validation set (total of 90% of the original dataset) and present performance results on the 10% testing set.” In other words, k=10 is K value, k-fold cross-validation is k-fold cross-validation, and training is training the neural network using k-fold cross-validation.)
It would be obvious to combine Mahmoud into the combination of Yoon, Shu, Rouesnel and Mahmoud to improve the effectiveness of training by using k-fold cross-validation, thereby saving time and money.
Regarding claim 11,
The combination of Yoon, Shu, Rouesnel, Tang, and Mahmoud teaches the method of claim 2, wherein determining the second model based on the first feature number further comprises:
receiving a second feature number, wherein the second feature number indicates a number of neurons for a penultimate layer of the neural network; and further determining the first model based on the second feature number (Mahmoud, page 33, paragraph 2, line 1 “GMTL is defined by a given user’s input data point xt,i, a feature mapping function f, and WT a weight matrix parameterizing the MTL model. GMTL is presented as the concatenation of
g1, g2, . . . , gT task-dependent models, commonly referred to as single-task learning (STL) models, where each defines a mapping function between the model weights w1,w2, . . . ,wT and input features f(xt,i). Finally, the weight parameter matrix WT is defined as being the set of all STL models’ weight parameters w1,w2, . . . ,wT .” In other words, input data feature mapping is receiving a second feature number and mapping function to the model’s weight matrix indicates the number of neurons in a neural network, the weight parameter matrix identifies the penultimate number of neurons in the neural network, and the feature mapping function and the weight parameter matrix determine the first model.)
Regarding claim 12,
The combination of Yoon, Shu, Rouesnel, Tang, and Mahmoud teaches the method of claim 2, wherein determining the second model based on the first feature number further comprises:
determining a number of unique classes present in the first dataset; and determining a number of neurons in an output layer of the neural network based on the number of unique classes (Mahmoud, page 68, paragraph 2, line 1 “We present experimental results under two source-target settings. The first we refer to as standard setting, describing the scenario where both the source and target tasks share similar input domain and output label distributions. Under this setting, we consider ImageNet and CIFAR10 as the source and target tasks, respectively, where they both share a similar input domain of camera-captured, real images and a similar label distribution of object classes.” In other words, distribution of object classes is determining a number of unique classes, and from above mapping, the feature mapping function used for the classes combined with the weight parameter matrix determines the number of neurons in the output layer.) .
Regarding claim 13,
The combination of Yoon, Shu, Rouesnel, Tang, and Mahmoud teaches the method of claim 2, wherein the second feature is based on:
determining a first time period for the first dataset; determining a first statistical variation for the first dataset over the first time period; and determining the second feature based on the first statistical variation (Mahmoud, Figure 2.1, and page 18, paragraph 2, line 1 “Time-series present a unique data type that exhibits unique characteristics in comparison to standard tabular data. A time-series dataset is typically composed of sequences of data collected over consecutive and equal time intervals. The timeseries data is usually accompanied by a timestamp alongside the recorded samples to preserve the time element of the data. Some examples of time-series data are the weather temperature, stock market prices, and road traffic collected over a given time period.” And page 35, paragraph 4, line 1 “The input to the network can be composed of multiple sensor time-series sequences, where each sensor is labeled as Sk, k ∈ {1...K}. Each sensor generates readings over time and across multiple channels, thus each sensor Sk’s reading is of size {m(k) × n(k)}, where m(k) is the number of channels and n(k) is the number of samples collected over time for sensor Sk.” and, page 35, paragraph 4, line 1 “In order to effectively learn from time-series data, we need to extract general features that describe the underlying first-order statistics of the sequences as well as dynamic features that capture the time-evolving nature and statistics of the timeseries
sequences.”
PNG
media_image19.png
600
814
media_image19.png
Greyscale
In other words, time period is time period, first-order statistics of the sequences is determining a first statistical variance, and from Figure 2.1, categorical labels shows determining features for the data.)
It would be obvious to combine the teaching of Mahmoud into the combination of Yoon, Shu, Rouesnel, Tang, and Mahmoud to be able to determine a statistical variation in the dataset over a time period.
26. Claims 15 and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Yoon, Shu, Rouesnel, Tang, and Mishra, et al (Elevator Fault Detection Using Profile Extraction and Deep Autoencoder Feature Extraction for Acceleration and Magnetic Signals, herein Mishra).
27. Regarding claim 15,
The combination of Yoon, Shu, Rouesnel, and Tang teaches the method of claim 2, wherein the second feature is based on:
Thus far, the combination of Yoon, Shu, Rouesnel, and Tang does not explicitly teach generating a profile matrix for the first dataset; and determining the second feature based on the profile matrix.
Mishra teaches generating a profile matrix for the first dataset; and determining the second feature based on the profile matrix (Mishra, abstract, line 1 “In this paper, we propose a new algorithm for data extraction from time-series data, and furthermore automatic calculation of highly informative deep features to be used in fault detection. In data extraction, elevator start and stop events are extracted from sensor data including both acceleration and magnetic signals. In addition, a generic deep autoencoder model is also developed for automated feature extraction from the extracted profiles.” And, page 5, paragraph 1, steps 7-9
“
7. Add aligned data points from X(k) as rows into an n x m profile matrix, alternatively separate matrices according to direction of travel (min/max).
8. Set travel window k = k + 1 and repeat steps 5–7 until end of dataset.
9. Update reference profile P with the signal-averaged profile obtained from the column-wise mean of the new profile matrix.”
In other words, add aligned data points into an n x m profile matrix is generate a profile matrix, signal-averaged profile is determining the second feature based on the profile matrix.)
28. Claim 20 is a non-transitory, computer-readable medium claim corresponding to method claim 15. Otherwise, they are not patentably distinct. Therefore, claim 15 is rejected for the same reasons as claim 20.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BART RYLANDER whose telephone number is (571)272-8359. The examiner can normally be reached Monday - Thursday 8:00 to 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Bart I Rylander/Examiner, Art Unit 2124
Prosecution Insights