DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The rejections from the Office Action of 1/28/2026 are hereby withdrawn. New grounds for rejection are presented below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-11, 13-22, and 24-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20190378022 A1)[hereinafter “Wang”]; Mishra et al. (US 10733515 B1)[hereinafter “Mishra”]; Iranfar et al., ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals, ArXiv, 4.29.2021 [hereinafter “Iranfar”]; and Konrardy et al. (US 20210116256 A1)[hereinafter “Konrardy”].
Regarding Claims 1, 9, and 17, Wang discloses a non-transitory computer readable medium comprising instructions which, when executed by one or more hardware processors, causes performance of operations (and correspond computer system and method)[Paragraph [0024]] comprising:
partitioning a dataset comprising a plurality of data points into at least a first subset of data points and a second subset of data points, wherein the first subset of data points comprises a first data point and the second subset of data points comprises a second data point [Paragraph [0039] – “The system then divides the time-series sensor data into a training set and an estimation set (step 208).”Paragraph [0034] – “The standard procedure for training an MSET model is to divide the available data into a training dataset and an analysis dataset (also referred to as an “estimation dataset”). For example, one can arbitrarily select the first half of the available data for the training dataset, and the second half for the analysis dataset. Or, one can alternatively use the second half of the data for training and the first half for analysis. One can also divide the available data into an even number of windows, and then use alternating windows for training, and the other alternating windows for analysis.”];
wherein a first plurality of values, corresponding to the first data point, comprises (a) a first set of sensor-detected values [Paragraph [0039] – “FIG. 2 presents a flow chart illustrating the MVI process in accordance with the disclosed embodiments. During operation, the system first obtains the time-series sensor data (step 202), which was gathered from sensors in a monitored system during operation of the monitored system.”] and (b) a first set of interpolated values [Paragraph [0039] – “Next, the system identifies missing values in the time-series sensor data (step 204), and fills in the missing values through interpolation (step 206).”]; and
wherein a second plurality of values, corresponding to a second data point, comprises (a) a second set of sensor-detected values [Paragraph [0039] – “FIG. 2 presents a flow chart illustrating the MVI process in accordance with the disclosed embodiments. During operation, the system first obtains the time-series sensor data (step 202), which was gathered from sensors in a monitored system during operation of the monitored system.”] and (b) a second set of interpolated values [Paragraph [0039] – “Next, the system identifies missing values in the time-series sensor data (step 204), and fills in the missing values through interpolation (step 206).”];
updating interpolated values in the first subset of data points and the second subset of data points with imputed values at least by:
(a) training a first data correlation model based on the first subset of data points [Paragraph [0039] – “Next, the system trains an MSET model on the training set (step 210),…”];
(b) applying the first data correlation model to at least a portion of the second subset of data points to replace one or more interpolated values of the second subset of data points with one or more imputed values generated by the first data correlation model to generate a revised second subset of data points [Paragraph [0039] – “…and uses the trained MSET model to replace interpolated values in the estimation set with corresponding inferential estimates (step 212).”];
(c) training a second data correlation model based on the revised second subset of data points [Paragraph [0039] – “The system then determines whether there are any interpolated values in the training set (step 214). If so (YES at step 214), the system switches the training and estimation sets (step 216). The system then trains a new MSET model on the new training set (step 218),…”]; and
(d) applying the second data correlation model to at least a portion of the first subset of data points to replace one or more interpolated values of the first subset of data points with one or more imputed values generated by the second data correlation model to generate a revised first subset of data points [Paragraph [0039] – “…and uses the new MSET model to replace interpolated values in the new estimation set with corresponding inferential estimates (step 220).”];
concatenating the revised first subset of data points and the revised second subset of data points to generate a revised plurality of data points [Paragraph [0039] – “Finally, the system combines the training and estimation sets to produce preprocessed time-series sensor data, wherein missing values are filled in with imputed values (step 224).”].
Wang fails to disclose repeating operations (a)-(d) on the first revised plurality of data points to generate a second revised plurality of data points, at least by replacing a first set of imputed values, comprising the one or more imputed values generated by the first data correlation model and the second data correlation model, with a second set of imputed values.
However, Mishra discloses repeatedly training such ML models (each training constituting the training of an additional model) and applying the models on continually “flipped” training and target data sets until an exit criterion such as appropriate convergence has been realized [See Fig. 3 and Column 7 line 65 to Column 8 line 11 – “At block 316, the missing feature values can be filled into Dataset A, such as by using Equation 2 discussed above. From here, blocks 306 through 316 may be iterated a number of times, by swapping Dataset A and Dataset B as training data and scoring data with each iteration. The number of required iterations depends on the number of records, the complexity of the records, and the number of observations. However, through the iterative process, the model will tend to converge and the process may stop at convergence, or within a predetermined threshold of convergence. In other words, the residual value that indicates the difference between the actual value and the predicted value will converge at zero, or close to zero.”]. It would have been obvious to take such an approach in order to obtain suitably accurate interpolation results.
Wang also fails to disclose that the second set of imputed values comprise at least one outlier value.
However, Iranfar discloses evaluating a machine learning implemented imputation process through the detection of imputed outliers (and that imputed data will include outliers)[Abstract – “In this paper, we propose ReLearn, a robust machine learning framework for stress detection from biomarkers extracted from multimodal physiological signals. ReLearn effectively copes with missing data and outliers both at training and inference phases. ReLearn, composed of machine learning models for feature selection, outlier detection, data imputation, and classification, allows us to classify all samples, including those with missing values at inference.”See Fig. 1 and Page 4, first column (2nd, 3rd, and 4th paragraphs), particularly – “We propose to refit our inlier detection model after applying the imputer, since in a real scenario it is first required to impute the missing values, otherwise, the outlier detector fails to find the true outliers.”]. Iranfar serves as evidence that the imputation of Wang will inherently produce imputed outliers.
The combination would disclose identifying an application utilizing the dataset to perform one or more operations for prognosticating one or more values based on the dataset; and providing the second revised plurality of data points to the application to perform the one or more operations for prognosticating the one or more values, wherein the application prognosticates the one or more values for a target system using the second revised plurality of data points [Paragraph [0030] of Wang – “Next, the system uses a difference module 112 to perform a pairwise differencing operation between the actual signal values and the estimated signal values to produce residuals 114. The system then performs a “detection operation” on the residuals 114 by using SPRT module 116 to detect anomalies and possibly to generate an alarm 118.” See Fig. 1, this process is performed subsequent to the MVI processing 120 that is described in Paragraph [0039].] including the at least one outlier value included in the second set of imputed values [As taught to exist per Iranfar].
Wang fails to disclose that the operations further comprise at least one of: turning on or turning off a device in the target system based on the one or more values prognosticated by the application; adjusting a power level to the device based on the one or more values prognosticated by the application; adjusting a flow rate of a fluid to the device based on the one or more values prognosticated by the application; and adjusting a position of the device within a system based on the one or more values prognosticated by the application.
However, Konrardy discloses the use of vehicle sensor/telematics data in order to implement appropriate vehicle component control in such a manner [See Paragraphs [0049] and [0067]]. It would have been obvious to impute values into such vehicle sensor data streams and to perform appropriate vehicle control in the manner recited in order to ensure proper operation of the vehicle.
Wang fails to disclose generating, for the dataset, metadata identifying, for each data point in the plurality of data points, which values in the data point are sensor-detected values and which values are interpolated values; that applying the first data correlation model comprises referencing the metadata to identify the one or more interpolated values of the second subset of data points and to refrain from replacing any sensor-detected values of the second subset of data points; and that applying the second data correlation model comprises referencing the metadata to identify the one or more interpolated values of the first subset of data points and to refrain from replacing any sensor-detected values of the first subset of data points.
However, Wang retains measurement values through the disclosed process [Paragraph [0033] – “So we instead use high-accuracy measured signals to serve as “ground truth” values. We then randomly select values to “make missing” through deletion. However, we retain the original high-accuracy measured values that were removed for subsequent evaluation of the new MVI procedure against conventional interpolation.”]. It would have been obvious to track such values (i.e., through corresponding metadata) and to refrain from replacing them because doing so would have caused the machine learning to produce more accurate results through continued use of high accuracy ground truth values.
Regarding Claims 2, 10, and 18, Wang discloses that applying the first data correlation model to at least the portion of the second subset of data points to revise the one or more interpolated values of the second subset of data points comprises revising at least one of the second set of interpolated values comprised in the second data point [Paragraph [0039] – “…and uses the trained MSET model to replace interpolated values in the estimation set with corresponding inferential estimates (step 212).”].
Regarding Claims 3, 11, and 19, Wang fails to disclose training a third data correlation model based on the revised first subset of data points; and applying the third data correlation model to at least a portion of the revised second subset of data points to revise the one or more interpolated values of the second subset of data points to further revise the revised second subset of data points.
However, Mishra discloses repeatedly training such ML models (each training constituting the training of an additional model) and applying the models on continually “flipped” training and target data sets until an exit criterion such as appropriate convergence has been realized [See Fig. 3 and Column 7 line 65 to Column 8 line 11 – “At block 316, the missing feature values can be filled into Dataset A, such as by using Equation 2 discussed above. From here, blocks 306 through 316 may be iterated a number of times, by swapping Dataset A and Dataset B as training data and scoring data with each iteration. The number of required iterations depends on the number of records, the complexity of the records, and the number of observations. However, through the iterative process, the model will tend to converge and the process may stop at convergence, or within a predetermined threshold of convergence. In other words, the residual value that indicates the difference between the actual value and the predicted value will converge at zero, or close to zero.”]. It would have been obvious to take such an approach in order to obtain suitably accurate interpolation results.
Regarding Claims 5 and 13, Wang discloses applying the first data correlation model to at least the portion of the second subset of data points includes replacing the one or more interpolated values with the one or more imputed values [Paragraph [0039] – “…and uses the trained MSET model to replace interpolated values in the estimation set with corresponding inferential estimates (step 212).”] and refraining from replacing any sensor-detected values of the second subset of data points with the one or more imputed values [Paragraph [0039], replacing interpolated values and not sensor-detected values].
Regarding Claims 6 and 14, Wang discloses that the dataset comprises a plurality of parallel time-series sensor data signals [Paragraph [0028] – “As illustrated in FIG. 1, prognostic-surveillance system 100 operates on a set of time-series signals 104 obtained from sensors in a system under surveillance 102. Note that system under surveillance 102 can generally include any type of machinery or facility, which includes sensors and generates time-series signals. Moreover, time-series signals 104 can originate from any type of sensor, which can be located in a component in system under surveillance 102, including: a voltage sensor; a current sensor; a pressure sensor; a rotational speed sensor; and a vibration sensor.”Paragraph [0039] – “FIG. 2 presents a flow chart illustrating the MVI process in accordance with the disclosed embodiments. During operation, the system first obtains the time-series sensor data (step 202), which was gathered from sensors in a monitored system during operation of the monitored system.”], wherein partitioning the dataset into at least the first subset of data points and the second subset of data points comprises dividing the dataset into the first subset of data points generated prior to a particular time and the second subset of data points generated at, or after, the particular time [Paragraph [0034] – “The standard procedure for training an MSET model is to divide the available data into a training dataset and an analysis dataset (also referred to as an “estimation dataset”). For example, one can arbitrarily select the first half of the available data for the training dataset, and the second half for the analysis dataset. Or, one can alternatively use the second half of the data for training and the first half for analysis. One can also divide the available data into an even number of windows, and then use alternating windows for training, and the other alternating windows for analysis.”].
Regarding Claims 7 and 15, Wang discloses that the first data correlation model [Paragraph [0039] – “Next, the system trains an MSET model on the training set (step 210),…”] and the second data correlation model are multivariate state estimation technique (MSET) models [Paragraph [0039] – “The system then determines whether there are any interpolated values in the training set (step 214). If so (YES at step 214), the system switches the training and estimation sets (step 216). The system then trains a new MSET model on the new training set (step 218),…”].
Regarding Claims 8 and 16, Wang discloses that the first set of interpolated values and the second set of interpolated values are generated by up-sampling [Paragraph [0039] – “Next, the system identifies missing values in the time-series sensor data (step 204), and fills in the missing values through interpolation (step 206).”] data streams from one or more sensors [Paragraph [0028] – “As illustrated in FIG. 1, prognostic-surveillance system 100 operates on a set of time-series signals 104 obtained from sensors in a system under surveillance 102. Note that system under surveillance 102 can generally include any type of machinery or facility, which includes sensors and generates time-series signals. Moreover, time-series signals 104 can originate from any type of sensor, which can be located in a component in system under surveillance 102, including: a voltage sensor; a current sensor; a pressure sensor; a rotational speed sensor; and a vibration sensor.”Paragraph [0039] – “FIG. 2 presents a flow chart illustrating the MVI process in accordance with the disclosed embodiments. During operation, the system first obtains the time-series sensor data (step 202), which was gathered from sensors in a monitored system during operation of the monitored system.”].
Regarding Claim 21, Wang fails to disclose that the application is a telemetry-assist application for controlling telemetry of a vehicle, wherein the dataset is generated from a real-time data stream of telemetry data generated by a set of sensors in the vehicle, wherein the operations further comprise transmitting the first revised plurality of data points to the vehicle in real-time, and generating telemetry prediction data from the first revised plurality of data points to control the telemetry of the vehicle, and wherein controlling the telemetry of the vehicle comprises at least one of: turning on or turning off a device in the vehicle; adjusting a power level to the device in the vehicle; adjusting a flow rate of a fluid to the device in the vehicle; and adjusting a position of a device within the vehicle.
However, Konrardy discloses the use of vehicle sensor/telematics data in order to implement appropriate vehicle component control in such a manner [See Paragraphs [0049] and [0067]]. It would have been obvious to impute values into such vehicle sensor data streams and to perform appropriate vehicle control in the manner recited in order to ensure proper operation of the vehicle.
Regarding Claim 22, the combination would disclose that the operations further comprise: concatenating the revised first subset of data points and the revised second subset of data points to generate a second revised set of data points [Paragraph [0039] of Wang – “Finally, the system combines the training and estimation sets to produce preprocessed time-series sensor data, wherein missing values are filled in with imputed values (step 224).”]; and partitioning the second revised set of data points into a third subset of data points and a fourth subset of data points, wherein the third subset of data points is different from the first subset of data points and the second subset of data points [Paragraph [0039] of Wang – “The system then divides the time-series sensor data into a training set and an estimation set (step 208).”Paragraph [0034] of Wang – “The standard procedure for training an MSET model is to divide the available data into a training dataset and an analysis dataset (also referred to as an “estimation dataset”). For example, one can arbitrarily select the first half of the available data for the training dataset, and the second half for the analysis dataset. Or, one can alternatively use the second half of the data for training and the first half for analysis. One can also divide the available data into an even number of windows, and then use alternating windows for training, and the other alternating windows for analysis.”].
Regarding Claim 24, Wang discloses that the application is a fault-detection application, wherein the dataset comprises a set of parallel data streams comprising sensor data of a system, and wherein the fault-detection application predicts a fault in the system based on the first revised plurality of data points [Paragraph [0030] – “Next, the system uses a difference module 112 to perform a pairwise differencing operation between the actual signal values and the estimated signal values to produce residuals 114. The system then performs a “detection operation” on the residuals 114 by using SPRT module 116 to detect anomalies and possibly to generate an alarm 118.” See Fig. 1, this process is performed subsequent to the MVI processing 120 that is described in Paragraph [0039].].
Regarding Claim 25, Wang discloses that the dataset is generated from a real-time data stream of sensor data, and wherein the operations further comprise streaming the first revised plurality of data points to the application in real-time [Paragraph [0030] – “Next, the system uses a difference module 112 to perform a pairwise differencing operation between the actual signal values and the estimated signal values to produce residuals 114. The system then performs a “detection operation” on the residuals 114 by using SPRT module 116 to detect anomalies and possibly to generate an alarm 118.” See Fig. 1, this process is performed subsequent to the MVI processing 120 that is described in Paragraph [0039].].
Claim(s) 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20190378022 A1)[hereinafter “Wang”]; Mishra et al. (US 10733515 B1)[hereinafter “Mishra”]; Iranfar et al., ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals, ArXiv, 4.29.2021 [hereinafter “Iranfar”]; Konrardy et al. (US 20210116256 A1)[hereinafter “Konrardy”]; and Avergun et al. (US 20070028070 A1)[hereinafter “Avergun”].
Regarding Claim 26, Wang fails to disclose that the dataset is generated from a real-time stream of sensor data comprising a plurality of time-interval segments; wherein the operations further comprise forming the dataset by combining a most-recently received time-interval segment of the real-time stream with a predetermined number of previously received time-interval segments of the real-time stream; and repeating the forming operation upon receipt of each subsequent time-interval segment by adding the subsequent time-interval segment to the dataset and removing an oldest time-interval segment from the dataset.
However, Avergun discloses a memory allocation scheme where new data values are taken in items/blocks and are stored in memory and oldest items/blocks are discarded when a predetermined memory-amount threshold is reached in order to allow for the storage of more incoming data [See Fig. 4 and Paragraphs [0040]-[0043]]. It would have been obvious to take such an approach in order to reduce the size of memory needed while still allowing for the storage of incoming data.
Response to Arguments
Applicant argues:
PNG
media_image1.png
484
774
media_image1.png
Greyscale
PNG
media_image2.png
75
784
media_image2.png
Greyscale
PNG
media_image3.png
578
784
media_image3.png
Greyscale
PNG
media_image4.png
441
779
media_image4.png
Greyscale
PNG
media_image5.png
79
782
media_image5.png
Greyscale
Examiner’s Response:
The Examiner agrees. However, Wang retains measurement values through the disclosed process [Paragraph [0033] – “So we instead use high-accuracy measured signals to serve as “ground truth” values. We then randomly select values to “make missing” through deletion. However, we retain the original high-accuracy measured values that were removed for subsequent evaluation of the new MVI procedure against conventional interpolation.”]. It would have been obvious to track such values (i.e., through corresponding metadata) and to refrain from replacing them because doing so would have caused the machine learning to produce more accurate results through continued use of high accuracy ground truth values. Improper hindsight is not present because Wang explicitly discusses such values as being of “high-accuracy.”
Applicant argues:
PNG
media_image6.png
711
784
media_image6.png
Greyscale
Examiner’s Response:
The Examiner respectfully disagrees. Iranfar discloses evaluating a machine learning implemented imputation process through the detection of imputed outliers (and that imputed data will include outliers)[Abstract – “In this paper, we propose ReLearn, a robust machine learning framework for stress detection from biomarkers extracted from multimodal physiological signals. ReLearn effectively copes with missing data and outliers both at training and inference phases. ReLearn, composed of machine learning models for feature selection, outlier detection, data imputation, and classification, allows us to classify all samples, including those with missing values at inference.”See Fig. 1 and Page 4, first column (2nd, 3rd, and 4th paragraphs), particularly – “We propose to refit our inlier detection model after applying the imputer, since in a real scenario it is first required to impute the missing values, otherwise, the outlier detector fails to find the true outliers.”]. Iranfar serves as evidence that the imputation of Wang will inherently produce imputed outliers.
Iranfar does not teach away from the recitation that the imputed data includes outliers because Iranfar is only relied on as evidence that the imputed outliers are present.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Audigier et al., Multiple imputation for continuous variables, arXiv, 2015
Kim et al., Reuse of imputed data, BioMed Central, 2004
Liu et al., Missing Value Imputation for Industrial IoT, IEEE, 2020
Zhou et al., Recover Missing Sensor Data with Iterative Imputing Network, AAAI, 2018
US 20200081817 A1 – REPLACING STAIR-STEPPED VALUES IN TIME-SERIES SENSOR SIGNALS WITH INFERENTIAL VALUES TO FACILITATE PROGNOSTIC-SURVEILLANCE OPERATIONS
US 20190243799 A1 – SYNTHESIZING HIGH-FIDELITY TIME-SERIES SENSOR SIGNALS TO FACILITATE MACHINE-LEARNING INNOVATIONS
US 20140279707 A1 – SYSTEM AND METHOD FOR VEHICLE DATA ANALYSIS
US 20030107548 A1 – System And Method For Executing Diagnosis Of Vehicle Performance
US 12149395 B1 – Coefficient Generator
US 20200201864 A1 – UPDATING A TOPLIST FOR A CONTINUOUS DATA STREAM
US 20120054454 A1 – SAMPLING FREQUENCY CONVERTER
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE ROBERT QUIGLEY whose telephone number is (313)446-4879. The examiner can normally be reached 9AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Arleen Vazquez can be reached at (571) 272-2619. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE R QUIGLEY/Primary Examiner, Art Unit 2857
Prosecution Insights