DETAILED ACTION
This communication is in response to the Application No. 18/032,867 filed on April 20, 2023
in which Claims 14 - 26 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 14-26 are rejected under 35 U.S.C. 101 because these claimed inventions are
directed to an abstract idea without significantly more.
Regarding Claim 14:
Step 1: Claim 14 is a method type claim. Therefore, Claims 14-26 fall within one of the four statutory
categories (i.e., process, machine, manufacture, or composition of matter).
2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance
of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. If a claim limitation, under its broadest reasonable
interpretation, covers performance of the limitation by mathematical calculation but for the recitation
of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract
ideas.
obtaining metadata characterizing data to be processed and performance metrics for evaluating the performance of the data processing and inference pipeline (mental process - obtaining metadata characterizing data to be processed and performance metrics may be performed manually by a user)
fuzzifying the obtained metadata to derive fuzzy metadata (mathematical concept – fuzzifying the obtained metadata to derive fuzzy metadata involves transforming numerical or symbolic data into fuzzy representations using mathematical membership functions. For example, as described in the specification (Par. [0020]), fuzzifying may be performed using functions such as Gaussian membership functions that map input values to degrees of membership)
constructing one or more candidate data processing and inference pipelines fulfilling the pre-defined problem specification, wherein a candidate data processing and inference pipeline comprises a set of interconnected data processing and inference blocks (mental process - constructing one or more candidate data processing and inference pipelines fulfilling the pre-defined problem specification may be performed mentally by a user observing/analyzing the pre-defined problem specification and accordingly using judgment/evaluation to construct one or more candidate data processing and inference pipelines based on said analysis)
selecting, from the one or more candidate data processing and inference pipelines, a data processing and inference pipeline characterized by an optimum overall performance score (mental process – selecting a data processing and inference pipeline characterized by an optimum overall performance score may be performed mentally by a user observing/analyzing the overall performance score and accordingly using judgement/evaluation to select a data processing and inference pipeline based on said analysis)
obtaining, from a repository, information characterizing respective data processing and inference blocks in terms of input and output data constraints and sets of rules for evaluating the performance of a respective data processing and interference block in terms of the obtained performance metrics by characterizing the respective identified data processing and inference blocks in terms of their respective input and output data constraints and the obtained performance metrics (mental process - obtaining, from a repository, information characterizing respective data processing and inference blocks may be performed manually by a user)
identifying, based on the obtained information, data processing and inference blocks characterized with corresponding input and output data constraints (mental process – identifying, based on the obtained information, data processing and inference blocks may be performed mentally by a user observing/analyzing the obtained information and accordingly using judgment/evaluation to identify data processing and inference blocks based on said analysis)
calculating , by means of fuzzy inference, one or more performance scores for the respective identified data processing and inference blocks by applying the sets of rules to the obtained metadata, the one or more performance scores thereby indicating the performance of the respective identified data processing and inference blocks in terms of the obtained metadata and performance metrics and its input and output data constraints (mathematical concept - calculating performance scores using fuzzy inference may be performed as a mathematical evaluation because it involves applying membership functions, rule sets, and numerical or symbolic operations to input data to generate output values representing performance measures)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Regarding Claim 15:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 15 depends on.
wherein the obtaining metadata comprises obtaining one or more qualitative and/or quantitative metrics characterizing the data to be processed from a user (mental process – obtaining one or more qualitative and/or quantitative metrics may be performed manually by a user)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 16:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 16 depends on.
wherein the obtaining metadata comprises obtaining data to be processed and deriving therefrom one or more qualitative and/or quantitative metadata characterizing the data to be processed (mental process – obtaining data to be processed and deriving therefrom one or more qualitative and/or quantitative metadata may be performed manually by a user)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 17:
Step 2A Prong 1: See the rejection of Claim 16 above, which Claim 17 depends on.
wherein the deriving comprises processing the received data by means of data analysers (mental process – processing the received data by means of data analysers may be performed mentally by a user observing/analyzing the received data and accordingly using judgment/evaluation to process the received data based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 18:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 18 depends on.
wherein the fuzzifying comprises applying sets of membership functions to the respective metadata (mathematical concept – applying sets of membership functions to the respective metadata involves evaluating mathematical functions on input values to compute degrees of membership, which is a numerical calculation)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 19:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 19 depends on.
Step 2A Prong 2 & Step 2B:
wherein the sets of rules for evaluating the performance of a respective data processing and interference block are derived based on its input and output constraints and the obtained performance metrics (field of use - limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that the sets of rules for evaluating the performance of a respective data processing and interference block are derived based on its input and output constraints and the obtained performance metrics does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, this additional element does not integrate the abstract idea into practical application because it does not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of Claim 14. The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Regarding Claim 20:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 20 depends on.
wherein the calculating further comprises aggregating the derived respective performance scores of the identified data processing and inference blocks, and, combining the aggregated performance scores, thereby deriving an overall performance score for a respective candidate data processing and inference pipeline, wherein the combining comprises weighting the aggregated performance scores in accordance with weight factors associated with respective performance metrics (mental process – aggregating the derived respective performance scores, combining the aggregated performance scores, and weighting the aggregated performance scores may be performed mentally by a user observing/analyzing the performance scores and accordingly using judgment/evaluation to aggregate, combine, and weight the performance scores based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 21:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 21 depends on.
Step 2A Prong 2 & Step 2B:
wherein the constructing is performed in an incremental manner (field of use - limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that constructing is performed in an incremental manner does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, this additional element does not integrate the abstract idea into practical application because it does not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of Claim 14. The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Regarding Claim 22:
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 22 depends on.
evaluating the performance of the selected data processing and inference pipeline in terms of the obtained performance metrics and based on the data to be processed, wherein the evaluating comprising at least partly deploying the selected data processing and inference pipeline and determining an actual overall performance score for the at least partly deployed data processing and inference pipeline (mental process – evaluating the performance of the selected data processing and inference pipeline and determining an actual overall performance score may be performed mentally by a user observing/analyzing the performance of the selected data processing and inference pipeline and accordingly using judgment/evaluation to evaluate the performance of the selected data processing and inference pipeline and determine an actual overall performance score based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 23:
Step 2A Prong 1: See the rejection of Claim 22 above, which Claim 23 depends on.
wherein the evaluating further comprises determining values of respective parameters associated with the respective data processing and inference blocks of the at least partly deployed pipeline (mental process – determining values of respective parameters associated with the respective data processing and inference blocks may be performed mentally by a user observing/analyzing the values of respective parameters and accordingly using judgment/evaluation to determine values of respective parameters associated with the respective data processing and inference blocks based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 24:
Step 1: Claim 24 is a computer program product type claim. Therefore, it falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 24 depends on.
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 25:
Step 1: Claim 25 is a computer readable storage medium type claim. In the specification (Par. [0091]), “computer readable storage medium” is interpreted as limited to physical storage media and does not encompass transitory propagating signals. Therefore, the claim falls within the statutory manufacture category (i.e., process, machine, manufacture, or composition of matter).
Step 2A Prong 1: See the rejection of Claim 24 above, which Claim 25 depends on.
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Regarding Claim 26:
Step 1: Claim 25 is a system type claim. Therefore, the claim falls within the statutory manufacture category (i.e., process, machine, manufacture, or composition of matter).
Step 2A Prong 1: See the rejection of Claim 14 above, which Claim 26 depends on.
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, there are no additional elements that integrate the abstract idea into practical application. The claim does not include additional elements considered
individually and in combination that are sufficient to amount to significantly more than the judicial
exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 14 - 26 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al.
(hereafter Chen) (US 20220036246), in view of Panfilov et al. (hereinafter Panfilov) (US 20050119986).
Regarding Claim 14, Chen teaches a computer-implemented method (Chen, Par. [0004], “According to an embodiment, a computer-implemented method is provided”, thus a computer-implemented method is disclosed) for deriving a data processing and inference pipeline (Chen, Par. [0033], “As used herein the terms “machine learning pipeline” and/or “machine learning pipelines” can refer to an end-to-end development process for one or more machine learning models. For example, a machine learning pipeline can include one or more steps of: data collection, data cleansing, feature engineering, model selection, parameter optimization, ensemble, model validation, model deployment, runtime monitoring, and/or model improvement”, thus a data processing and inference pipeline is disclosed) based on a pre-defined problem specification (Chen, Par. [0024], “In some examples, the one or more machine learning pipelines can be associated with one or more defined use cases, wherein data correlations specific to a context of the use case can be known and utilized expedite the machine learning”, thus a pre-defined problem specification is disclosed), the computer-implemented method comprising:
obtaining metadata characterizing data to be processed and performance metrics for evaluating the performance of the data processing and inference pipeline (Chen, Par. [0046], “For example, previous executions of the candidate machine learning pipelines can result in one or more observations regarding the machine learning pipelines' performance. These observations can be captured as meta-data associated with the machine learning pipelines. The meta-data can regard how well the machine learning pipeline accomplished a given machine learning task with respect to one or more evaluation metrics (e.g., accuracy of predictions and/or classifications). In various embodiments, the meta-data can characterize features of the machine learning pipeline, datasets analyzed by the machine learning pipeline, and/or interdependencies between the machine learning pipelines and the datasets.”, & Par. [0047], “For instance, the meta-data can describe one or more features of the previously analyzed datasets, such as, but not limited to: a domain of the dataset, number of datapoints in the dataset, number of attributes, percentage of missing values, the scope of the algorithms to be considered, the candidate algorithm's hyperparameter values, the selection of variables, a combination thereof, and/or the like. In another instance, the meta-data can describe one or more interdependencies determined by the machine learning pipelines from the historical datasets, such as, but not limited to: correlations between various attributes and target values, Pearson correlations between various attributes and aggregations (e.g., average and/or standard deviations), cosine similarity, neural-network based comparison (“CNN”), a combination thereof, and/or the like. In a further instance, the meta-data can characterize a topology of the one or more machine learning pipelines (e.g., via a string of identifiers sequenced based on the flow and/or stages of the machine learning pipelines). For example, machine learning pipeline topology can comprise a collection of algorithms and/or an execution sequence of the algorithms, wherein the topology can be represented by a sequence of words. Each of the algorithms can be represented by an identifier (e.g., a hash), wherein the order of the identifiers can represent the execution sequence of the algorithms. Moreover, the meta-data can describe the type of machine learning task performed by the machine learning pipeline and/or the type of machine learning models generated and/or optimized by the machine learning pipeline (e.g., predictive machine learning models for classification, regression, deep learning, a combination thereof, and/or the like). Thereby the meta-data can describe the structure of the machine learning pipelines, a task of the machine learning pipelines, features of the historic datasets previously analyzed by the machine learning pipelines, and/or performance observations (e.g., determined data correlations) achieved by the machine learning pipelines.” & Par. [0049], “The learner component 112 can compare the meta-data of the machine learning pipelines with the characteristics of the time series data to predict how the respective machine learning pipelines will perform on the time series data. By employing the meta-data in the comparison, the learner component 112 can leverage insights previously developed by the machine learning pipelines to predict one or more performance metrics with regards to the time series data. Thereby, the learner component 112 can identify machine learning pipelines suitable for execution with the time series data without running the machine learning pipeline candidates on the full time series data, thus obtaining metadata characterizing data to be processed and performance metrics for evaluating the performance of the data processing and inference pipeline is disclosed, because Chen teaches capturing performance observations from prior pipeline executions as metadata, describing dataset characteristics and pipeline structure using that metadata, and using the metadata to predict performance metrics of pipelines on data, which corresponds to obtaining metadata characterizing data to be processed and performance metrics for evaluating pipeline performance.)
constructing one or more candidate data processing and inference pipelines fulfilling the pre-defined problem specification, wherein a candidate data processing and inference pipeline comprises a set of interconnected data processing and inference blocks (Chen, Par. [0033], “For example, a machine learning pipeline can include one or more steps of: data collection, data cleansing, feature engineering, model selection, parameter optimization, ensemble, model validation, model deployment, runtime monitoring, and/or model improvement.”, & Par. [0045], “In one or more embodiments, a machine learning pipeline candidate pool can be the entirety of the pipeline library 120 , a subset of the pipeline library 120 (e.g., defined by one or more runtime or pipeline thresholds defined by the one or more input devices 106 ), and/or a defined set of machine learning pipelines (e.g., wherein the one or more input devices 106 can be employed to target specific machine learning pipelines from the pipeline library 120 for consideration by the learner component 112 ). Also, in one or more embodiments the number of machine learning pipelines identified and/or ranked by the learner component 112 can be defined by the one or more input devices 106 .”, & Par. [0047], “In a further instance, the meta-data can characterize a topology of the one or more machine learning pipelines (e.g., via a string of identifiers sequenced based on the flow and/or stages of the machine learning pipelines). For example, machine learning pipeline topology can comprise a collection of algorithms and/or an execution sequence of the algorithms, wherein the topology can be represented by a sequence of words. Each of the algorithms can be represented by an identifier (e.g., a hash), wherein the order of the identifiers can represent the execution sequence of the algorithms.”, thus constructing one or more candidate data processing and inference pipelines fulfilling the pre-defined problem specification, wherein each pipeline comprises a set of interconnected data processing and inference blocks, is disclosed, because Chen teaches that a machine learning pipeline includes multiple processing steps such as data collection, cleansing, feature engineering, model selection, validation, and deployment. Chen also teaches selecting candidate pipelines from a pipeline library for consideration, and discloses that pipeline topology comprises a collection of algorithms arranged in an execution sequence, corresponding to interconnected processing blocks forming candidate pipelines)
selecting, from the one or more candidate data processing and inference pipelines, a data processing and inference pipeline characterized by an optimum overall performance score (Chen, Par. [0056], “In various embodiments, the joint optimization component 302 can optimize the one or more identified machine learning pipelines via data allocation using upper bound algorithms, alternating direction method of multiplies (“ADMM”) algorithms, and/or continuous joint optimization algorithms. For example, the joint optimization component 302 can employ one or more data allocation using upper bound algorithms to optimize the identified machine learning pipelines on subsets of the time series data”, & Par. [0068], “In various embodiments, the hyperparameter component 502 can employ a hyperparameter optimization to select a set of optimal hyperparameters for the identified machine learning pipelines. Thereby, the hyperparameter component 502 can select a set of hyperparameters used to control the automated machine learning process executed by the time series analysis component 108”, & Par. [0087], “At 1104 , the computer-implemented method 1100 can comprise selecting (e.g., via learner component 112 and/or joint optimization component 302 ), by the system 100 , one or more machine learning pipelines for meta transfer learning on the time series data by sequentially allocating subsets of training data from the time series data amongst a plurality of machine learning pipeline candidates. In accordance with various embodiments described herein, the one or more machine learning pipelines can be selected from a plurality of machine learning pipelines included in a pipeline library 120”, thus selecting, from the one or more candidate data processing and inference pipelines, a data processing and inference pipeline characterized by an optimum overall performance score is disclosed, because Chen teaches optimizing candidate machine learning pipelines using optimization algorithms to improve their performance, selecting optimal configurations via hyperparameter optimization, and selecting one or more machine learning pipelines from a plurality of pipeline candidates stored in a library, which collectively correspond to selecting a pipeline from candidate pipelines based on optimal performance)
obtaining, from a repository, information characterizing respective data processing and inference blocks for evaluating the performance of a respective data processing and interference block in terms of the obtained performance metrics by characterizing the respective identified data processing and inference blocks in terms of their respective input and output data constraints and the obtained performance metrics (Chen, Par. [0042], “In one or more embodiments, one or more pipeline libraries 120 can be stored in the one or more memories 114 (e.g., on the one or more servers 102 and/or on one or more external devices associable via the one or more networks 104 ). The one or more pipeline libraries 120 can include one or more machine learning pipelines that can be employed in one or more automated machine learning process of the time series analysis component 108 . For example, the one or more machine learning pipelines included in the pipeline library 120 can be machine learning pipelines developed from past machine learning tasks”, & Par. [0070], “In one or more embodiments, the feature component 602 can use semantic relations and rules defined within a knowledge database to define one or more data transformations applicable to the one or more identified machine learning pipelines. For example, the feature component 602 can analyze the knowledge library 122 to select one or more knowledge databases to employ in the feature engineering of the machine learning pipelines. For instance, the feature component 602 can select one or more knowledge databases from the knowledge library 122 based on, but not limited to: a domain of the time series data, tabular data, a combination thereof, and/or the like. As described herein, the knowledge database can include one or more semantic relationships and/or rules that can direct one or more data transformations performed by the feature component 602”, & Par. [0071], “In various embodiments, the knowledge database can include one or more semantic relationships (e.g., domain-specific semantic relationships), wherein the feature component 602 can transform one or more data points of the time series data based on a semantic relationship (e.g., domain-specific semantic relationship) from the knowledge database to engineer one or more features for the one or more machine learning pipelines”, thus obtaining, from a repository, information characterizing respective data processing and inference blocks for evaluating the performance of a respective data processing and interference block in terms of the obtained performance metrics by characterizing the respective identified data processing and inference blocks in terms of their respective input and output data constraints and the obtained performance metrics is disclosed, because Chen teaches retrieving machine learning pipelines from a stored pipeline library containing pipelines developed from past tasks, and obtaining semantic relationships and rules from knowledge databases in a knowledge library that guide how data is transformed and processed for those pipelines, which correspond to obtaining repository-stored information describing processing components, their operational rules, and characteristics used to evaluate their performance)
identifying, based on the obtained information, data processing and inference blocks characterized with corresponding input and output data constraints (Chen, Par. [0041], “Additionally, the one or more input devices 106 can be employed to receive and/or visualize one or more outputs generated by the time series analysis component 108 . Further, the one or more input devices 106 can be employed to set one or more characteristics of the one or more outputs generated by the time series analysis component 108 . For example, wherein the time series analysis component 108 can generate an ensemble of machine learning pipelines to execute the machine learning task, the one or more input devices 106 can be employed to delineate the number of machine learning pipelines included in the ensemble”, & Par. [0044], “Wherein the learner component 112 identifies a plurality of machine learning pipelines that can be applicable to analyze the time series data, the learner component 112 can further rank the identified machine learning pipelines in order of expected accuracy. For example, the pipeline library 120 can include 20 machine learning pipelines, wherein the learner component 112 can employ meta transfer learning to identify and rank 3 of the 20 machine learning pipelines for further optimization and/or configuration in the automated machine learning processes executed by the time series analysis component 108”, thus identifying, based on the obtained information, data processing and inference blocks characterized with corresponding input and output data constraints is disclosed, because Chen teaches that a learner component identifies applicable machine learning pipelines from a plurality of pipelines and ranks selected ones for further optimization, and also discloses configuring characteristics of outputs and execution of generated pipelines via input devices, which together correspond to identifying processing components based on their operational characteristics and suitability)
calculating one or more performance scores for the respective identified data processing and inference blocks by applying […] to the obtained metadata, the one or more performance scores thereby indicating the performance of the respective identified data processing and inference blocks in terms of the obtained metadata and performance metrics (Chen, Par. [0049], “The learner component 112 can compare the meta-data of the machine learning pipelines with the characteristics of the time series data to predict how the respective machine learning pipelines will perform on the time series data. By employing the meta-data in the comparison, the learner component 112 can leverage insights previously developed by the machine learning pipelines to predict one or more performance metrics with regards to the time series data. Thereby, the learner component 112 can identify machine learning pipelines suitable for execution with the time series data without running the machine learning pipeline candidates on the full time series data. In various embodiments, the learner component 112 can identify a machine learning pipeline for further development by the time series analysis component 108 , wherein the predicted performance of the machine learning pipeline with respect to the time series data based on the meta data is greater than a defined threshold with regards to a defined performance metric (e.g., an accuracy metric)”, & Par. [0051], “Additionally, the learner component 112 can rank the identified machine learning pipelines in an order based on the defined evaluation metric. For example, the learner component 112 can rank the identified machine learning pipelines in order based on the predicted accuracy of the machine learning pipelines on the time series data. Example algorithms and/or techniques that can be employed by the learner component 112 to facilitate the ranking can include, but are not limited to: linear regression, support vector machines, decision trees, gradient boost machines, a combination thereof, and/or the like. Wherein the one or more input devices 106 are employed to define the number of machine learning pipelines to be further developed by the time series analysis component 108 , the identified machine learning pipelines selected for further development can be based on the ranking and the defined number”, thus calculating one or more performance scores for the respective identified data processing and inference blocks by applying operations to the obtained metadata, the scores indicating performance in terms of the metadata and performance metrics, is disclosed, because Chen teaches comparing pipeline metadata with characteristics of input data to predict how respective pipelines will perform and to generate predicted performance metrics used to identify suitable pipelines, and also discloses ranking the identified pipelines based on predicted accuracy using evaluation algorithms such as regression or decision models, which together correspond to computing performance scores for identified processing components based on metadata-driven evaluation)
Chen does not explicitly teach fuzzifying to derive fuzzy metadata, input and output data constraints and sets of rules, and fuzzy inference.
However, Panfilov teaches fuzzifying the obtained metadata to derive fuzzy metadata (Panfilov, Par. [0126], “Define the linguistic variables for each of the components. A linguistic variable is usually defined as a quintuple: (x,T(x),U,G,M), where x is the name of the variable, T(x) is a term set of the x, that is the set of the names of the linguistic values of x, with a fuzzy set defined in U as a value, G is a syntax rule for the generation of the names of the values of the x and M is a semantic rule for the association of each value with its meaning. In the present case, x is associated with the signal name from x or y, term set T(x) is defined using vertical relations, U is a signal range. In some cases one can use normalized teaching signals, then the range of U is [0,1]. The syntax rule G in the linguistic variable optimization can be omitted, and replaced by indexing of the corresponding variables and their fuzzy sets”, & Par. [0127], “Semantic rule M varies depending on the structure of the FIS, and on the choice of the fuzzy model. For the representation of all signals in the system, it is necessary to define m+n linguistic variables”, & Par. [0128], “Let[X,Y], X=(X1, . . . , Xm), Y=(Y1, . . . , Yn) be the set of the linguistic variables associated with the input and output signals correspondingly. Then for each linguistic variable one can define a certain number of fuzzy sets to represent the variable: [mathematical formula - see original document][mathematical formula - see original document] Where [mu]Xi⟨j⟩ ⟨i⟩ , i=1, . . . , m, ji=1, . . . , lXi are membership functions of the ith component of the input variable; and [mu]Yi⟨j⟩ ⟨i⟩ , i=1, . . . , n, ji=1, . . . , lYi are membership functions of the ith component of the output variable”, thus fuzzifying to derive fuzzy metadata is disclosed, because Panfilov teaches defining linguistic variables for signal components, where each variable is associated with fuzzy sets representing its values, defining sets of linguistic variables required to represent system signals, and assigning multiple membership functions to each variable to represent its values in fuzzy form, which together correspond to transforming input data into fuzzy representations, i.e., fuzzifying the data)
Panfilov also teaches input and output data constraints and sets of rules (Panfilov, Par. [0132], “Knowing the number of membership functions, it is possible to introduce a constraint on the possibility of activation of each fuzzy set, denoted as pXi⟨j⟩”, & Par. [0133], “One of the possible constraints can be introduced as: [mathematical formula - see original document]”, & Par. [0134], “This constraint will cluster the signal into the regions with equal probability, which is equal to division of the signal's histogram into curvilinear trapezoids of the same surface area. Supports of the fuzzy sets in this case are equal or greater to the base of the corresponding trapezoid. How much greater the support of the fuzzy set should be, can be defined from an overlap parameter. For example, the overlap parameter takes zero, when there is no overlap between two attached trapezoids. If it is greater than zero then there is some overlap. The areas with higher probability will have in this case "sharper" membership functions. Thus, the overlap parameter is another candidate for the GA1 search. The fuzzy sets obtained in this case will have uniform possibility of activation”, & Par. [0109], “In Stage 3 precedent part of the rule base is created and rules are ranked according to their firing strength. Rules with high firing strength are kept, whereas weak rules with small firing strength are eliminated. [0110] In Stage 4, a second GA (GA2) optimizes a rule base, using the fuzzy model obtained in Stage 1, optimal linguistic variable parameters obtained in Stage 2, selected set of rules obtained in Stage 3 and the teaching signal”, thus input and output data constraints and sets of rules are disclosed, because Panfilov teaches introducing constraints governing activation of fuzzy sets and signal regions, including constraints defining activation possibility, clustering signals into regions, and overlap parameters controlling membership-function support, which correspond to defining operational constraints on input/output data representations, and also discloses creating a rule base, ranking rules according to firing strength, retaining strong rules, eliminating weak rules, and optimizing the selected rule set using genetic algorithms, which correspond to sets of rules used to evaluate and govern processing behavior)
Panfilov further teaches calculating values by means of fuzzy inference (Panfilov, Par. [0104], “In one embodiment, the SC optimizer 242 includes (as shown in FIG. 3) a fuzzy inference engine in the form of a FNN. The SC optimizer also allows FIS structure selection using models, such as, for example, Sugeno FIS order 0 and 1, Mamdani FIS, Tsukamoto FIS, etc. The SC optimizer 242 also allows selection of the FIS structure optimization method including optimization of linguistic variables, and/or optimization of the rule base. The SC optimizer 242 also allows selection of the teaching signal source, including: the teaching signal as a look up table of input-output patterns; the teaching signal as a fitness function calculated as a dynamic system response; the teaching signal as a fitness function is calculated as a result of control of a real plant; etc.”, thus calculating values by means of fuzzy inference using rule sets applied to input data is disclosed, because panfilov teaches a fuzzy inference engine that applies rule bases and linguistic variables to input signals or fitness-function data to generate evaluated outputs)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chen’s metadata-based pipeline evaluation approach, which reads on calculating performance scores for respective processing components by applying operations to metadata to predict performance and rank pipelines, with Panfilov’s fuzzy logic optimization framework, which reads on transforming input data into fuzzy representations using linguistic variables and membership functions, applying rule bases with defined constraints, and computing evaluated outputs using a fuzzy inference engine, because Panfilov teaches that its SC optimizer generates a knowledge base that is less sensitive under signal variation, reduces the number of rules, produces a smaller model, and is more computationally efficient (Panfilov, Par. [0286], “Table 7 shows that the FC prepared with a KB generated by the SC optimizer 242 is more robust in the presence of reference signal variation. Thus, the SC optimizer creates a robust KB for FC and reduces the number of rules in comparison with a KB created with other approaches. The KB created by the SC optimizer 242 automatically has a relatively more optimal number of rules based. The KB created by the SC optimizer 242 tends to be smaller and thus more computationally efficient. The KB created by the SC optimizer tends to be more robust for excitation signal variation as well as for reference signal variation. Swing Dynamic System Simulation Results, Motion Under Fuzzy Control with Two P(I)D Controllers. Comparison Between Back Propagation FNN and SC Optimizer Control Results”, thereby providing a technique for improving efficiency)
Regarding Claim 15, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
wherein the obtaining metadata comprises obtaining one or more qualitative and/or quantitative metrics characterizing the data to be processed from a user (Chen, Par. [0047], “For instance, the meta-data can describe one or more features of the previously analyzed datasets, such as, but not limited to: a domain of the dataset, number of datapoints in the dataset, number of attributes, percentage of missing values, the scope of the algorithms to be considered, the candidate algorithm's hyperparameter values, the selection of variables, a combination thereof, and/or the like. In another instance, the meta-data can describe one or more interdependencies determined by the machine learning pipelines from the historical datasets, such as, but not limited to: correlations between various attributes and target values, Pearson correlations between various attributes and aggregations (e.g., average and/or standard deviations), cosine similarity, neural-network based comparison (“CNN”), a combination thereof, and/or the like. In a further instance, the meta-data can characterize a topology of the one or more machine learning pipelines (e.g., via a string of identifiers sequenced based on the flow and/or stages of the machine learning pipelines). For example, machine learning pipeline topology can comprise a collection of algorithms and/or an execution sequence of the algorithms, wherein the topology can be represented by a sequence of words. Each of the algorithms can be represented by an identifier (e.g., a hash), wherein the order of the identifiers can represent the execution sequence of the algorithms. Moreover, the meta-data can describe the type of machine learning task performed by the machine learning pipeline and/or the type of machine learning models generated and/or optimized by the machine learning pipeline (e.g., predictive machine learning models for classification, regression, deep learning, a combination thereof, and/or the like). Thereby the meta-data can describe the structure of the machine learning pipelines, a task of the machine learning pipelines, features of the historic datasets previously analyzed by the machine learning pipelines, and/or performance observations (e.g., determined data correlations) achieved by the machine learning pipelines”, thus obtaining metadata comprising one or more qualitative and/or quantitative metrics characterizing the data to be processed from a user is disclosed, because Chen teaches metadata describing dataset characteristics such as domain, number of datapoints, number of attributes, missing-value percentage, correlations, statistical measures, and algorithm-selection parameters, which correspond to qualitative and quantitative metrics characterizing the data used for processing, and also discloses that such metadata characterizes datasets and tasks associated with machine-learning pipelines, thereby reading on obtaining metrics that describe the data to be processed)
Regarding Claim 16, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
wherein the obtaining metadata comprises obtaining data to be processed and deriving therefrom one or more qualitative and/or quantitative metadata characterizing the data to be processed (Chen, Par. [0047], “For instance, the meta-data can describe one or more features of the previously analyzed datasets, such as, but not limited to: a domain of the dataset, number of datapoints in the dataset, number of attributes, percentage of missing values, the scope of the algorithms to be considered, the candidate algorithm's hyperparameter values, the selection of variables, a combination thereof, and/or the like. In another instance, the meta-data can describe one or more interdependencies determined by the machine learning pipelines from the historical datasets, such as, but not limited to: correlations between various attributes and target values, Pearson correlations between various attributes and aggregations (e.g., average and/or standard deviations), cosine similarity, neural-network based comparison (“CNN”), a combination thereof, and/or the like. In a further instance, the meta-data can characterize a topology of the one or more machine learning pipelines (e.g., via a string of identifiers sequenced based on the flow and/or stages of the machine learning pipelines). For example, machine learning pipeline topology can comprise a collection of algorithms and/or an execution sequence of the algorithms, wherein the topology can be represented by a sequence of words. Each of the algorithms can be represented by an identifier (e.g., a hash), wherein the order of the identifiers can represent the execution sequence of the algorithms. Moreover, the meta-data can describe the type of machine learning task performed by the machine learning pipeline and/or the type of machine learning models generated and/or optimized by the machine learning pipeline (e.g., predictive machine learning models for classification, regression, deep learning, a combination thereof, and/or the like). Thereby the meta-data can describe the structure of the machine learning pipelines, a task of the machine learning pipelines, features of the historic datasets previously analyzed by the machine learning pipelines, and/or performance observations (e.g., determined data correlations) achieved by the machine learning pipelines”, thus obtaining data to be processed and deriving therefrom qualitative and/or quantitative metadata characterizing the data is disclosed, because Chen teaches that metadata is generated describing features of datasets such as domain, number of datapoints, number of attributes, percentage of missing values, correlations, averages, standard deviations, and similarity measures, which correspond to metadata derived from the data itself that characterizes the data to be processed)
Regarding Claim 17, Chen combined with Panfilov teaches all of the limitations of claim 16 as cited above and Chen further teaches:
wherein the deriving comprises processing the received data by means of data analysers (Chen, Par. [0047], “For instance, the meta-data can describe one or more features of the previously analyzed datasets, such as, but not limited to: a domain of the dataset, number of datapoints in the dataset, number of attributes, percentage of missing values, the scope of the algorithms to be considered, the candidate algorithm's hyperparameter values, the selection of variables, a combination thereof, and/or the like. In another instance, the meta-data can describe one or more interdependencies determined by the machine learning pipelines from the historical datasets, such as, but not limited to: correlations between various attributes and target values, Pearson correlations between various attributes and aggregations (e.g., average and/or standard deviations), cosine similarity, neural-network based comparison (“CNN”), a combination thereof, and/or the like. In a further instance, the meta-data can characterize a topology of the one or more machine learning pipelines (e.g., via a string of identifiers sequenced based on the flow and/or stages of the machine learning pipelines). For example, machine learning pipeline topology can comprise a collection of algorithms and/or an execution sequence of the algorithms, wherein the topology can be represented by a sequence of words. Each of the algorithms can be represented by an identifier (e.g., a hash), wherein the order of the identifiers can represent the execution sequence of the algorithms. Moreover, the meta-data can describe the type of machine learning task performed by the machine learning pipeline and/or the type of machine learning models generated and/or optimized by the machine learning pipeline (e.g., predictive machine learning models for classification, regression, deep learning, a combination thereof, and/or the like). Thereby the meta-data can describe the structure of the machine learning pipelines, a task of the machine learning pipelines, features of the historic datasets previously analyzed by the machine learning pipelines, and/or performance observations (e.g., determined data correlations) achieved by the machine learning pipelines”, & Par. [0049], “The learner component 112 can compare the meta-data of the machine learning pipelines with the characteristics of the time series data to predict how the respective machine learning pipelines will perform on the time series data. By employing the meta-data in the comparison, the learner component 112 can leverage insights previously developed by the machine learning pipelines to predict one or more performance metrics with regards to the time series data. Thereby, the learner component 112 can identify machine learning pipelines suitable for execution with the time series data without running the machine learning pipeline candidates on the full time series data. In various embodiments, the learner component 112 can identify a machine learning pipeline for further development by the time series analysis component 108 , wherein the predicted performance of the machine learning pipeline with respect to the time series data based on the meta data is greater than a defined threshold with regards to a defined performance metric (e.g., an accuracy metric). In one or more embodiments, the learner component 112 can narrow the field of machine learning pipeline candidates based on the predicted runtime (e.g., determined from the meta-data) associated with the machine learning pipelines in order to meet one or more runtime thresholds defined by the one or more input devices 106”, thus deriving metadata by processing the received data using data analysers is disclosed, because Chen teaches generating metadata describing dataset characteristics such as number of datapoints, number of attributes, percentage of missing values, correlations, averages, standard deviations, and similarity measures, which correspond to metadata obtained through analytical processing of the data, and further teaches comparing such metadata with characteristics of time series data to predict performance metrics, thereby indicating that the data is processed to derive metadata characterizing the data)
Regarding Claim 18, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Panfilov further teaches:
wherein the fuzzifying comprises applying sets of membership functions to the respective metadata (Panfilov, Par. [0126], “Define the linguistic variables for each of the components. A linguistic variable is usually defined as a quintuple: (x,T(x),U,G,M), where x is the name of the variable, T(x) is a term set of the x, that is the set of the names of the linguistic values of x, with a fuzzy set defined in U as a value, G is a syntax rule for the generation of the names of the values of the x and M is a semantic rule for the association of each value with its meaning. In the present case, x is associated with the signal name from x or y, term set T(x) is defined using vertical relations, U is a signal range. In some cases one can use normalized teaching signals, then the range of U is [0,1]. The syntax rule G in the linguistic variable optimization can be omitted, and replaced by indexing of the corresponding variables and their fuzzy sets”, & Par. [0127], “Semantic rule M varies depending on the structure of the FIS, and on the choice of the fuzzy model. For the representation of all signals in the system, it is necessary to define m+n linguistic variables”, & Par. [0128], “Let[X,Y], X=(X1, . . . , Xm), Y=(Y1, . . . , Yn) be the set of the linguistic variables associated with the input and output signals correspondingly. Then for each linguistic variable one can define a certain number of fuzzy sets to represent the variable: [mathematical formula - see original document][mathematical formula - see original document] Where [mu]Xi⟨j⟩ ⟨i⟩ , i=1, . . . , m, ji=1, . . . , lXi are membership functions of the ith component of the input variable; and [mu]Yi⟨j⟩ ⟨i⟩ , i=1, . . . , n, ji=1, . . . , lYi are membership functions of the ith component of the output variable”, thus fuzzifying by applying sets of membership functions to respective data is disclosed, because Panfilov teaches defining linguistic variables for system components where each variable is represented by fuzzy sets and associated membership functions, and also teaches that for each input and output variable multiple membership functions are defined to represent its values, which corresponds to applying sets of membership functions to data to obtain fuzzy representations)
Regarding Claim 19, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Panfilov further teaches:
wherein the sets of rules for evaluating the performance of a respective data processing and interference block are derived based on its input and output constraints and the obtained performance metrics (Panfilov, Par. [0213], “The structure realizing evaluation procedure of GA2 or GA3 is shown in FIG. 17. In FIG. 17, the SC optimizer 17001 sends the KB structure presented in the current chromosome of GA2 or of GA3 to FC 17101. An input part of the teaching signal 17102 is provided to the input of the FC 17101. The output part of the teaching signal is provided to the positive input of adder 17103. An output of the FC 17101 is provided to the negative input of adder 17103. The output of adder 17103 is provided to the evaluation function calculation block 17104. Output of evaluation function calculation block 17104 is provided to a fitness function input of the SC optimizer 17001, where an evaluation value is assigned to the current chromosome”, thus sets of rules for evaluating performance derived based on input and output constraints and obtained performance metrics are disclosed, because Panfilov teaches that a rule-base structure is supplied to a functional component, input signals are applied, system outputs are compared against reference outputs, and the resulting difference is processed by an evaluation function that generates a fitness value assigned to the rule structure, which corresponds to deriving and assessing rule sets based on input/output data and calculated performance results)
Regarding Claim 20, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
wherein the calculating further comprises aggregating the derived respective performance scores of the identified data processing and inference blocks, and, combining the aggregated performance scores, thereby deriving an overall performance score for a respective candidate data processing and inference pipeline, wherein the combining comprises weighting the aggregated performance scores in accordance with weight factors associated with respective performance metrics (Chen, Par. [0059], “In various embodiments, a joint optimization executed by the joint optimization component 302 can include a fixed data allocation part and a data allocation acceleration part. At a first step of the fixed data allocation part, the joint optimization component 302 can allocate a minimum data subset of the time series data to each identified machine learning pipeline. For example, the data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Next, the joint optimization component 302 can allocate an additional data subset of the time series data (e.g., an older data subset) to each identified machine learning pipeline. For example, the additional data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Additionally, the joint optimization component 302 can repeatedly allocate additional data subsets and record testing results for a defined number of repetitions”, & Par. [0060], “Once the joint optimization component 302 has allocated the defined number of additional subsets, the joint optimization component 302 can determine a vector of the recorded test scores for each identified machine learning pipeline. For example, the vector can correspond to a sample size as a function of: the minimum allocation size, the size of the additional subset increments, and the number of additional subset increments. Additionally, the joint optimization component 302 can fit a linear regression using the determined vectors to predict a score for each of the identified machine learning pipelines at a target sample size. Thereby, the joint optimization component 302 can predict the performance of the identified machine learning pipelines at a target sample size of the time series data based on the allotted subsets of the time series data, wherein the size of the allotted subsets can be smaller than the target sample size. Further, the joint optimization component 302 can rank the identified machine learning pipelines based on the predicted performance scores at the target sample size. In one or more embodiments, as the predicted score decreases, the predicted accuracy of the machine learning pipeline at the target sample size can increase. The ranking can facilitate the joint optimization component 302 in selecting those machine learning pipelines anticipated to receive the most benefit from further optimization”, & Par. [0079], “Additionally, the ensemble component 902 can assign the one or more weights based on the machine learning task being performed on the time series data and/or one or more evaluation metrics (e.g., defined by the one or more input devices 106 ) of the machine learning pipelines with regards to executing the machine learning task. Thereby, the ensemble component 902 can vary the weights included in an ensemble of machine learning pipelines from machine learning task to machine learning task even if the machine learning pipelines included in the ensemble remains the same and/or if the time series data remains the same”, thus aggregating the derived respective performance scores of the identified data processing and inference blocks, and combining the aggregated performance scores, thereby deriving an overall performance score for a respective candidate data processing and inference pipeline, wherein the combining comprises weighting the aggregated performance scores in accordance with weight factors associated with respective performance metrics, is disclosed, because Chen teaches recording multiple test scores for each identified machine learning pipeline across different allocated data subsets, determining a vector of the recorded test scores and predicting a score for each pipeline based on those recorded results, and assigning weights based on the machine learning task and/or evaluation metrics when combining pipelines, which together correspond to aggregating multiple performance scores, combining them to produce an overall performance score for a candidate pipeline, and weighting the combined scores according to associated performance metrics)
Regarding Claim 21, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
wherein the construction is performed in an incremental manner (Chen, Par. [0059], “In various embodiments, a joint optimization executed by the joint optimization component 302 can include a fixed data allocation part and a data allocation acceleration part. At a first step of the fixed data allocation part, the joint optimization component 302 can allocate a minimum data subset of the time series data to each identified machine learning pipeline. For example, the data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Next, the joint optimization component 302 can allocate an additional data subset of the time series data (e.g., an older data subset) to each identified machine learning pipeline. For example, the additional data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Additionally, the joint optimization component 302 can repeatedly allocate additional data subsets and record testing results for a defined number of repetitions”, thus wherein the construction is performed in an incremental manner, is disclosed, because Chen teaches allocating a minimum data subset to each identified machine learning pipeline, training and scoring the pipelines, then allocating additional data subsets and repeating the training and scoring process for a defined number of repetitions, which corresponds to constructing or refining the pipelines progressively using successive increments of data rather than performing the construction in a single step)
Regarding Claim 22, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
evaluating the performance of the selected data processing and inference pipeline in terms of the obtained performance metrics and based on the data to be processed, wherein the evaluating comprising at least partly deploying the selected data processing and inference pipeline and determining an actual overall performance score for the at least partly deployed data processing and inference pipeline (Chen, Par. [0059], “In various embodiments, a joint optimization executed by the joint optimization component 302 can include a fixed data allocation part and a data allocation acceleration part. At a first step of the fixed data allocation part, the joint optimization component 302 can allocate a minimum data subset of the time series data to each identified machine learning pipeline. For example, the data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Next, the joint optimization component 302 can allocate an additional data subset of the time series data (e.g., an older data subset) to each identified machine learning pipeline. For example, the additional data subset can be further split into training and testing sets. The joint optimization component 302 can then train the identified machine learning pipelines on the training set and score the machine learning pipelines on the testing set, wherein the resulting scores can be recorded (e.g., stored in the one or more memories 114 ). Additionally, the joint optimization component 302 can repeatedly allocate additional data subsets and record testing results for a defined number of repetitions”, & Par. [0060], “Once the joint optimization component 302 has allocated the defined number of additional subsets, the joint optimization component 302 can determine a vector of the recorded test scores for each identified machine learning pipeline. For example, the vector can correspond to a sample size as a function of: the minimum allocation size, the size of the additional subset increments, and the number of additional subset increments. Additionally, the joint optimization component 302 can fit a linear regression using the determined vectors to predict a score for each of the identified machine learning pipelines at a target sample size. Thereby, the joint optimization component 302 can predict the performance of the identified machine learning pipelines at a target sample size of the time series data based on the allotted subsets of the time series data, wherein the size of the allotted subsets can be smaller than the target sample size. Further, the joint optimization component 302 can rank the identified machine learning pipelines based on the predicted performance scores at the target sample size. In one or more embodiments, as the predicted score decreases, the predicted accuracy of the machine learning pipeline at the target sample size can increase. The ranking can facilitate the joint optimization component 302 in selecting those machine learning pipelines anticipated to receive the most benefit from further optimization”, thus evaluating the performance of the selected data processing and inference pipeline in terms of the obtained performance metrics and based on the data to be processed, wherein the evaluating comprises at least partly deploying the selected data processing and inference pipeline and determining an actual overall performance score for the at least partly deployed data processing and inference pipeline, is disclosed, because Chen teaches allocating subsets of the time series data to identified machine learning pipelines, training the pipelines on the training sets, and scoring the pipelines on the testing sets, with the resulting scores recorded. Chen also teaches determining vectors of recorded test scores and predicting performance at a target sample size based on those recorded results, which corresponds to at least partial deployment of the selected pipeline on actual data and determining an actual overall performance score based on performance metrics derived from that data)
Regarding Claim 23, Chen combined with Panfilov teaches all of the limitations of claim 22 as cited above and Chen further teaches:
wherein the evaluating further comprises determining values of respective parameters associated with the respective data processing and inference blocks of the at least partly deployed pipeline (Chen, Par. [0067], “FIG. 5 illustrates a diagram of the example, non-limiting system 100 further comprising hyperparameter component 502 in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. In various embodiments, hyperparameter component 502 can configure one or more hyperparameters employed in the identified machine learning pipelines. Although FIG. 5 depicts the hyperparameter component 502 as separate from the joint optimization component 302 , embodiments in which the hyperparameter component 502 is combined with the joint optimization component 302 are also envisaged”, & Par. [0068], “In various embodiments, the hyperparameter component 502 can employ a hyperparameter optimization to select a set of optimal hyperparameters for the identified machine learning pipelines. Thereby, the hyperparameter component 502 can select a set of hyperparameters used to control the automated machine learning process executed by the time series analysis component 108 . Example hyperparameter optimization approaches that can be employed by the hyperparameter component 502 can include, but are not limited to: grid search, random search, gradient-based optimization, Bayesian optimization, evolutionary optimization, population-based training, alternating direction method of multipliers (“ADMM”), a combination thereof, and/or the like. For instance, the hyperparameter component 502 can employ a principal component analysis (“CPA”) and/or k-nearest neighbors algorithm to automatically configure hyperparameters for the machine learning pipelines”, thus determining values of respective parameters associated with the respective data processing and inference blocks of the at least partly deployed pipeline is disclosed, because Chen teaches a hyperparameter component that configures hyperparameters used in identified machine learning pipelines and further teaches selecting a set of optimal hyperparameters through optimization techniques such as grid search, Bayesian optimization, and methods to control operation of the pipelines, which together correspond to determining parameter values for respective processing components of the pipeline)
Regarding Claim 24, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
a computer program product comprising computer-executable instructions for causing a computer to perform the method according to claim 14 when the program is run on the computer (Chen, Par. [0024], “Various embodiments of the present invention can be directed to computer processing systems, computer-implemented methods, apparatus and/or computer program products that facilitate the efficient, effective, and autonomous (e.g., without direct human guidance) generation of machine learning pipelines for time series data analysis”, thus a computer program product comprising computer-executable instructions for causing a computer to perform the method is disclosed, because Chen teaches computer program products and computer-implemented methods that facilitate automated generation of machine learning pipelines for time-series data analysis, which corresponds to software instructions executed by a computer to perform the recited method)
Regarding Claim 25, Chen combined with Panfilov teaches all of the limitations of claim 24 as cited above and Chen further teaches:
A computer readable storage medium comprising computer program product according to claim 24 (Chen, Par. [0005], “According to an embodiment, a computer program product for generating an automated machine learning process that analyzes time series data is provided. The computer program product can comprise a computer readable storage medium having program instructions embodied therewith”, thus a computer readable storage medium comprising a computer program product is disclosed, because Chen teaches a computer program product that includes a computer-readable storage medium having program instructions embodied therewith for generating an automated machine learning process that analyzes time-series data)
Regarding Claim 26, Chen combined with Panfilov teaches all of the limitations of claim 14 as cited above and Chen further teaches:
A data processing system programmed for carrying out the method according to claim 14 (Chen, Par. [0024], “Various embodiments of the present invention can be directed to computer processing systems, computer-implemented methods, apparatus and/or computer program products that facilitate the efficient, effective, and autonomous (e.g., without direct human guidance) generation of machine learning pipelines for time series data analysis”, thus …Thus, a data processing system programmed for carrying out the method is disclosed, because Chen teaches computer processing systems configured to facilitate autonomous generation of machine learning pipelines for time-series data analysis)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. US20210209488A1 is pertinent because it teaches an inference computing apparatus that receives a trained inference model generated from historical manufacturing data, applies the model to new data to produce inference results, and evaluates the model’s performance to determine whether updating is required. It also teaches a feedback-based evaluation loop in which model performance on processed data is assessed and used to trigger model modification, thereby supporting adaptive optimization of machine-learning models. Because the applicant likewise concerns evaluating performance of data processing or inference models and modifying or selecting models based on performance metrics derived from processed data, this reference is relevant to the claimed invention as it addresses automated inference execution, performance assessment, and iterative model updating within a computing system.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHLIET ADMASU whose telephone number is (571)272-0034. The examiner can normally be reached Mon-Fri, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.T.A./
Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123