Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 08/06/2024 and 05/07/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claim 1, the claim recites that “an output of at least one task of the tasks is input to the task in a subsequent stage”. The phrases “the task in a subsequent stage” lacks clear antecedent basis. It is unclear whether “the task” refers to the previously recited “at least one task”, another task of the plurality of task or a different task in a later stage.
Regarding claims 4 and 5, the claims recite inputting and output or correct answer label to “the provisional models to the provisional model in a subsequent stage” lacks clear antecedent basis. It is unclear whether the provisional model is the previously recited “at least one provisional model” or another model in a later stage.
Regarding claims 19 and 20, the claims recites similar limitation as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar rationale.
Regarding claims 2-3 and 6-18, dependent claims inherit the deficiencies of the respective parent claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 6-7 and 19-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Taher (US 20220066813 A1).
Regarding claim 1, Taher teaches:
A device for determining an execution order of a plurality of tasks, the device comprising: a processor; and a memory connected to or built in the processor, the processor being configured to. ([0111] A computer system of the present invention comprises one or more processors, one or more memories, and one or more computer readable hardware storage devices, said one or more hardware storage device containing program code executable by the one or more processors via the one or more memories to implement the methods of the present invention.)
execute execution order determination processing of determining an execution order of the plurality of tasks under a condition that an output of at least one task of the tasks is input to the task in a subsequent stage. ([0029] A pipeline framework is a software framework which allows a developer to define one or more data processing pipelines via a graphical user interface, declarative configuration files, or otherwise. The framework subsequently schedules and executes each stage of the data processing pipeline using a compute framework, passing the data resulting from each stage to any subsequent stages which require it as defined by the edges of the DAG. [0044] The method may create one or more optimized data processing pipelines by performing a dependency resolution procedure on stages of all tasks in parallel using the task dependencies to determine the order of the stages and removing duplication of stages between tasks. This may include constructing 305 a dependency super graph of the stages of the tasks and dynamically converting 306 the graph into one or more pipelines when the tasks are scheduled for execution. The dependency super graph may be a directed acyclic graph (DAG) in which the nodes of the graph represent stages of one or more tasks, and directed edges represent data dependencies between stages and which contains no cycles. The dependency super graph refers to the relationships between the stages within the tasks, before optimization, representing the order that they must be executed in. These relationships can be represented/modelled as a DAG. The ‘super’ aspect here refers to the fact that all the dependency relationships between stages within all tasks are amalgamated into a single DAG.[0045] Creating one or more optimized data processing pipelines by constructing a dependency super graph may collate multiple tasks into a single pipeline optimizing the pipeline stages in real time, wherein the optimization includes removing tasks that are not required by users and deduplicating common stages between tasks. [0046] The identifying 303 of one or more tasks to be currently executed from the defined tasks determines which tasks need to be executed at any given time. As this is dynamic, the set of tasks which need to be executed may be constantly changing. At any given time, the set of tasks is fed into the dependency resolution algorithm to generate one or more optimized data processing pipelines. This may have to happen in real time based on the current set of tasks which need to be executed to ensure the data processing pipelines contain all the stages of all the tasks which need to be executed at that time. [0047] The method may use a depth first search to identify if only a single optimized pipeline or multiple pipelines exist and may use a topological sorting to order stages in a pipeline to satisfy dependencies and for deduplication of stages in a defined order. [0060] Once each stage completes, the runtime mediation program stores any output data sets. When a stage to be declares an incoming stage dependency as determined from the Pipeline Definition, the runtime mediation program passes the output data sets from the incoming stage as input to the stage to be executed, this resolving the dependency. The runtime mediation program is considered part of the pipeline framework 110. Pseudocode is provided for this process in Listing. 2.)
Regarding claim 6, Taher teaches:
The device for determining an execution order according to claim 1, wherein the processor is configured to further execute display processing of displaying the execution order on a display unit. ([0035] In a typical pipeline framework 110, the developer defines an execution graph as a DAG with edges representing a directional data set dependency between the nodes, which are stages of the tasks to be executed. This process is typically carried out via a GUI or declarative configuration files.)
Regarding claim 7, Taher teaches:
The device for determining an execution order according to claim 1, wherein, in the execution order determination processing, the processor is configured to determine the execution order in consideration of an execution order designated by a user. ([0040] The method may start 301 and the pipeline framework 110 may read in 302 task definitions with each task defining of stages of input data, transformation of data, and output data. Tasks are defined declaratively according to task manifests, for example, as declarative configuration files or via a graphical user interface.[0041] The method may identify 303 one or more tasks to be currently executed from the defined tasks. This may identify one or more tasks to be executed dynamically by a schedule maintained by the pipeline framework 110, actions performed by a user, and/or reported state changes within the system. The tasks to be executed (or not executed) may change depending on external factors in the system. This would result in dynamic change to the pipeline as different tasks would be fed into the pipeline generation.)
Regarding claim 19, the claim recites similar limitation as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale.
Regarding claim 20, the claim recites similar limitation as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 are rejected under 35 U.S.C. 103 as being unpatentable over Taher (US 20220066813 A1) in view of Chen (US 20220092407 A1).
Regarding claim 2, Taher does not appear to explicitly teach:
The device for determining an execution order according to claim 1, wherein: the plurality of tasks include a first task and a second task, and in the execution order determination processing, the processor is configured to: create, for each of correct answer labels of the first task, a frequency distribution of each of correct answer labels of the second task by using a training data set, and calculate a first statistic based on the frequency distribution. create, for each of the correct answer labels of the second task, a frequency distribution of each of the correct answer labels of the first task by using a training data set, and calculate a second statistic based on the frequency distribution; and determine the execution order of the plurality of tasks based on the first statistic and the second statistic.
However, Chen teaches:( [0031] As illustrated in FIG. 1, in addition to input data transformation via an adversarial program, the system and/or method in one or more embodiments also map the source task's output labels (e.g., different objects) 106 to the target task's output labels, for example, shown at 116 as ‘A’ and ‘non-A’. For example, in medical imaging, target labels can include different medical conditions such as Autism Spectrum Disorder (ASD) (e.g., shown as ‘A’ in FIG. 1) or non-ASD, diabetic retinopathy (DR) (e.g., shown as ‘B’ in FIG. 1), and melanoma (e.g., shown as ‘C’ in FIG. 1), and/or others. The system and/or method in one or more embodiments may map a source label to a target label. The system and/or method in one or more embodiments may map multiple-source-labels to one-target-label. Such mapping can further improve the accuracy of the target task, for example, when compared to one-to-one label mapping. For instance, the prediction of a transformed data input from the source label set {Tench, Goldfish, Hammerhead} can be reprogrammed for predicting the target class, e.g., ASD. Let K (K′) be the total number of classes for the source (target) task. The system and/or method in one or more embodiments use the notation h.sub.j(⋅) to denote the k-to-1 mapping function that averages the predictions of a group of k source labels as the prediction of the j-th target domain's label. For example, if the source labels {Tench, Goldfish, Hammerhead} map to the target label {ASD}, then h.sub.ASD(F(X))=[F.sub.Tench(X)+F.sub.Goldfish(X)+F.sub.Hammerhead(X)]/3. More generally, if a subset of source labels ⊂[K] map to a target label jϵ[K′], then hj(F(X))=1.Math.S.Math.Σs∈SFs(X),where is the set quantity or the number of elements in S. In an embodiment, the system and/or method may use or implement random label mapping of source labels to a target label. In another embodiment, the system and/or method may use or implement a frequency-based label mapping scheme by matching target labels to source labels according to the label distribution of initial predictions on the target-domain data before reprogramming. For example, frequency-based multi-label mapping (MLM) derived from the initial predictions of the target-domain data before reprogramming can be used. [0049] In one or more embodiments, different number of random vectors q and multi-label mapping (MLM) size m for BAR, which can be configurable, can be used or implemented (m and q can be numbers, e.g., integer values). Different loss functions (e.g., CE-loss, F-loss) and label mapping methods (e.g., random mapping, frequency mapping) can be implemented or used for BAR. [0050] For example, for random mapping, for each target-domain class, the system and/or method in one or more embodiments can randomly assign m separate labels from the source domain. For frequency mapping, in each task, the system and/or method in one or more embodiments may obtain the source-label prediction distribution of the target-domain data before reprogramming. Based on the distribution, the system and/or method in one or more embodiments may then sequentially assign the most frequent source-label to the corresponding dominating target-label until each target-label has been assigned with m source-labels. [0055] At 308, output labels of the machine learning model can be mapped to target labels associated with the target domain training data. In an embodiment, mapping may include multiple-to-one mapping, wherein multiple of the output labels of the machine learning model are mapped to a target label of the target labels. In an embodiment, mapping may include mapping m output labels of the machine learning model to a target label of the target labels, and m can be configurable. In an embodiment, mapping may include randomly mapping the output labels to the target labels. In an embodiment, mapping may include frequency-based mapping, wherein output label prediction distribution of the target domain training data is obtained from the machine learning model before reprogramming, and a most frequent output label prediction is assigned to a corresponding dominating target label until each target label is assigned with a source label. For instance, the most frequent output label(s) predicted may be assigned to the most dominating target label, the next most frequent output label(s) predicted may be assigned to the next most dominating target label, and so on, until all target labels are assigned with one or more label labels. A source label refers to a label, which the machine learning model is pre-trained to predict. A target label refers to a label associated with the target domain training data.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Taher and Chen before them, to include Chen’s frequency based label mapping technique in Taher’s task-ordering pipeline system. Chen teaches mapping task output labels using a frequency based mapping based on label distribution. One would have been motivated to use Chen’s technique in Taher’s system to more accurately determine the relationship between a first machine learning task and second machine learning task and use that to determine an appropriate execution order.
Claims 3, 9-13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Taher (US 20220066813 A1) in view of Bansal (US 20210097444 A1).
Regarding claim 3, Taher does not appear to explicitly teach:
The device for determining an execution order according to claim 1, wherein, in the execution order determination processing, the processor is configured to: extract a plurality of order patterns that are considered for the plurality of tasks; generate a plurality of provisional models for respectively solving the plurality of tasks according to each of the extracted order patterns; and determine the execution order of the plurality of tasks based on a result obtained by training the plurality of provisional models and evaluating prediction accuracy.
However, Bansal teaches: [0044] The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116. The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column, etc. The pipeline recommender system 112 can use the dataset provided by the user and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines. Thereafter, the pipeline optimizer system 116 can use the pipelines recommended by pipeline recommender system 112 to start an optimization job, which typically involves running multiple training jobs to identify the most performant ones. As the optimization job progresses, the pipeline optimizer system 116 can discard the low-performing models and can tune the hyperparameters of the most performant ones.[0045] In some embodiments, users may utilize both the pipeline recommendation system 112 (to generate candidate ML pipelines to explore) as well as the pipeline optimizer system 116 (to explore and evaluate the candidate ML pipelines). However, in some embodiments users may use portions of the AMPGS 102 independently—e.g., only a pipeline recommendation system 112 to identify candidate ML pipelines (and thereafter use this data elsewhere), or only a pipeline optimization system 116 to evaluate candidate ML pipelines (e.g., using candidate ML pipelines that may have been developed with or without use of the pipeline recommendation system 112). Thus, in some embodiments these components are designed in a manner such that they are not tightly coupled, and may thus optionally be used independent of one another. [0052] Assuming that the full exploration is to be continued, with reference to FIG. 5, a feature preprocessing analyzer 510A (e.g., each implemented by a separate one or more compute instance(s)) may run for each distinct preprocessing step/transform identified within the recommended ML pipeline plans. Each feature preprocessing analyzer 510A-510M may perform an initialization of a preprocessing task so that the task can be later implemented by a feature preprocessor 515A-515N, and this analysis may be performed at least partially in parallel.[0053] For example, a feature preprocessing analyzer 510A for one-hot encoding may need to run a ML job to find out how many columns it should produce. Thus, it could be the case that a column of a dataset may have 10,000 unique values, where many of these values are only referenced once or twice. Accordingly, the feature preprocessing analyzer 510A may determine to only generate columns for the top X (e.g., 200) values and one column as a “catch-all” for any other value. Thus, the feature preprocessing analyzer 510A will perform this analysis, which may include identifying what all distinct categories are referenced within a column, what the counts of each of these values are, etc., to ultimately determine to what columns should be generated. Similar types of preprocessing analysis can be performed for other types of pipeline preprocessing tasks, e.g., for principal component analysis there is a need to learn the result (e.g., which features should be used), etc.See also [0073-0077] and [0101-0102]
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Taher and Bansal before them, to include Bansal’s machine learning pipeline and in Taher’s task ordering pipeline system. One would have been motivated to make such combination to improve Taher’s pipeline ordering by evaluating multiple candite pipeline patterns and selecting a better performing pipeline based on training results and prediction accuracy.
Regarding claim 9, Bansal teaches:
The device for determining an execution order according to claim 1, wherein the output of at least one task of the tasks is a numerical value obtained by regression. ([0044] The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116. The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column, etc. The pipeline recommender system 112 can use the dataset provided by the user and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model.) Refer to claim 3 for the motivation to combine.
Regarding claim 10, Bansal teaches:
The device for determining an execution order according to claim 1, wherein the output of at least one task of the tasks is a result of classification. ([0035] In some embodiments, users having sufficient ML knowledge may be provided the ability to customize additional aspects of the ML pipeline exploration process. FIG. 4 is a diagram illustrating one exemplary user interface 400 for configuring advanced options for automated machine learning pipeline exploration according to some embodiments.[0036] In this user interface 400, a user may specify via input element 405 what machine learning problem type is at hand, enabling the use to control what type of ML algorithms will be used for training. For example, many different machine learning problem types are known to those of skill in the art, such as binary classification, multi-class classification, linear regression, and so on. In some embodiments, the user may be able to select an “auto detection” option, as the machine learning problem type may be inferred based on the type and/or values of the “target column” the user specified.[0037] A user may also specify an objective metric for the exploration. For example, user interface 400 includes a user interface element 410 where the user can select an objective metric type (e.g., from a list of metrics) that is to be used to evaluate which ML pipeline is the best for the user. In some embodiments, the objective metric can be “auto” (automatic), allowing the AMPGS 102 to use its own selected metric or metrics to determine a best-performing ML pipeline. Various types of metrics can be used and are known to those of skill in the art, including but not limited to mean square error (MSE), classification accuracy, logarithmic loss, area under curve (AUC), mean absolute error (MAE), root mean squared error (RMSE), root mean squared logarithmic error (RMSLE), R squared, F1 score, etc.). Refer to claim 3 for the motivation to combine.
Regarding claim 11, Bansal teaches:
The device for determining an execution order according to claim 1, wherein the output of at least one task of the tasks is a probability value obtained in a classification process. ([0035] In some embodiments, users having sufficient ML knowledge may be provided the ability to customize additional aspects of the ML pipeline exploration process. FIG. 4 is a diagram illustrating one exemplary user interface 400 for configuring advanced options for automated machine learning pipeline exploration according to some embodiments.[0036] In this user interface 400, a user may specify via input element 405 what machine learning problem type is at hand, enabling the use to control what type of ML algorithms will be used for training. For example, many different machine learning problem types are known to those of skill in the art, such as binary classification, multi-class classification, linear regression, and so on. In some embodiments, the user may be able to select an “auto detection” option, as the machine learning problem type may be inferred based on the type and/or values of the “target column” the user specified.[0037] A user may also specify an objective metric for the exploration. For example, user interface 400 includes a user interface element 410 where the user can select an objective metric type (e.g., from a list of metrics) that is to be used to evaluate which ML pipeline is the best for the user. In some embodiments, the objective metric can be “auto” (automatic), allowing the AMPGS 102 to use its own selected metric or metrics to determine a best-performing ML pipeline. Various types of metrics can be used and are known to those of skill in the art, including but not limited to mean square error (MSE), classification accuracy, logarithmic loss, area under curve (AUC), mean absolute error (MAE), root mean squared error (RMSE), root mean squared logarithmic error (RMSLE), R squared, F1 score, etc.). Refer to claim 3 for the motivation to combine.
Regarding claim 12, Bansal teaches:
A device for generating a plurality of machine learning models, the device comprising: a processor; and a memory connected to or built in the processor, the processor being configured to: execute model generation processing of generating a plurality of machine learning models for respectively solving the plurality of tasks according to the execution order determined by the device for determining an execution order according to claim 1. ([0056] Next, the pipeline optimizer system 116 (also with reference to circle (5) of FIG. 1) can utilize the preprocessed datasets and pipeline recommendations (including identifiers of the particular ML algorithms to be used to generate ML models) and can cause the multiple ML models to be trained accordingly, e.g., by calling a model training system 120 described herein (optionally in parallel), utilizing a ML framework, etc., according to any user-stipulated configuration parameters (e.g., how many trials are to be run in total, how many trials can be run at a time, what type of compute instances to use, etc.). The result of each training includes the model artifacts, together with the value of the objective metric to be optimized. The pipeline optimizer system 116 may use this information, e.g., via use of another one or more ML models or databases, to identify additional ML pipeline variants to test (having different hyperparameter values, for example) that are likely to result in a good model. Such hyperparameter tuning systems and techniques are known to those of skill in the art and can be utilized to work to find better and better pipelines.[0057] Notably, in some embodiments during these training trials, the AMPGS 102 may provide updates to the user 109 via an application 103 (e.g., via a console 105 and/or interactive code application 107 such as a Jupyter Notebook), who may view the intermediate results, halt certain trainings (or the entire ML pipeline exploration), etc.[0058] A model ensembler 525 may also be utilized to create one or more ensemble models, if desired by the user or configured by the system, based on use of ones of the explored models. For example, as indicated above, multiple models can be used in an ensemble using one or more ensemble techniques known to those of skill in the art (e.g., using some sort of aggregation or selection of outputs). Based on all results, an output generator 530 may produce output in the form of one or more of a comprehensive result in the form of updated result data to be viewed by the user (e.g., at optional circle (6)), a package of the pipeline exploration artifacts 122 (e.g., code 123 for implementing a preprocessing pipeline, which may include ML models, and/or inference model(s) 124) stored at a storage location (with reference to optional circle (7A) of FIG. 1) to be later obtained by the user or another application, results stored to a database, etc. [0103] As described below, in some embodiments, the model data stored in the training model data store 1075 is used by the model hosting system 140 to deploy machine learning models. Alternatively, or additionally, a user device 1002 or another computing device (not shown) can retrieve the model data from the training model data store 1075 to implement a learning algorithm in an external device. As an illustrative example, a robotic device can include sensors to capture input data. A user device 1002 can retrieve the model data from the training model data store 1075 and store the model data in the robotic device. The model data defines a machine learning model. Thus, the robotic device can provide the captured input data as an input to the machine learning model, resulting in an output. The robotic device can then perform an action (e.g., move forward, raise an arm, generate a sound, etc.) based on the resulting output.[0104] While the virtual machine instances 1022 are shown in FIG. 10 as a single grouping of virtual machine instances 1022, some embodiments of the present application separate virtual machine instances 1022 that are actively assigned to execute tasks from those virtual machine instances 1022 that are not actively assigned to execute tasks. For example, those virtual machine instances 1022 actively assigned to execute tasks are grouped into an “active pool,” while those virtual machine instances 1022 not actively assigned to execute tasks are placed within a “warming pool.” In some embodiments, those virtual machine instances 1022 within the warming pool can be pre-initialized with an operating system, language runtimes, and/or other software required to enable rapid execution of tasks (e.g., rapid initialization of machine learning model training in ML training container(s) 1030) in response to training requests. See also [0117-0120]). Refer to claim 3 for the motivation to combine.
Regarding claim 13, Bansal teaches:
A learning device of a plurality of machine learning models, the device comprising: a processor; and a memory connected to or built in the processor, the processor being configured to: execute training processing of connecting and training the plurality of machine learning models according to the execution order determined by the device for determining an execution order according to claim 1. ([0048] Next, the pipeline recommender system 112 can analyze the dataset (e.g., via use of one or more ML models along with the target column of the dataset, etc.) to recommend pipelines (a combination of one or more preprocessing tasks and a machine learning algorithm) to explore. For example, the pipeline recommender system 112 may utilize one or more ML models that have been trained to identify particular pipeline types that have worked well for particular datasets based on characteristics of those datasets (e.g., numbers and/or types of the columns, type of column to be inferred, etc.). Additionally, or alternatively, the pipeline recommender system 112 may use a database (or other data structure) of prior pipelines (and characteristics of the involved datasets) and compare the corresponding dataset/inference characteristics of the current dataset to identify the most similar historic pipelines, which can be suggested.[0049] Additionally, or alternatively, the pipeline recommender system 112 may generate a number of different preprocessing pipelines (and corresponding training datasets) according to “strategies” that have been configured—e.g., a “baseline” strategy that will 1-hot encode (1HE) all categorical variables, median-impute null values with indicators; a “quadratic” strategy that will hash-encode any detected categorical variables, bucketize numeric features, and add cartesian-product features for predictive feature combinations: instance-type×instance-count, num_layers×instance-type, dataset_size×num_layers; a “log bucketize” strategy that will hash-encode any categorical variables, bucketize numeric variables, and log-transform all numerics. For example, a feature processing pipeline may be generated using two machine learning jobs, which may be performed using a batch inference system 142 (that can train and/or host machine learning models in a batch manner) as shown at optional circle (3B): a training job to learn the transformations, and then a batch processing job to apply the transformation to the dataset to generate transformed datasets for use in exploring different ML models.[0052] Assuming that the full exploration is to be continued, with reference to FIG. 5, a feature preprocessing analyzer 510A (e.g., each implemented by a separate one or more compute instance(s)) may run for each distinct preprocessing step/transform identified within the recommended ML pipeline plans. Each feature preprocessing analyzer 510A-510M may perform an initialization of a preprocessing task so that the task can be later implemented by a feature preprocessor 515A-515N, and this analysis may be performed at least partially in parallel. [0075] In some embodiments, the operations 900 further include splitting the dataset into a plurality of sets, the plurality of sets including a training set, wherein the training of the plurality of the plurality of ML models utilizes at least the training set.[0076] In some embodiments, the training of the plurality of ML models is performed at least partially in parallel in that at least two of the plurality of ML models are actively trained at least partially at a same point in time.[0077] The operations 900 further include, at block 930, transmitting data to the computing device of the user indicating a result of the training.[0078] In some embodiments, operations 900 further include receiving a fifth request message originated by the computing device of the user indicating a request to deploy an ML pipeline corresponding to one of the plurality of ML pipeline plans; transmitting a sixth request message to cause a model hosting system of a provider network to deploy the ML pipeline behind an endpoint; and transmitting an identifier of the endpoint to the computing device or to a storage location.[0079] In some embodiments, the transmitting of the data to the computing device of the user causes the computing device of the user to present the result to the user, the result including identifiers of a plurality of trials corresponding to the plurality of ML models, wherein for each of the plurality of trails the result includes a value of an objective metric generated by the training of the corresponding ML model. In some embodiments, the request further identifies the objective metric.). Refer to claim 3 for the motivation to combine.
Regarding claim 18, Bansal teaches:
A prediction device of a plurality of machine learning models, the device comprising: a processor; and a memory connected to or built in the processor, the processor being configured to: execute prediction processing of connecting the plurality of machine learning models and causing the plurality of machine learning models to perform prediction according to the execution order determined by the device for determining an execution order according to claim 1. ([0032] The user interface 200 also includes a user interface element 220 where the user can identify a column from the dataset including values that should be inferred by the model (that is, that the model should ultimately output as its prediction given input data). The user interface element 220 may provide a list of columns to choose (e.g., after the AMPGS 102 has obtained the dataset identified via element 215 and identified the columns contained therein), allow the user to provide an identifier (e.g., a column name or number) of the column, etc. See also [0039], [0081], [0118-0119]). Refer to claim 3 for the motivation to combine.
Claims 4 is rejected under 35 U.S.C. 103 as being unpatentable over Taher (US 20220066813 A1) in view of Bansal (US 20210097444 A1) and further view of Grayson (US 20230259817 A1).
Regarding claim 4, Taher does not appear to explicitly teach:
The device for determining an execution order according to claim 3, wherein, in the execution order determination processing, the processor is configured to, in a case of training the plurality of provisional models, input an output of at least one provisional model of the provisional models to the provisional model in a subsequent stage.
However, Grayson teaches: [0020] The machine learning pipeline 101 comprises a plurality of pipeline stages 102_n arranged in a pipeline, n=1 . . . N. In other words the pipeline stages 102 are arranged in a sequence from first to last, wherein each but the last stage 102_N in the sequence receives an input from the output of the preceding stage 102_n−1 in the sequence, with the last stage 102_N providing the output of the pipeline as a whole. Each pipeline stage 102 receives a respective input state (input data) and processes this to produce a respective output state (output data). The input state of each successive stage 102_n in the sequence (n=2 . . . N) comprises at least part of the output state of the preceding stage 102_n−1. The first pipeline stage 102_1 receives as its input state the input of the pipeline as a whole (the pipeline input), and the output state of the last pipeline stage 102_N provides the output of the pipeline as a whole (the pipeline output). [0023] In some embodiments each of the pipeline stages 102 is a respective machine learning model. Alternatively however, one or some of the pipeline stages 102 could be other forms of algorithm, e.g. an analytical algorithm such as filter that can remove data from the input should it not meet a desired threshold. [0024] The pipeline output from the last stage 102_N in the pipeline provides the output of the pipeline as a whole (i.e. at the end of the sequence), and this may be considered the primary function of the pipeline. In addition to this, the tool 103 is configured to be able to perform further processing on data from somewhere mid pipeline (in addition to the processing performed by the pipeline 101 to produce the pipeline output, i.e. in addition to the primary function of the pipeline itself).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Taher and Greyson before them, to include Greyson’s machine learning pipeline stage where each successive stage receives part of the output of the previous stage, in Taher’s task ordering pipeline system. One would have been motivated to make such a combination to more efficiently transfer the output of one machine learning stage to the next stage.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Taher (US 20220066813 A1) in view of Grayson (US 20230259817 A1).
Regarding claim 8, Taher does not appear to explicitly teach:
The device for determining an execution order according to claim 1, wherein, in the execution order determination processing, the processor is configured to automatically define an auxiliary task for predicting information that is not determined at a time of performing prediction.
However, Grayson teaches: ([0014] The disclosed approach introduces the idea of a pipeline feature extractor. A feature extractor can be an algorithm or a machine learning model that can perform additional processing on the output state of at least one of the components within the pipeline. In embodiments, it may be able to perform such processing one any of the pipeline component output states, or the pipeline end state, without interrupting the main processing pipeline.[0015] For instance, in the above example, the engineer or domain specialist can use the feature extractor to evaluate the output state of the component that initially detects where people are in the image—in this case, an algorithm to count the detections. The feature extractor will then create a marker for the pipeline input. The marker allows the engineer to identify instantly in any sequence of inputs which input met the criteria of more than three people in the image, allowing them to navigate quickly between the input images to review all of the states. [0016] In some embodiments, a marker created by a feature extractor may just be a pointer to the pipeline input, but in other embodiments it may also contain complex data. For instance in the above example, the engineer may also record the bounding boxes of each of the people in the input image.[0017] In embodiments, the feature extractor can be applied to an existing continuous agent pipeline, without interrupting the pipeline, or altering the output states of any of the pipeline components or the pipeline end state itself. One or more feature extractors can be added to each component, or at the end of the pipeline, and in embodiments they will be invoked every time an output state is ready to be evaluated.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Taher and Greyson before them, to include Greyson’s feature extractor in Taher’s task ordering pipeline system. Greyson teaches a feature extractor to perform additional processing on an output state with interrupting the main pipeline. One would have been motivated to combine the teachings to define an auxiliary processing task that derives additional information while preserving the ordered execution of the main pipeline.
Claims 5 and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Taher (US 20220066813 A1) in view of Bansal (US 20210097444 A1) and further view of Chen (US 20200334520 A1).
Regarding claim 5, Taher does not appear to explicitly teach:
The device for determining an execution order according to claim 3, wherein, in the execution order determination processing, the processor is configured to, in a case of training the plurality of provisional models, input a correct answer label corresponding to an output of at least one provisional model of the provisional models to the provisional model in a subsequent stage.
However, Chen teaches: [0051] To train the relevance ranking layer 702, a fourth training dataset can be obtained with sentences that are labeled to indicate whether they contain correct answers to a given query. The training can proceed in a manner similar to that discussed above, e.g., the error in the relevance ranking output 704 relative to the labels can be used to update certain components of the domain-adapted multi-task model 700. For example, the learned parameters of relevance ranking layer 702, lexicon encoder 304(1), and transformer encoder 304(2) can be updated, whereas the learned parameters of single sentence classification layer 310(1), pairwise text similarity 310(2), and pairwise text classification layer 310(3) may remain unmodified.[0052] The objective for relevance ranking tasks can be based on a pairwise learning-to-rank paradigm. For example, given a query Q, obtain a list of candidate answers A which contains a positive example A.sup.+ that includes the correct answer, and |A|−1 negative examples. Next, minimize the negative log likelihood of the positive example given queries across the training data: See also [0053-0055]
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Chen and Taher before them to include Chen’s neural network and training techniques in in Taher’s task ordering pipeline system. One would have been motivated to make such combination to more efficiently train ordered machine learning task using known neural network training techniques.
Regarding claim 14, Chen teaches:
The learning device according to claim 13, wherein: the plurality of machine learning models are based on a neural network, and in the training processing, the processor is configured to train the plurality of machine learning models while propagating an error of the machine learning model in a subsequent stage to the machine learning model in a previous stage. ([0044] FIG. 4 shows how training can proceed with the first training data set, which can be used to train single-sentence classification layer 310(1). The components of multi-task natural language processing model 300 that are active during training using the first data set are shown in bold in FIG. 4. The first training data set is fed into the shared layers 304, and the context embedding vectors 308 produced from the first training data set are used by single-sentence classification layer 310(1) to produce single-sentence classification output 312(1). Generally, the learned parameters of single-sentence classification layer 312(1), lexicon encoder 304(1), and transformer encoder 304(2) can be updated based on the error in the outputs of the single-sentence classification layer 312(1) relative to the labels for the first training data set. Note that the other task specific layers, pairwise text similarity layer 310(2) and pairwise text classification layer (3), are not updated using the first training data set.[0045] FIG. 5 shows how training can proceed with the second training data set, which can be used to train pairwise text similarity layer 310(2). Again, the components of multi-task natural language processing model 300 that are active during training using the second training data set are shown in bold. The second training data set is fed into the shared layers 304, and the context embedding vectors 308 produced from the second training data set are used by pairwise text similarity layer 310(2) to produce pairwise text similarity layer output 312(2). Generally, the learned parameters of pairwise text similarity layer 312(2), lexicon encoder 304(1), and transformer encoder 304(2) can be updated based on the error in the outputs of the pairwise text similarity layer 312(2) relative to the labels for the second training data set. Note that the other task specific layers, single sentence classification layer 310(1) and pairwise text classification layer (3), are not updated using the second training data set.[0046] FIG. 6 shows how training can proceed with the third training data set, which can be used to train pairwise text classification layer 310(3). Again, the components of multi-task natural language processing model 300 that are active during training using the third training data set are shown in bold. The third training data set is fed into the shared layers 304, and the context embedding vectors 308 produced from the second training data set are used by pairwise text classification layer 310(3) to produce pairwise text classification layer output 312(3). Generally, the learned parameters of pairwise text classification layer 312(3), lexicon encoder 304(1), and transformer encoder 304(2) can be updated based on the error in the outputs of the pairwise text classification layer 312(3) relative to the labels for the third training data set. Note that the other task specific layers, single sentence classification layer 310(1) and pairwise text similarity layer (2), are not updated using the third training data set.) Refer to claim 5 for the motivation to combine.
Regarding claim 15, Chen teaches:
The learning device according to claim 13, wherein: the plurality of machine learning models are based on a neural network, and in the training processing, the processor is configured to train the plurality of machine learning models without propagating an error of the machine learning model in a subsequent stage to the machine learning model in a previous stage. ([0059] Method 800 begins at block 802, where candidate teacher instances of a model can be trained. For example, the candidate teacher instances can be different instances of a multi-task machine learning model such as shown above in FIGS. 1 and/or 3.[0060] Method 800 continues at block 804, where the candidate teacher instances can be evaluated. For instance, each candidate teacher instance can be evaluated using held-out labeled test data to determine the accuracy of each candidate teacher instance for each of the individual tasks.[0061] Method 800 continues at block 806, where one or more teacher instances are selected for each task. For instance, the three highest-ranked (e.g., most accurate) candidate teacher instances for a first task can be identified as the selected teacher instances for the first task. Similarly, the three highest-ranked candidate teacher instances for a second task can be identified as the selected teacher instances for the second task, and so on. Note that some candidate teacher instances trained at block 802 may be selected for multiple tasks, and other candidate teacher instances may not be selected for any tasks. [0062] Method 800 continues at block 808, where task-specific outputs are obtained for each of the selected teacher instances. For example, each selected teacher instance for a given task can be used to process additional labeled data for that task. The selected teacher instances can output different values representing assessments of each labeled data instance, e.g., the probabilities of each possible label. [0063] Method 800 continues at block 810, where a student instance of the model is trained using the outputs of the teacher instances. Generally, the student instance can be trained to produce the outputs of the teacher instances for each selected task. In some cases, this can mean that the student instance is trained not only to generate the same final answer as the teacher instances, but to generate an output distribution that is approximately the same as the teacher instances, as discussed more below.) Refer to claim 5 for the motivation to combine.
Regarding claim 16, Chen teaches:
The learning device according to claim 13, wherein, in the training processing, the processor is configured to train the machine learning model in a subsequent stage after training the machine learning model in a previous stage and confirming the machine learning model. ([0025] Method 200 continues at block 206, where a tuning stage is performed on the shared layers and at least two task-specific layers of the multi-task machine learning model. For example, labeled task-specific data can be obtained for each task performed by the task-specific layers. Then, the one or more shared layers and the task-specific layers can be trained together using the labeled task-specific data, as discussed more below.[0026] Method 200 continues at block 208, where the trained multi-task machine learning model is output. For example, the trained multi-task machine learning model can be finalized and deployed when a convergence condition is reached and/or available training data is exhausted. The trained multi-task machine learning model can be deployed locally for execution in the same processing environment where the model is trained (e.g., on a server), or can be exported to another processing environment for execution (e.g., on a client device, a different server, etc.).[0027] Method 200 continues at block 210, where the tasks are performed using the one or more shared layers and the respective task-specific layers. For example, an application can provide input data to the trained multi-task machine learning model. The trained multi-task machine learning model can process the input data to produce a task-specific result using one of the task-specific layers, and the application can use the task-specific result to perform further processing.) Refer to claim 5 for the motivation to combine.
Regarding claim 17, Chen teaches:
The learning device according to claim 13, wherein: the plurality of machine learning models include a common model, and in the training processing, the processor is configured to combine and change a configuration of the common model. ([0026] Method 200 continues at block 208, where the trained multi-task machine learning model is output. For example, the trained multi-task machine learning model can be finalized and deployed when a convergence condition is reached and/or available training data is exhausted. The trained multi-task machine learning model can be deployed locally for execution in the same processing environment where the model is trained (e.g., on a server), or can be exported to another processing environment for execution (e.g., on a client device, a different server, etc.).[0027] Method 200 continues at block 210, where the tasks are performed using the one or more shared layers and the respective task-specific layers. For example, an application can provide input data to the trained multi-task machine learning model. The trained multi-task machine learning model can process the input data to produce a task-specific result using one of the task-specific layers, and the application can use the task-specific result to perform further processing.[0028] Method 200 continues at block 212, where a domain adaptation process is performed for an additional task. To do so, a new task-specific layer for the additional task can be added to the multi-task machine learning model. Next, the new task-specific layer and the shared layers can be trained using additional training data for the additional task.) Refer to claim 5 for the motivation to combine.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARLOS A ESPANA whose telephone number is (703)756-1069. The examiner can normally be reached Monday - Friday 8 a.m - 5 p.m EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, LEWIS BULLOCK JR can be reached at (571)272-3759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.A.E./Examiner, Art Unit 2199
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199
Prosecution Insights