DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the application filed on 02/13/2024. Claims 1-20 are presented in the case. Claims 1, 12 and 20 are independent claims.
Information Disclosure Statement
The information disclosure statement submitted on 02/14/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-11 are directed to a method, claims 12-19 are directed to an apparatus and claim 20 is directed to a medium. Therefore, the claims are eligible under Step 1 for being directed to a process, a machine and a manufacture respectively.
Independent claims 1, 12 and 20:
Step 2A Prong 1:
Claims recite:
obtaining, by a device, one or more modular natural language processing pipelines each having a respective plurality of selected pipeline stage components - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of selecting data and generating data based on judgement, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper;
processing, by the device, an input text by the one or more modular natural language processing pipelines to produce an output from each of the one or more modular natural language processing pipelines - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of selecting data and generating data based on judgement, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper;
generating, by the device, benchmarking metrics regarding each output processed from each of the one or more modular natural language processing pipelines - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of selecting data and generating data based on judgement, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
providing, by the device, the benchmarking metrics on a visual dashboard interface for assessment of the benchmarking metrics corresponding to each of the one or more modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II).
An apparatus, comprising: one or more network interfaces to communicate with a network; a processor coupled to the one or more network interfaces and configured to execute one or more processes; and a memory configured to store a process that is executable by the processor, the process; A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
providing, by the device, the benchmarking metrics on a visual dashboard interface for assessment of the benchmarking metrics corresponding to each of the one or more modular natural language processing pipelines - which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II).
An apparatus, comprising: one or more network interfaces to communicate with a network; a processor coupled to the one or more network interfaces and configured to execute one or more processes; and a memory configured to store a process that is executable by the processor, the process; A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 2 and 13:
Step 2A Prong 1:
Claims recite:
generating the benchmarking metrics based on performance at each of the respective plurality of selected pipeline stage components - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of selecting data and generating data based on judgement, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
wherein each stage of the respective plurality of selected pipeline stage components of the one or more modular natural language processing pipelines is hosted on a respective microservice - the step recited at a high level of generality, and amounts to more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
wherein each stage of the respective plurality of selected pipeline stage components of the one or more modular natural language processing pipelines is hosted on a respective microservice - the step recited at a high level of generality, and amounts to more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 3 and 14:
Step 2A Prong 1: The claims recite the abstract ideas of claims 1 and 12.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
wherein the benchmarking metrics are selected from a group consisting of: accuracy; execution speed; latency; and throughput - the step recited at a high level of generality, and amounts to selecting a particular data source or type of data to be manipulated, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
wherein the benchmarking metrics are selected from a group consisting of: accuracy; execution speed; latency; and throughput - viewed individually or in combination, describes selecting a particular data source or type of data to be manipulated similar to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display described in MPEP § 2106.05(g).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 4 and 15:
Step 2A Prong 1:
Claims recite:
performing comparative testing between two or more versions of a particular modular natural language processing pipeline, each of the two or more versions having at least one difference in the respective plurality of selected pipeline stage components from other versions of the two or more versions - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data and comparing data, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claims do not provide a practical application and is not considered to be significantly more. As such, the claims are ineligible.
Dependent claim 5:
Step 2A Prong 1: The claim recites the abstract ideas of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
receiving a specified percentage of users to be served by each of the two or more versions - the step recited at a high level of generality amount to mere data gathering which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
receiving a specified percentage of users to be served by each of the two or more versions - the step recited at a high level of generality amount to mere data gathering which is a form of insignificant extra-solution activity which is well known which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 6 and 16:
Step 2A Prong 1: The claims recite the abstract ideas of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
providing a side-by-side visual comparison between benchmarking metrics of two or more differently configured modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
providing a side-by-side visual comparison between benchmarking metrics of two or more differently configured modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claim 7:
Step 2A Prong 1: The claim recites the abstract ideas of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
wherein the side-by-side visual comparison is provided in real-time during simultaneous processing of the two or more differently configured modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
wherein the side-by-side visual comparison is provided in real-time during simultaneous processing of the two or more differently configured modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 8 and 17:
Step 2A Prong 1: The claims recite the abstract ideas of claims 1 and 12.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
providing an interface to receive selection of the respective plurality of selected pipeline stage components for the one or more modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
providing an interface to receive selection of the respective plurality of selected pipeline stage components for the one or more modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 9 and 18:
Step 2A Prong 1: The claims recite the abstract ideas of claims 1 and 12.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
wherein the interface is either a conversational artificial intelligence chatbot, a visual drag-and-drop interface of pipeline building blocks, or both - the step recited at a high level of generality, and amounts to merely indicating a field of use or technological environment in which the judicial exception is performed (see MPEP § 2106.05(h)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
wherein the interface is either a conversational artificial intelligence chatbot, a visual drag-and-drop interface of pipeline building blocks, or both - the step recited at a high level of generality, and amounts to merely indicating a field of use or technological environment in which the judicial exception is performed (see MPEP § 2106.05(h)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claims 10 and 19:
Step 2A Prong 1: The claims recite the abstract ideas of claims 1 and 12.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
providing options to select, configure, place, and connect an abstraction of each component of the respective plurality of selected pipeline stage components to form the one or more modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
providing options to select, configure, place, and connect an abstraction of each component of the respective plurality of selected pipeline stage components to form the one or more modular natural language processing pipelines - the step recited at a high level of generality, and amounts to mere data outputting, which is a well-understood, routine, conventional activity similar to presenting offers and gathering statistics described in MPEP 2106.05(d)(II)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Dependent claim 11:
Step 2A Prong 1: The claim recites the abstract ideas of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
wherein the respective plurality of selected pipeline stage components are based on one or more user-specified components selected from a group consisting of: knowledge bases; text datasets for training the knowledge bases; pre-processing steps; models; post-processing steps; user interface inputs; user interface outputs; and evaluation modules - the step recited at a high level of generality, and amounts to selecting a particular data source or type of data to be manipulated, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
wherein the respective plurality of selected pipeline stage components are based on one or more user-specified components selected from a group consisting of: knowledge bases; text datasets for training the knowledge bases; pre-processing steps; models; post-processing steps; user interface inputs; user interface outputs; and evaluation modules - viewed individually or in combination, describes selecting a particular data source or type of data to be manipulated similar to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display described in MPEP § 2106.05(g).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-8, 11-12, 14-17 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jaeger et al. (hereinafter Jaeger), US 20220092470 A1.
Regarding independent claim 1, Jaeger teaches a method, comprising:
obtaining, by a device, one or more modular natural language processing pipelines each having a respective plurality of selected pipeline stage components ([0049] FIG. 2A depicts a schematic diagram illustrating an example of a data processing pipeline having modular pipeline elements, in accordance with some example embodiments. As shown in FIG. 2A, the data processing pipeline may include different combination of the elements for data preparation, feature engineering, feature selection, model training, ensembling, and/or the like. Each element of the data processing pipeline may be associated with one or more hyper-parameters; [0051] Furthermore, FIG. 2B shows the first operator node 210a, the second operator node 210b, the third operator node 210c, the fourth operator node 210d, and/or the fifth operator node 210e as interconnected via one or more directed edges. A directed edge may indicate a flow of data between the data processing operations corresponding to operator nodes interconnected by the directed edge; [0052] In some example embodiments, a data processing pipeline may be constructed to include one or more specific operator nodes in order to implement a machine learning model trained to perform a cognitive task such as, for example, object identification, natural language processing, information retrieval, speech recognition, classification, and/or regression);
processing, by the device, an input text by the one or more modular natural language processing pipelines to produce an output from each of the one or more modular natural language processing pipelines ([0053] the preparator node 240 may be configured to validate and preprocess an input dataset received, for example, from the client 120. Furthermore, the preparator node 240 may be configured to generate, based at least on the input dataset, a training dataset and a validation dataset. For example, the input dataset may include text associated with one or more errors reported to an issue tracking system. The preparator node 240 may validate the input dataset and terminate additional processing of the input dataset in response to identifying one or more errors present in the input dataset. Upon validating the input dataset, the preparator node 240 may preprocess the input dataset including by removing invalid rows and/or columns of data from the input dataset as well as encoding any text included in the input dataset. The preparator node 240 may partition the validated and preprocessed input dataset into a training dataset for training a machine learning model to perform text classification and a validation dataset for evaluating a performance of the trained machine learning model performing text classification; [0057] Upon receiving, from the preparator node 240, an indication that the preparator node 240 has generated the training dataset and the validation dataset, the orchestrator node 230 may determine a machine learning model including a set of model parameters and hyper-parameters for performing the task associated with the input dataset (e.g., classify text associated with an issue tracking system and/or the like). For example, the orchestrator node 230 may determine a machine learning model including a set of model parameters and hyper-parameters for performing the task associated with the input dataset by at least triggering, at the executor node 280, the execution of one or more machine learning trials, each of which including a different type of machine learning model and/or a different set of trial parameters; [0060] the executor node 280 may execute the first machine learning trial and the second machine learning trial in sequence. However, it should be appreciated that the data processing pipeline 250 may be constructed to include multiple executor nodes and that orchestrator node 230 may coordinate the operations of the multiple executor nodes executing multiple machine learning trials in parallel; [0063] The executor node 280 may further store, in the experimental persistence 300, the results of the machine learning trials corresponding, for example, to the respective performances of the first machine learning model having the first set of trial parameters, the first machine learning model having the second set of trial parameters, and/or the second machine learning model having the third set of trial parameters. In order to identify the machine learning model including the set of model parameters and hyper-parameters for performing the specified task, the orchestrator node 230 may at least access the experimental persistence 300 to evaluate the results of the machine learning trials relative, for example, to the target metric specified by the user 125 as part of the initial configurations for the machine learning model);
generating, by the device, benchmarking metrics regarding each output processed from each of the one or more modular natural language processing pipelines ([0069] FIGS. 3C-D depict examples of the user interface 150, in accordance with some example embodiments. As shown in FIGS. 3C-D, the user interface 150 may be updated to display, at the client 120, a progress as well as a result of the one or more machine learning trials. For example, the user interface 150 may be updated to display, at the client 120, a model accuracy, a calibration curve, a confusion matrix, a significance of each feature (e.g., a relevance of each column in the training dataset for a machine learning model), and/or the like. In the example of the user interface 150 shown in FIG. 3C, the progress and the results associated with multiple types of machine learning models may be sorted in order to identify the one or more machine learning model having a best result. FIG. 3D depicts an example of the user interface 150 displaying the progress and the result of a single type of machine learning model (e.g., an XGBoost Classifier)); and
providing, by the device, the benchmarking metrics on a visual dashboard interface for assessment of the benchmarking metrics corresponding to each of the one or more modular natural language processing pipelines ([0069] FIGS. 3C-D depict examples of the user interface 150, in accordance with some example embodiments. As shown in FIGS. 3C-D, the user interface 150 may be updated to display, at the client 120, a progress as well as a result of the one or more machine learning trials. For example, the user interface 150 may be updated to display, at the client 120, a model accuracy, a calibration curve, a confusion matrix, a significance of each feature (e.g., a relevance of each column in the training dataset for a machine learning model), and/or the like. In the example of the user interface 150 shown in FIG. 3C, the progress and the results associated with multiple types of machine learning models may be sorted in order to identify the one or more machine learning model having a best result. FIG. 3D depicts an example of the user interface 150 displaying the progress and the result of a single type of machine learning model (e.g., an XGBoost Classifier)).
Regarding dependent claim 3, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches wherein the benchmarking metrics are selected from a group consisting of: accuracy; execution speed; latency; and throughput ([0069] FIGS. 3C-D depict examples of the user interface 150, in accordance with some example embodiments. As shown in FIGS. 3C-D, the user interface 150 may be updated to display, at the client 120, a progress as well as a result of the one or more machine learning trials. For example, the user interface 150 may be updated to display, at the client 120, a model accuracy, a calibration curve, a confusion matrix, a significance of each feature (e.g., a relevance of each column in the training dataset for a machine learning model), and/or the like. In the example of the user interface 150 shown in FIG. 3C, the progress and the results associated with multiple types of machine learning models may be sorted in order to identify the one or more machine learning model having a best result. FIG. 3D depicts an example of the user interface 150 displaying the progress and the result of a single type of machine learning model (e.g., an XGBoost Classifier); [0078] Referring again to FIG. 3A, the orchestrator node 230 of the data processing pipeline 250 a may include a runtime estimator 320 as well as an optimizer 330 and a budget counter 340. In some example embodiments, the runtime estimator 320 may be configured to determine a runtime estimate for the data processing pipeline 250 a executing one or more machine learning trials, each of which including a different type of machine learning model and/or a different set of trial parameters. For example, the runtime estimator 320 may define checkpoints for collecting time information in order to avoid inconsistencies introduced by different timing definitions at different cloud computing platforms including, for example, the first cloud computing platform 510 a, the second cloud computing platform 510 b, and/or the like. The runtime estimator 320 may generate a runtime estimate for each individual machine learning trial as well as a runtime estimate for a process including the machine learning trials).
Regarding dependent claim 4, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches further comprising:
performing comparative testing between two or more versions of a particular modular natural language processing pipeline, each of the two or more versions having at least one difference in the respective plurality of selected pipeline stage components from other versions of the two or more versions ([0055] The executor node 280 may be configured to execute a sequence of machine learning trials, each of which including a different type of machine learning model and/or a different set of trial parameters. For instance, the executor node 280 may execute a first machine learning trial by at least applying, to the training dataset and/or the validation dataset, a first machine learning model having a first set of trial parameters. The executor node 280 may also execute a second machine learning trial by at least applying, to the training dataset and/or the validation dataset, the first machine learning model having a second set of trial parameters or a second machine learning model having a third set of trial parameters. As used herein, a “process” may refer to a test process or sequence that includes one or more machine learning trials; [0082] In some example embodiments, the runtime associated with the data processing pipeline 250 may be determined based on timing information associated with test trials performed at a cloud computing platform, for example, the first cloud computing platform 510 a or the second cloud computing platform 510 b, using benchmark datasets covering various use cases and data characteristics (e.g., datasets with many rows or many features, difficult classification task or multiple text columns to preprocess, and/or the like). To ensure reproducible results, the test trials may be performed with pre-defined configurations as well as fixed seeds for the random number generators used during the optimization process. The test trials may be performed upon initial deployment to a cloud computing platform, at scheduled intervals, and/or in response to subsequent changes at the cloud computing platform (e.g., upgrades of the computing resources, modifications to storage system, and/or the like)).
Regarding dependent claim 5, Jaeger teaches all the limitations as set forth in the rejection of claim 4 that is incorporated. Jaeger teaches further comprising:
receiving a specified percentage of users to be served by each of the two or more versions ([0083] Alternatively and/or additionally, the runtime associated with the data processing pipeline 250 may be determined based on timing information collected from user trials. The timing information associated with the user trials may be supplemented by the timing information associated with the test trials when there is insufficient timing information associated with user trials. However, to determine the runtime associated with the data processing pipeline 250, the runtime estimator 320 may be configured to prioritize the timing information associated with the user trials and/or more recent timing information. For example, the runtime estimator 320 may assign a lower weight to the timing information associated with the test trials and/or less recent timing information when determining the runtime of the data processing pipeline 250; [0084] In some example embodiments, the runtime estimator 320 may include one or more runtime estimation models trained, based at least on the timing information associated with the test trials and/or the user trials, to determine a runtime for the process including the one or more individual machine learning trials that are executed by executing the data processing pipeline 250).
Regarding dependent claim 6, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches wherein providing the benchmarking metrics on the visual dashboard interface comprises:
providing a side-by-side visual comparison between benchmarking metrics of two or more differently configured modular natural language processing pipelines (Fig. 3C; [0069] FIGS. 3C-D depict examples of the user interface 150, in accordance with some example embodiments. As shown in FIGS. 3C-D, the user interface 150 may be updated to display, at the client 120, a progress as well as a result of the one or more machine learning trials. For example, the user interface 150 may be updated to display, at the client 120, a model accuracy, a calibration curve, a confusion matrix, a significance of each feature (e.g., a relevance of each column in the training dataset for a machine learning model), and/or the like. In the example of the user interface 150 shown in FIG. 3C, the progress and the results associated with multiple types of machine learning models may be sorted in order to identify the one or more machine learning model having a best result).
Regarding dependent claim 7, Jaeger teaches all the limitations as set forth in the rejection of claim 6 that is incorporated. Jaeger teaches wherein the side-by-side visual comparison is provided in real-time during simultaneous processing of the two or more differently configured modular natural language processing pipelines ([0012] In some variations, the first runtime may be generated based at least on a quantity of the plurality of machine learning models being executed in parallel; [0060] As noted, the executor node 280 may execute the first machine learning trial and the second machine learning trial in sequence. However, it should be appreciated that the data processing pipeline 250 may be constructed to include multiple executor nodes and that orchestrator node 230 may coordinate the operations of the multiple executor nodes executing multiple machine learning trials in parallel; [0069] As shown in FIGS. 3C-D, the user interface 150 may be updated to display, at the client 120, a progress as well as a result of the one or more machine learning trials).
Regarding dependent claim 8, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches further comprising:
providing an interface to receive selection of the respective plurality of selected pipeline stage components for the one or more modular natural language processing pipelines ([0096] At 702, the pipeline engine 110 may generate a user interface configured to receive one or more inputs for constructing of a data processing pipeline for generating a machine learning model. For example, the pipeline engine 110 may generate the user interface 150 which may be configured to display, at the client 120, a selection of operator nodes including, for example, the orchestrator node 230, the preparator node 240, and the executor node 280. The selection of operator nodes displayed, at the client 120, as part of the user interface 150 may also include one or more auxiliary operator nodes including, for example, the start node 260, the user interface node 270, and/or the like. As part of a data processing pipeline, the start node 260 may be configured to receive inputs configuring a process including one or more machine learning trials while the user interface node 270 may be configured to output the progress and/or the result of the one or more machine learning trials. Alternatively, instead of displaying a selection of operator nodes, the user interface 150 may display one or more dialog boxes prompting the user 125 to select one or more operator nodes to include in a data processing pipeline).
Regarding dependent claim 11, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches wherein the respective plurality of selected pipeline stage components are based on one or more user-specified components selected from a group consisting of: knowledge bases; text datasets for training the knowledge bases; pre-processing steps; models; post-processing steps; user interface inputs; user interface outputs; and evaluation modules ([0049] FIG. 2A depicts a schematic diagram illustrating an example of a data processing pipeline having modular pipeline elements, in accordance with some example embodiments. As shown in FIG. 2A, the data processing pipeline may include different combination of the elements for data preparation, feature engineering, feature selection, model training, ensembling, and/or the like. Each element of the data processing pipeline may be associated with one or more hyper-parameters. A machine learning model for performing the task associated with the input dataset may be identified by at least evaluating a performance of the data processing pipeline across different combinations of pipeline elements and hyper-parameters. For example, an executor node may be configured to execute one or more machine learning trials, each of which corresponding to a different combination of pipeline elements and hyper-parameters. Moreover, the orchestrator node may identify, based at least on the performance of the one or more machine learning trials, the machine learning model for performing the task associated with the input dataset; [0052] In some example embodiments, a data processing pipeline may be constructed to include one or more specific operator nodes in order to implement a machine learning model trained to perform a cognitive task such as, for example, object identification, natural language processing, information retrieval, speech recognition, classification, and/or regression. FIG. 2C depicts examples of operator nodes forming a data processing pipeline 250 configured to generate a machine learning model, in accordance with some example embodiments. As shown in FIG. 2C, the data processing pipeline 250 may be constructed to include an orchestrator node 230 and one more preparator nodes such as, for example, a preparator node 240. Furthermore, the data processing pipeline 250 may be constructed to include one or more executor nodes including, for example, an executor node 280. Alternatively and/or additionally, the data processing pipeline 250 may be constructed to include one or more auxiliary operator nodes including, for example, a start node 260, a user interface node 270, and a graph terminator node 290. The start node 260 may receive an initial configuration to generate a machine learning model as specified, for example, by the user 125 at the client 120. Meanwhile, the user interface node 270 may be configured to generate and/or update the user interface 150 to display, at the client 120, a progress of executing the data processing pipeline 200. The graph terminator node 290 may be invoked to terminate the execution of the data processing pipeline 250. It should be appreciated that the data processing pipeline 250 may implement a process in which one or more machine learning trials are executed to generate a machine learning model for performing task).
Regarding independent claim 12, it is an apparatus claim that corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claim 1 above. Jaeger further teaches an apparatus (Fig. 8; [0108]), comprising:
one or more network interfaces to communicate with a network ([0111]);
a processor coupled to the one or more network interfaces and configured to execute one or more processes ([0109]); and
a memory configured to store a process that is executable by the processor ([0110]).
Regarding dependent claim 14, it is an apparatus claim that corresponding to the method of claim 3. Therefore, it is rejected for the same reason as claim 3 above.
Regarding dependent claim 15, it is an apparatus claim that corresponding to the method of claim 4. Therefore, it is rejected for the same reason as claim 4 above.
Regarding dependent claim 16, it is an apparatus claim that corresponding to the method of claim 6. Therefore, it is rejected for the same reason as claim 6 above.
Regarding dependent claim 17, it is an apparatus claim that corresponding to the method of claim 8. Therefore, it is rejected for the same reason as claim 8 above.
Regarding independent claim 20, it is a medium claim that corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claim 1 above. Jaeger further teaches a tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process ([0022]-[0023]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 9-10, 13 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Jaeger as applied in claims 1, 8, 12 and 17, in view of Polleri et al. (hereinafter Polleri), US 20210081819 A1.
Regarding dependent claim 2, Jaeger teaches all the limitations as set forth in the rejection of claim 1 that is incorporated. Jaeger teaches the method further comprising: generating the benchmarking metrics based on performance at each of the respective plurality of selected pipeline stage components ([0049] A machine learning model for performing the task associated with the input dataset may be identified by at least evaluating a performance of the data processing pipeline across different combinations of pipeline elements and hyper-parameters. For example, an executor node may be configured to execute one or more machine learning trials, each of which corresponding to a different combination of pipeline elements and hyper-parameters).
Jaeger does not explicitly teach wherein each stage of the respective plurality of selected pipeline stage components of the one or more modular natural language processing pipelines is hosted on a respective microservice.
However, in the same field of endeavor, Polleri teaches wherein each stage of the respective plurality of selected pipeline stage components of the one or more modular natural language processing pipelines is hosted on a respective microservice ([0045] FIG. 1 is a block diagram illustrating an exemplary machine learning platform 100 for generating a machine learning model. The machine learning platform 100 has various components that can be distributed between different networks and computing systems; [0049] The machine learning platform 100 can generate highly customizable applications. The library components 168 contain a set of predefined, off-the-shelf workflows or pipelines 136, which the application developer can incorporate into a new machine learning application 112. A workflow specifies various micro services routines 140, software modules 144 and/or infrastructure modules 148 configured in a particular way for a type or class of problem. In addition to this, it is also possible to define new workflows or pipelines 136 by re-using the library components or changing an existing workflow or pipeline 136; [0056] Micro services routines 140 can be used in an architectural approach to building applications. As an architectural framework, micro services are distributed and loosely coupled, to allow for changes to one aspect of an application without destroying the entire application. The benefit to using micro services is that development teams can rapidly build new components of applications to meet changing development requirements. Micro service architecture breaks an application down into its core functions. Each function is called a service, and can be built and deployed independently, meaning individual services can function (and fail) without negatively affecting the others. A micro service can be a core function of an application that runs independent of other services. By storing various micro service routines 140, the machine learning platform 100 can generate a machine learning application incrementally by identifying and selecting various different components from the library components 168).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of using micro services in an architectural approach to building applications as suggested in Polleri into Jaeger’s system because both of these systems are addressing generating and managing a library of machine learning applications. This modification would have been motivated by the desire to be able to rapidly build new components of applications to meet changing development requirements (Polleri, [0056]).
Regarding dependent claim 9, Jaeger teaches all the limitations as set forth in the rejection of claim 8 that is incorporated. Jaeger does not explicitly teach wherein the interface is either a conversational artificial intelligence chatbot, a visual drag-and-drop interface of pipeline building blocks, or both.
However, in the same field of endeavor, Polleri teaches wherein the interface is either a conversational artificial intelligence chatbot, a visual drag-and-drop interface of pipeline building blocks, or both (Fig. 1; [0050] A model composition engine 132 can be executed on one or more computing systems (e.g., infrastructure 128). The model composition engine 132 can receive inputs from a user 116 through an interface 104. The interface 104 can include various graphical user interfaces with various menus and user selectable elements. The interface 104 can include a chatbot (e.g., a text based or voice based interface). The user 116 can interact with the interface 104 to identify one or more of: a location of data, a desired prediction of machine learning application, and various performance metrics for the machine learning model. The model composition engine 132 can interface with library components 168 to identify various pipelines 136, micro service routines 140, software modules 144, and infrastructure models 148 that can be used in the creation of the machine learning model 112).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of providing an intuitive interface to allow the user to generate a machine learning application as suggested in Polleri into Jaeger’s system because both of these systems are addressing generating and managing a library of machine learning applications. This modification would have been motivated by the desire to provide intuitive interfaces assisting user without considerable programming experience constructing a machine learning application through a series of queries (Polleri, [0005]; [0011]).
Regarding dependent claim 10, Jaeger teaches all the limitations as set forth in the rejection of claim 8 that is incorporated. Jaeger does not explicitly teach wherein providing the interface comprises:
providing options to select, configure, place, and connect an abstraction of each component of the respective plurality of selected pipeline stage components to form the one or more modular natural language processing pipelines.
However, in the same field of endeavor, Polleri teaches wherein providing the interface comprises:
providing options to select, configure, place, and connect an abstraction of each component of the respective plurality of selected pipeline stage components to form the one or more modular natural language processing pipelines ([0050] A model composition engine 132 can be executed on one or more computing systems (e.g., infrastructure 128). The model composition engine 132 can receive inputs from a user 116 through an interface 104. The interface 104 can include various graphical user interfaces with various menus and user selectable elements. The interface 104 can include a chatbot (e.g., a text based or voice based interface). The user 116 can interact with the interface 104 to identify one or more of: a location of data, a desired prediction of machine learning application, and various performance metrics for the machine learning model. The model composition engine 132 can interface with library components 168 to identify various pipelines 136, micro service routines 140, software modules 144, and infrastructure models 148 that can be used in the creation of the machine learning model 112).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of providing an intuitive interface to allow the user to generate a machine learning application as suggested in Polleri into Jaeger’s system because both of these systems are addressing generating and managing a library of machine learning applications. This modification would have been motivated by the desire to provide intuitive interfaces assisting user without considerable programming experience constructing a machine learning application through a series of queries (Polleri, [0005]; [0011]).
Regarding dependent claim 13, it is an apparatus claim that corresponding to the method of claim 2. Therefore, it is rejected for the same reason as claim 2 above.
Regarding dependent claim 18, it is an apparatus claim that corresponding to the method of claim 9. Therefore, it is rejected for the same reason as claim 9 above.
Regarding dependent claim 19, it is an apparatus claim that corresponding to the method of claim 10. Therefore, it is rejected for the same reason as claim 10 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
ZHOU (US 20250181426 A1) discloses generating data processing pipelines.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY P HOANG whose telephone number is (469)295-9134. The examiner can normally be reached M-TH 8:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER WELCH can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMY P HOANG/ Examiner, Art Unit 2143
/JENNIFER N WELCH/ Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights