Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This Office Action is sent in response to Applicant’s Communication received on 07/27/2026 for application number 17/773,100.
Response to Amendments
3. The Amendment filed 07/27/2026 has been entered. Claims 1 and 16-18 have been amended. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant argues that Zeiler fails to disclose that a user may select the training data to train the AI agents. Accordingly, Zeiler fails to disclose "receiving, via the workflow editor interface, a first user selection of training data for the first AI agent based on the first data source; receiving, via the workflow editor interface, a second user selection of training data for the second AI agent based on the second data source," as recited in amended independent claim 1. Examiner respectfully disagrees and notes that although Zeiler para. [0032] describes model subsystem 114 selecting training information, Zeiler elsewhere expressly teaches that the user interface enables users to link “training or other data that is to be provided as input to a model” with selected models. Zeiler further teaches user selection of input/output representations through the workflow interface and user configuration of model inputs and outputs. Zeiler also teaches that training data may include input provided to a machine learning model, and that first model 304 learns from its inputs while second model 306 may learn from the output of the first model 304 provided as its input. Accordingly, Zeiler teaches the above claim features.
Applicant argues that cited references do not teach all of the features of the amended independent claim 16. However, the argument is moot since this is a newly presented limitation, thus changes the scope of the claim. However, a newly found reference, Singhal, is applied.
Claim Rejections – 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 1-10 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Zeiler et al. (U.S. Patent Application Pub. No. US 20180089592 A1) in view of Gray et al. (U.S. Patent Application Pub. No. US 20170017903 A1).
Claim 1: Zeiler teaches a method comprising:
receiving, via a workflow editor interface (i.e. one or more service platforms described herein may provide one or more user interfaces that present one or more options to select/access the model representations. Responsive to user selection of such options, the selected model representations may be presented on the user interface. As an example, the model representations may include a first prediction model representation corresponding to a first prediction model, a second prediction model representation corresponding to a second prediction model; para. [0005, 0006, 0034]), a user selection of a first artificial intelligence (Al) agent for a workflow (i.e. fig. 3A, service interface subsystem 112 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. As an example, one or more options to select/access the representations may be presented on the user interface. Responsive to user selection of such options, the corresponding representations may be presented on the user interface. The model representations may include a first machine learning model representation corresponding to a first machine learning model, a second machine learning model representation corresponding to a second machine learning model; para. [0037, 0041, 0042]);
receiving, via the workflow editor interface, a user selection of a second Al agent (i.e. fig. 3A, service interface subsystem 112 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. As an example, one or more options to select/access the representations may be presented on the user interface. Responsive to user selection of such options, the corresponding representations may be presented on the user interface. The model representations may include a first machine learning model representation corresponding to a first machine learning model, a second machine learning model representation corresponding to a second machine learning model; para. [0037, 0041, 0042]);
receiving, via the workflow editor interface, a selection of a first data source to input to the first Al agent (i.e. FIG. 3A, user interface 300 of a service platform displays representations of components related to functionalities to be provided via an application. For example, such components include machine learning models 302, 304, 306, and 308, non-machine-learning models 310 and 312, designated input sources (or type) 314, designated output destinations (or type) 316, 318, and 320, and input/output paths 322; para. [0035, 0036, 0037]);
receiving, via the workflow editor interface, a selection of a second data source to input to the second Al agent, wherein the second data source to input to the second Al agent comprises data output by the first Al agent (i.e Based on the arrangement of machine learning models 304 and 306 in the workflow shown in FIG. 3C, the output of machine learning model 304 is provided as input to machine learning model 306; para. [0048, 0055]);
receiving, via the workflow editor interface, a first user selection of training data for the first Al agent based on the first data source (i.e. a user interface of the service platform may enable users to link different models, versions of the same model, training or other data that is to be provided as input to a model, or other items to one another … Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model … providing inputs to machine learning model 304, which may learn based on the provided inputs and its processing of the provided inputs; para. [0028, 0047, 0048]); receiving, via the workflow editor interface, a second user selection of training data for the second Al agent based on the second data source (i.e. Based on the arrangement of machine learning models 304 and 306 in the workflow shown in FIG. 3C, the output of machine learning model 304 is provided as input to machine learning model 306, which may also learn based on the output of machine learning model 304 and machine learning model 306's processing of such output … FIG. 3C, feedback subsystem 116 may obtain input/output information (e.g., training data 332), and model subsystem 114 may provide the input/output information as training data to one or more machine learning models (or instances thereof) represented in user interface 300. The input/output information may include (i) information related to first items to be provided as input to the machine learning models (or previously provided as input to one or more other machine learning models), (ii) information related to first prediction outputs (or other outputs) derived from the other machine learning models' processing of the first items); para. [0047-0049]);
training, based on the training data, the first Al agent and the second Al agent (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed. In some embodiments, the machine learning models of the workflow may be trained/re-trained/further trained subsequent to being added to the workflow and subsequent to being incorporated as at least part of the application. Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model (e.g., inputs provided to and processed by other machine learning models or other inputs), (ii) reference outputs that are to be derived from a machine learning model's processing of such inputs (e.g., user-confirmed or user-provided outputs, outputs confirmed through one or more machine learning models' processing of such inputs, outputs confirmed multiple times by processing of such inputs by respective sets of machine learning models, or other reference outputs); para. [0047-0049]);
after the first Al agent and the second Al agent are trained (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed; para. [0047]), activating (i.e. a single call to a workflow may cause multiple machine learning models of the workflow to be executed; para. [0037]) the first Al agent and the second Al agent to perform operations based on the selection of the first data source and the selection of the second data source (i.e. system 100 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components. In one use case, suppose a user wishes to use an API service's general image classification model and face detection model to find scenes containing a certain number of people in outdoor settings in a stream of images from a social media service; para. [0036, 0047-0049]);
receiving, via the workflow editor interface, an edit to the workflow, wherein the edit is associated with updating at least one of the first Al agent or the second Al agent (i.e. the service platform may facilitate collaboration among multiple users by enabling multiple users to supplement or otherwise modify the same workflow, including (i) adding prediction models or other processing operations to a workflow, (ii) removing prediction models or other processing operations from the workflow, (iii) commenting on portions of the workflow (e.g., commenting on a particular prediction model or other processing operation, commenting on a group of prediction models and other processing operations, etc.), (iv) setting inputs and outputs for prediction models and other processing operations of the workflow, or (v) other operations; para. [0035]); and
updating an updated version of the first set of key performance indicators and the second set of key performance indicators (i.e. model subsystem 114 may update the second instance based on changes for the machine learning model (e.g., code changes, architecture changes, training parameter changes, data changes, etc., provided by users of the service platform); para. [0029]).
Zeiler does not explicitly teach generating a first set of key performance indicators associated with a first operation performance of the first agent based on a first data output of the first agent; generating a second set of key performance indicators associated with a second operation performance of the second agent based on a second data output of the second agent; generating for display a dashboard interface indicating the first set of key performance indicators for the first agent and the second set of key performance indicators for the second agent simultaneously; receiving, via the editor interface, an edit to the workflow, wherein the edit is associated with updating at least one of the first agent or the second agent; and updating the dashboard interface to display an updated version of the first set of key performance indicators and the second set of key performance indicators.
However, Gray teaches generating a first set of key performance indicators associated with a first operation performance of the first Al agent based on a first data output of the first Al agent (i.e. The performance can be measured by and optimized using one or more measures of fitness. The one or more measures of fitness used may vary based on the specific goal of a project. Examples of potential measures of fitness include, but are not limited to, error rate, F-score, area under curve (AUC), Gini, precision, performance stability, time cost, etc … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0068, 0090, 0098]);
generating a second set of key performance indicators associated with a second operation performance of the second Al agent based on a second data output of the second Al agent (i.e. The performance can be measured by and optimized using one or more measures of fitness. The one or more measures of fitness used may vary based on the specific goal of a project. Examples of potential measures of fitness include, but are not limited to, error rate, F-score, area under curve (AUC), Gini, precision, performance stability, time cost, etc … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0068, 0090, 0098]);
generating for display a dashboard interface indicating the first set of key performance indicators for the first Al agent and the second set of key performance indicators for the second Al agent simultaneously (i.e. generate a user interface for displaying a scoreboard of the models, or experiments involving models … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object. Examples of multiple cards in a dashboard include a machine learning/data science scoreboard, a workflow diagram, and a machine learning/data science checklist as shown in FIG. 3 … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0071, 0090, 0098]);
receiving, via the workflow editor interface, an edit to the workflow (i.e. The model management module 255 monitors the building and exporting of the workflow and sends data to the auditing module 260 for building an audit trail changes that have transpired in the building and exporting of the workflow … the user moves a card from the historical area 308 into the main workspace area 304 which reproduces the information represented by the card so that e.g. the information may be modified or a process (e.g. transformation, plot generating, etc. represented by the card) may be run again within the user interface 300 on another or the same machine learning object; para. [0070, 0088, 0095]), wherein the edit is associated with updating at least one of the first AI agent or the second AI agent (i.e. receive, from the user, a confirmation to perform the first action; and manipulate one or more of the first machine learning object and the second learning object related to the first machine learning object in the first context based on the first action; para. [0009, 0097]); and
updating the dashboard interface to display an updated version of the first set of key performance indicators and the second set of key performance indicators (i.e. the user may move one of the cards from the palette area 310 or historical area 308 into the dashboard area 306, which makes the moved card live-updating within the user interface 300 … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object. Any card from other screen areas can be placed into the dashboard area 306 for visualizing a dynamic and live-updating of such a card; para. [0083, 0088, 0090, 0098]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Zeiler to include the feature of Gray. One would have been motivated to make this modification because it improves monitoring and management of the workflow models by simultaneously presenting model specific performance information and dynamically updating such information as the underlying model data changes.
Claim 2: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising receiving a user selection to add an additional input source to the first Al agent (i.e. The input/output representations may include (i) connection components corresponding to one or more input/output data paths between the machine learning models or the non-machine-learning models, (ii) connection components corresponding to input sources (e.g., the application that incorporates such models, third party input sources, etc.), (iii) connection components corresponding to output sources (e.g., the application that incorporates such models, third party output sources, etc.), (iv) representations corresponding to input or output data types compatible with such models, or (v) other representations; para. [0041, 0044]).
Claim 3: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising receiving a user selection to retrain the first Al agent or the second Al agent (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed. In some embodiments, the machine learning models of the workflow may be trained/re-trained/further trained subsequent to being added to the workflow and subsequent to being incorporated as at least part of the application. Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model (e.g., inputs provided to and processed by other machine learning models or other inputs), (ii) reference outputs that are to be derived from a machine learning model's processing of such inputs (e.g., user-confirmed or user-provided outputs, outputs confirmed through one or more machine learning models' processing of such inputs, outputs confirmed multiple times by processing of such inputs by respective sets of machine learning models, or other reference outputs); para. [0047]).
Claim 4: Zeiler and Gray teach the method of claim 1. Zeiler further teaches wherein the workflow editor interface provides control to a user on input data provided to the first Al agent or the second Al agent (i.e. The input/output representations may include (i) connection components corresponding to one or more input/output data paths between the machine learning models or the non-machine-learning models, (ii) connection components corresponding to input sources (e.g., the application that incorporates such models, third party input sources, etc.), (iii) connection components corresponding to output sources (e.g., the application that incorporates such models, third party output sources, etc.), (iv) representations corresponding to input or output data types compatible with such models, or (v) other representations; para. [0041, 0044]).
Claim 5: Zeiler and Gray teach the method of claim 4. Zeiler further teaches wherein the workflow editor interface provides control to the user on data exchanged between the first Al agent and the second Al agent (i.e. system 100 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components. In one use case, suppose a user wishes to use an API service's general image classification model and face detection model to find scenes containing a certain number of people in outdoor settings in a stream of images from a social media service; para. [0036]).
Claim 6: Zeiler and Gray teach the method of claim 1. Zeiler further teaches wherein the first Al agent and the second Al agent are operated in series (i.e. FIG. 3A, user interface 300 of a service platform displays representations of components related to functionalities to be provided via an application. For example, such components include machine learning models 302, 304, 306, and 308, non-machine-learning models 310 and 312, designated input sources (or type) 314, designated output destinations (or type) 316, 318, and 320, and input/output paths 322; para. [0036, 0037]).
Claim 7: Zeiler and Gray teach the method of claim 1. Zeiler further teaches wherein the first Al agent and the second Al agent form an operating environment, and further comprising updating the first Al agent without interrupting operations of the operating environment (i.e. Model subsystem 114 may cause updating of the application's instance of the first machine learning model by replacing, for the application, the first instance of the first machine learning model with the second instance of the first machine learning model. In this way, for example, the application may continue to operate simultaneously (e.g., using the first instance of the first machine learning model) while the second instance of the first machine learning model is updated, thereby reducing downtime in the application's operations; para. [0054, 0058]).
Claim 8: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising: receiving, via the workflow editor interface, a user selection of a third Al agent (i.e. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components; para. [0034, 0036]); receiving, via the workflow editor interface, a selection of data to input to the third Al agent, wherein the data to input to the third Al agent comprises the data output by the first Al agent (i.e. The single call may cause a first machine learning model of the workflow to process one or more inputs and provide its outputs as inputs to one or more other machine learning models or non-machine-learning models, and cause those other machine learning models or non-machine-learning models to process the first machine learning model's outputs and provide their outputs to other components of the workflow or to the application (e.g., via the API client or other component of the application); para. [0036, 0037]); and executing the third Al agent in parallel with the second Al agent (i.e. a single call to a workflow may cause multiple machine learning models of the workflow to be executed. The single call may cause a first machine learning model of the workflow to process one or more inputs and provide its outputs as inputs to one or more other machine learning models or non-machine-learning models, and cause those other machine learning models or non-machine-learning models to process the first machine learning model's outputs and provide their outputs to other components of the workflow or to the application (e.g., via the API client or other component of the application); para. [0037, 0038]).
Claim 9: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising: receiving, via the workflow editor interface, a user selection of a user input node (i.e. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application; para. [0035, 0036]); receiving, via the workflow editor interface, a selection of data to input to the user input node, wherein the data to input to the use input node comprises the data output by the first Al agent (i.e. a user may download and integrate an API client into the user's application. In some embodiments, the application may utilize the API client to make calls to one or more workflows created via system 100's service platform or user interface thereof. As an example, a single call to a workflow may cause multiple machine learning models of the workflow to be executed. The single call may cause a first machine learning model of the workflow to process one or more inputs and provide its outputs as inputs to one or more other machine learning models or non-machine-learning models, and cause those other machine learning models or non-machine-learning models to process the first machine learning model's outputs and provide their outputs to other components of the workflow or to the application (e.g., via the API client or other component of the application); para. [0037]); and receiving a selection of a user input to display when the user input node is activated (i.e. building blocks of a workflow may include a set of operation blocks for processing inputs and creating outputs to be passed into other stages of the workflow. Some examples of these operation blocks include (i) machine learning models, (ii) conditional statements to decide whether an output should pass to a next stage or which block to pass the output, (iii) mathematical/statistical models, (iv) image/video/audio/text/other data processing operation blocks, (v) operation blocks to initiate requests to third party APIs or programs, (vi) user-interface-related operation blocks (e.g., which cause one or more interactive features of a user interface), (vii) notification operation blocks (e.g., which cause one or more notifications to be sent or presented); para. [0038]).
Claim 10: Zeiler and Gray teach the method of claim 9. Zeiler further teaches wherein the first Al agent outputs a confidence score (i.e. building blocks of a workflow may include a set of operation blocks for processing inputs and creating outputs to be passed into other stages of the workflow. Some examples of these operation blocks include (i) machine learning models, (ii) conditional statements to decide whether an output should pass to a next stage or which block to pass the output; para. [0038, 0040]), and further comprising: in response to the confidence score satisfing a threshold, activating the second Al agent (i.e. building blocks of a workflow may include a set of operation blocks for processing inputs and creating outputs to be passed into other stages of the workflow. Some examples of these operation blocks include (i) machine learning models, (ii) conditional statements to decide whether an output should pass to a next stage or which block to pass the output; para. [0038, 0040]); and in response the confidence score failing to satisfy the threshold, activating the user input node thereby requesting user input (i.e If the moderation model outputs a score based on its processing of the input image, and the threshold score comparison model determines that the score does not satisfy a threshold score (e.g., not equal to or greater than 0.9 or other threshold), the threshold score comparison model may cause the input image to be added to the confirmation moderation queue; para. [0038, 0040]).
Claim 13: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising: receiving an updated Al agent corresponding to the first Al agent; and replacing the first Al agent with the updated Al agent (i.e. model subsystem 114 may provide the given input and the reference indication to a second instance of the machine learning model (different from the first instance of the machine learning model). Model subsystem 114 may cause updating of the application's instance of the machine learning model by replacing, for the application, the first instance of the machine learning model with the second instance of the machine learning model. In this way, for example, the application may continue to operate simultaneously (e.g., using the first instance of the machine learning model) while the second instance of the machine learning model is being updated by training the second instance on training data (e.g., the given input, the reference indication, or other training data); para. [0054]).
Claim 14: Zeiler and Gray teach the method of claim 1. Zeiler further teaches comprising, after receiving the selection of data to input to the second Al agent (i.e. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application; para. [0035, 0036]), determining that a format of the data output by the first Al agent is compatible with a format of the data to input to the second Al agent (i.e. a user interface of system 100 may guide the developer to select and connect operation blocks that are compatible with one another, thereby reducing errors resulting from application component incompatibility. The user interface (or its associated service platform) may provide a developer with a set of compatible options when the developer selects an operation block that he/she wants to input to, output to, or otherwise extend. Additionally, or alternatively, the user interface may prevent a developer from connecting two operation blocks that are not compatible with one another. Compatibility may, for example, be determined based on input/output data types of the operation blocks. As an example, if a developer tries to add a block that takes as input the output of a previous block, the blocks may be determined to be compatible responsive to the input data type/format of that block and the output data type/format of the previous block being of the same data type/format. That simple check, in addition to other checks for constraints (e.g., such as where the processing of each block is happening or other checks), facilitates workflow operations; para. [0039]).
6. Claims 16-20 rejected under 35 U.S.C. 103 as being unpatentable over Zeiler in view of Gray, and further in view of Singhal et al. (U.S. Patent Application Pub. No. US 20190156124 A1).
Claim 16: Zeiler teaches a system comprising:
at least one processor (i.e. processors; para. [0083]), and
memory storing executable instructions that, when executed by the at least one processor, cause the system to (i.e. one or more processors; and memory storing instructions that when executed by the processors cause the processors to effectuate operations; para. [0115]):
generate a workflow editor interface for editing a workflow (i.e. a graph of arbitrary data workflow methods that incorporate artificial intelligence blocks with other processing operations may be built via the service platform's user interface to meet an entire application or enterprise need. Such workflows may be configured via a user interface and a corresponding API to enable developers to embed features of prediction models in their applications (e.g., as API calls to the prediction models to provide input to and obtain outputs from the prediction models or workflow incorporating the prediction models); para. [0035]);
receive, via the workflow editor interface (i.e. one or more service platforms described herein may provide one or more user interfaces that present one or more options to select/access the model representations. Responsive to user selection of such options, the selected model representations may be presented on the user interface. As an example, the model representations may include a first prediction model representation corresponding to a first prediction model, a second prediction model representation corresponding to a second prediction model; para. [0005, 0006, 0034]), a user selection of a first artificial intelligence (Al) agent for the workflow (i.e. fig. 3A, service interface subsystem 112 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. As an example, one or more options to select/access the representations may be presented on the user interface. Responsive to user selection of such options, the corresponding representations may be presented on the user interface. The model representations may include a first machine learning model representation corresponding to a first machine learning model, a second machine learning model representation corresponding to a second machine learning model; para. [0037, 0041, 0042]);
receive, via the workflow editor interface, a user selection of a second Al agent for the workflow (i.e. fig. 3A, service interface subsystem 112 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. As an example, one or more options to select/access the representations may be presented on the user interface. Responsive to user selection of such options, the corresponding representations may be presented on the user interface. The model representations may include a first machine learning model representation corresponding to a first machine learning model, a second machine learning model representation corresponding to a second machine learning model; para. [0037, 0041, 0042]);
receive, via the workflow editor interface, a selection of data source to input to the first Al agent (i.e. FIG. 3A, user interface 300 of a service platform displays representations of components related to functionalities to be provided via an application. For example, such components include machine learning models 302, 304, 306, and 308, non-machine-learning models 310 and 312, designated input sources (or type) 314, designated output destinations (or type) 316, 318, and 320, and input/output paths 322; para. [0035, 0036, 0037]);
receive, via the workflow editor interface, a selection of data to input to the second Al agent, wherein the data to input to the second Al agent comprises data output by the first Al agent (i.e Based on the arrangement of machine learning models 304 and 306 in the workflow shown in FIG. 3C, the output of machine learning model 304 is provided as input to machine learning model 306; para. [0048, 0055]);
receive, via the workflow editor interface, a selection of training data for the first Al agent (i.e. a user interface of the service platform may enable users to link different models, versions of the same model, training or other data that is to be provided as input to a model, or other items to one another … Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model … providing inputs to machine learning model 304, which may learn based on the provided inputs and its processing of the provided inputs; para. [0028, 0047, 0048]) and second Al agent (i.e. Based on the arrangement of machine learning models 304 and 306 in the workflow shown in FIG. 3C, the output of machine learning model 304 is provided as input to machine learning model 306, which may also learn based on the output of machine learning model 304 and machine learning model 306's processing of such output … FIG. 3C, feedback subsystem 116 may obtain input/output information (e.g., training data 332), and model subsystem 114 may provide the input/output information as training data to one or more machine learning models (or instances thereof) represented in user interface 300. The input/output information may include (i) information related to first items to be provided as input to the machine learning models (or previously provided as input to one or more other machine learning models), (ii) information related to first prediction outputs (or other outputs) derived from the other machine learning models' processing of the first items); para. [0047-0049]);
train, based on the training data, the first Al agent and the second Al agent (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed. In some embodiments, the machine learning models of the workflow may be trained/re-trained/further trained subsequent to being added to the workflow and subsequent to being incorporated as at least part of the application. Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model (e.g., inputs provided to and processed by other machine learning models or other inputs), (ii) reference outputs that are to be derived from a machine learning model's processing of such inputs (e.g., user-confirmed or user-provided outputs, outputs confirmed through one or more machine learning models' processing of such inputs, outputs confirmed multiple times by processing of such inputs by respective sets of machine learning models, or other reference outputs); para. [0047-0049]);
after the first Al agent and the second Al agent are trained (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed; para. [0047]), activate (i.e. a single call to a workflow may cause multiple machine learning models of the workflow to be executed; para. [0037]) the first Al agent and the second Al agent to perform operations for the workflow based on the selection of the data source and the selection of data (i.e. system 100 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components. In one use case, suppose a user wishes to use an API service's general image classification model and face detection model to find scenes containing a certain number of people in outdoor settings in a stream of images from a social media service; para. [0036, 0047-0049]);
receive, via the workflow editor interface, an edit to the workflow, wherein the edit is associated with updating the first Al agent (i.e. the service platform may facilitate collaboration among multiple users by enabling multiple users to supplement or otherwise modify the same workflow, including (i) adding prediction models or other processing operations to a workflow, (ii) removing prediction models or other processing operations from the workflow, (iii) commenting on portions of the workflow (e.g., commenting on a particular prediction model or other processing operation, commenting on a group of prediction models and other processing operations, etc.), (iv) setting inputs and outputs for prediction models and other processing operations of the workflow, or (v) other operations; para. [0035]);
replace the first AI agent with an updated AI agent (i.e. one or more machine learning models may replace the first instance of the machine learning model with the second instance of the machine learning model at the service platform; para. [0031, 0082]);
update an updated version of the first set of key performance indicators (i.e. model subsystem 114 may update the second instance based on changes for the machine learning model (e.g., code changes, architecture changes, training parameter changes, data changes, etc., provided by users of the service platform); para. [0029]).
Zeiler does not explicitly teach generate a first set of key performance indicators associated with a first operation performance of the first agent based on a first data output of the first agent; generate a second set of key performance indicators associated with a second operation performance of the second agent based on a second data output of the second agent; generate for display a dashboard interface indicating the first set of key performance indicators for the first agent and the second set of key performance indicators for the second agent simultaneously; receive, via the editor interface, an edit to the workflow, wherein the edit is associated with updating the first agent; provide a list of uncompleted commands of the first AI agent to the updated AI agent for executing; and update the dashboard interface to display an updated version of the first set of key performance indicators.
However, Gray teaches generate a first set of key performance indicators associated with a first operation performance of the first Al agent based on a first data output of the first Al agent (i.e. The performance can be measured by and optimized using one or more measures of fitness. The one or more measures of fitness used may vary based on the specific goal of a project. Examples of potential measures of fitness include, but are not limited to, error rate, F-score, area under curve (AUC), Gini, precision, performance stability, time cost, etc … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0068, 0090, 0098]);
generate a second set of key performance indicators associated with a second operation performance of the second Al agent based on a second data output of the second Al agent (i.e. The performance can be measured by and optimized using one or more measures of fitness. The one or more measures of fitness used may vary based on the specific goal of a project. Examples of potential measures of fitness include, but are not limited to, error rate, F-score, area under curve (AUC), Gini, precision, performance stability, time cost, etc … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0068, 0090, 0098]);
generate for display a dashboard interface indicating the first set of key performance indicators for the first Al agent and the second set of key performance indicators for the second Al agent simultaneously (i.e. generate a user interface for displaying a scoreboard of the models, or experiments involving models … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object. Examples of multiple cards in a dashboard include a machine learning/data science scoreboard, a workflow diagram, and a machine learning/data science checklist as shown in FIG. 3 … each row represents a model, and columns include one or more measures of model quality or other information about the model. Examples of model quality may include, but is not limited to, predictive accuracy, size, training time, scoring time, etc; para. [0071, 0090, 0098]);
receive, via the workflow editor interface, an edit to the workflow (i.e. The model management module 255 monitors the building and exporting of the workflow and sends data to the auditing module 260 for building an audit trail changes that have transpired in the building and exporting of the workflow … the user moves a card from the historical area 308 into the main workspace area 304 which reproduces the information represented by the card so that e.g. the information may be modified or a process (e.g. transformation, plot generating, etc. represented by the card) may be run again within the user interface 300 on another or the same machine learning object; para. [0070, 0088, 0095]), wherein the edit is associated with updating the first AI agent (i.e. receive, from the user, a confirmation to perform the first action; and manipulate one or more of the first machine learning object and the second learning object related to the first machine learning object in the first context based on the first action; para. [0009, 0097]); and
update the dashboard interface to display an updated version of the first set of key performance indicators (i.e. the user may move one of the cards from the palette area 310 or historical area 308 into the dashboard area 306, which makes the moved card live-updating within the user interface 300 … the dashboard card 306 may provide an at-a-glance view of one or more key performance indicators relevant to the context of the machine learning object. Any card from other screen areas can be placed into the dashboard area 306 for visualizing a dynamic and live-updating of such a card; para. [0083, 0088, 0090, 0098]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Zeiler to include the feature of Gray. One would have been motivated to make this modification because it improves monitoring and management of the workflow models by simultaneously presenting model specific performance information and dynamically updating such information as the underlying model data changes.
However, Singhal teaches provide a list of uncompleted commands of the first AI agent to the updated AI agent for executing (i.e. the worker 811 and services 813 and 815 are virtual machines executing a machine learning model. In other embodiments, the worker 811 and services 813 and 815 are containers executing a machine learning model … When a worker fails to heartbeat (deployments/failures, etc.), another worker picks up the request from the queue, acquires the database lock and starts processing the stream from the previous checkpoint; para. [0075, 0077, 0078]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Zeiler and Gray to include the feature of Singhal. One would have been motivated to make this modification because model replacement would preserve outstanding work during replacement, avoid loss or unnecessary restarting of previously assigned processing and maintain continuity of workflow execution.
Claim 17: Zeiler, Gray, and Singhal teach the system of claim 16. Zeiler further teaches comprising: receiving a particular updated Al agent corresponding to the second Al agent; and replacing the second Al agent with the particular updated Al agent (i.e. model subsystem 114 may provide the given input and the reference indication to a second instance of the machine learning model (different from the first instance of the machine learning model). Model subsystem 114 may cause updating of the application's instance of the machine learning model by replacing, for the application, the first instance of the machine learning model with the second instance of the machine learning model. In this way, for example, the application may continue to operate simultaneously (e.g., using the first instance of the machine learning model) while the second instance of the machine learning model is being updated by training the second instance on training data (e.g., the given input, the reference indication, or other training data); para. [0054]).
Claim 18: Zeiler, Gray, and Singhal teach the system of claim 16. Zeiler further teaches wherein the instructions further cause the system to: input a first portion of data from the data source to the first Al agent; and input a second portion of data from the data source to the updated Al agent (i.e. model subsystem 114 may provide the first input and the reference indication to the first instance of the second machine learning model to cause the first instance of the second machine learning model to be updated based on the first input and the reference indication. In some embodiments, model subsystem 114 may provide the first input and the reference indication to a second instance of the second machine learning model (different from the first instance of the second machine learning model) to cause the second instance of the second machine learning model to be updated. Model subsystem 114 may cause updating of the application's instance of the second machine learning model by replacing, for the application, the first instance of the second machine learning model with the second instance of the second machine learning model. In this way, for example, the application may continue to operate simultaneously (e.g., using the first instance of the second machine learning model) while the second instance of the second machine learning model is being updated by training the second instance on training data (e.g., the first input, the reference indication, or other training data); para. [0057, 0058]).
Claim 19: Zeiler, Gray, and Singhal teach the method of claim 16. Zeiler further teaches wherein the executable instructions, when executed by the at least one processor, further cause the system to receive a user selection to retrain the first Al agent or the second Al agent (i.e. model subsystem 114 may enable one or more machine learning models to be trained with respect to a workflow or other arrangement of the machine learning models. In some embodiments, the machine learning models of a workflow may be trained/re-trained/further trained subsequent to being added to the workflow and prior to being incorporated as at least part of an application being developed. In some embodiments, the machine learning models of the workflow may be trained/re-trained/further trained subsequent to being added to the workflow and subsequent to being incorporated as at least part of the application. Training data used to train the machine learning models may include (i) inputs to be provided to a machine learning model (e.g., inputs provided to and processed by other machine learning models or other inputs), (ii) reference outputs that are to be derived from a machine learning model's processing of such inputs (e.g., user-confirmed or user-provided outputs, outputs confirmed through one or more machine learning models' processing of such inputs, outputs confirmed multiple times by processing of such inputs by respective sets of machine learning models, or other reference outputs); para. [0047]).
Claim 20: Zeiler, Gray, and Singhal teach the system of claim 16. Zeiler further teaches wherein the executable instructions, when executed by the at least one processor, further cause the system to receive, via the workflow editor interface, a user selection of a third Al agent (i.e. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components; para. [0034, 0036]); receive, via the workflow editor interface, a selection of data to input to the third Al agent, wherein the data to input to the third Al agent comprises the data output by the first Al agent (i.e. The single call may cause a first machine learning model of the workflow to process one or more inputs and provide its outputs as inputs to one or more other machine learning models or non-machine-learning models, and cause those other machine learning models or non-machine-learning models to process the first machine learning model's outputs and provide their outputs to other components of the workflow or to the application (e.g., via the API client or other component of the application); para. [0036, 0037]); and executing the third Al agent in parallel with the second Al agent (i.e. a single call to a workflow may cause multiple machine learning models of the workflow to be executed. The single call may cause a first machine learning model of the workflow to process one or more inputs and provide its outputs as inputs to one or more other machine learning models or non-machine-learning models, and cause those other machine learning models or non-machine-learning models to process the first machine learning model's outputs and provide their outputs to other components of the workflow or to the application (e.g., via the API client or other component of the application); para. [0037, 0038]).
7. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zeiler in view of Singhal, and further in view of Amer et al. (U.S. Patent Application Pub. No. US 20190304157 A1).
Claim 11: Zeiler and Singhal teach the method of claim 1. Zeiler further teaches comprising: receiving, via the workflow editor interface, a user selection of an interface to the first Al agent (i.e. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application; para. [0035, 0036]); and after activating the first Al agent, displaying the interface (i.e. building blocks of a workflow may include a set of operation blocks for processing inputs and creating outputs to be passed into other stages of the workflow. Some examples of these operation blocks include (i) machine learning models, (ii) conditional statements to decide whether an output should pass to a next stage or which block to pass the output, (iii) mathematical/statistical models, (iv) image/video/audio/text/other data processing operation blocks, (v) operation blocks to initiate requests to third party APIs or programs, (vi) user-interface-related operation blocks (e.g., which cause one or more interactive features of a user interface), (vii) notification operation blocks (e.g., which cause one or more notifications to be sent or presented); para. [0038, 0049]).
Zeiler does not explicitly teach an explainability interface; and displaying the explainability interface.
However, Amer teaches an explainability interface to attach to the first Al agent; and after activating the first Al agent, displaying the explainability interface (i.e. An effective explainable artificial intelligence system (“XAI”) system will typically maintain performance quality while also being explainable. In such a system, the end user who depends on decisions, recommendations, or actions produced by the XAI system will preferably understand at least aspects the rationale for the system's decisions. For example, a test operator of a newly developed autonomous system will need to understand why the system makes its decisions so that the text operator can decide how to use it in the future. Accordingly, at least some successful XAI systems may provide end users with an explanation of individual decisions, enable users to understand the system's overall strengths and weaknesses, convey an understanding of how the system will behave in the future, and perhaps how to correct the system's mistakes; para. [0153-0155]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Zeiler and Singhal to include the feature of Amer. One would have been motivated to make this modification because users can develop a greater level of trust in the system. Transparent decision-making reduces uncertainty and fosters confidence in the AI’s operations.
8. Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Zeiler in view of Singhal, and further in view of Biswas et al. (U.S. Patent Application Pub. No. US 20190102695 A1).
Claim 12: Zeiler and Singhal teach the method of claim 1. Zeiler further teaches comprising: displaying of the first Al agent when the first AI agent is selected (i.e. one or more service platforms described herein may provide one or more user interfaces that present one or more options to select/access the model representations. Responsive to user selection of such options, the selected model representations may be presented on the user interface; para. [0034]); receiving, via user input, a modification; and modifying the first Al agent (i.e. system 100 may cause user-selectable model representations, input/output representations, or other representations to be available via a user interface. Based on user input indicating selection of one or more model or input/output representations, system 100 may cause a presentation of the model representations, input/output representations, or other representations. Additionally, or alternatively, based on the user input or subsequent user-initiated changes related to the representations, system 100 may generate at least a portion of a software application. As an example, responsive to the generation, the resulting software application may include one or more instances of machine learning models or non-machine-learning models that correspond to the presented model representations, one or more input/output paths between respective ones of the models or from/to other sources/destinations, or other components. In one use case, suppose a user wishes to use an API service's general image classification model and face detection model to find scenes containing a certain number of people in outdoor settings in a stream of images from a social media service; para. [0036, 0047]).
Zeiler does not explicitly teach configurable parameters.
However, Biswas teaches comprising: displaying configurable parameters of the first Al agent when the first AI agent is selected; receiving, via user input, a modification to the configurable parameters; and modifying the first Al agent based on the modification to the configurable parameters (i.e. At step 902, one or more machine learning configuration files are stored at a machine learning server computer. A particular machine learning configuration file of the one or more machine learning configuration files comprises instructions for configuring a machine learning system of a particular machine learning type with one or more first machine learning parameters. For example, the machine learning server computer may store a configuration file that includes instructions for building a Niave Bayes classifier with one or more default parameters that are configurable through the graphical user interface. Default parameters may include default values and/or placeholder values. For example, the machine learning server computer may be programmed or configured to input values into the configuration file based on selections of parameters through the advanced settings in the graphical user interface. While some parameter values may be selected by default, they may not be initially entered in the configuration file; para. [0153]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Zeiler and Singhal to include the feature of Biswas. One would have been motivated to make this modification because it enhances the overall user experience by making the system interactive, transparent, and aligned with user preferences.
9. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Zeiler in view of Singhal, and further in view of Ambati et al. (U.S. Patent Application Pub. No. US 20180293462 A1).
Claim 15: Zeiler and Singhal teach the method of claim 1. Zeiler further teaches comprising: after receiving the selection of data to input to the second Al agent, determining that a format of the data output by the first Al agent is not compatible with a format of the data to input to the second Al agent (i.e. a user interface of system 100 may guide the developer to select and connect operation blocks that are compatible with one another, thereby reducing errors resulting from application component incompatibility. The user interface (or its associated service platform) may provide a developer with a set of compatible options when the developer selects an operation block that he/she wants to input to, output to, or otherwise extend. Additionally, or alternatively, the user interface may prevent a developer from connecting two operation blocks that are not compatible with one another. Compatibility may, for example, be determined based on input/output data types of the operation blocks. As an example, if a developer tries to add a block that takes as input the output of a previous block, the blocks may be determined to be compatible responsive to the input data type/format of that block and the output data type/format of the previous block being of the same data type/format. That simple check, in addition to other checks for constraints (e.g., such as where the processing of each block is happening or other checks), facilitates workflow operations; para. [0039]).
Zeiler does not explicitly teach determining a transform to apply to the data output to convert the data output to the format of the data to input; and applying the transform to the data output.
However, Ambati teaches determining a transform to apply to the data output by the first Al agent to convert the data output by the first Al agent to the format of the data to input to the second Al agent; and applying the transform to the data output by the first Al agent (i.e. Transformer(s) 122 are configured to transform data associated with a plurality of databases into a common ontology. A common ontology is specified such that data associated with the plurality of databases is described in a consistent format. This allows a machine learning model to be generated using data from a plurality of different data sources and ensures that the machine learning model is trained using feature values that are in a consistent format. Transformer(s) 122 may include one or more transformers that transform data to a common format. For example, transformer(s) 122 may include a transformer to convert name information associated with an entry from a “last name, first name, middle initial” format to a “first name last name” format. This transformer may enable entries from a plurality of databases to be combined into a single entry because the entries may be associated with the same entity; para. [0039-0041, 0070-0079]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Zeiler and Singhal to include the feature of Ambati. One would have been motivated to make this modification because it ensures smooth communication and data transfer between agents.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Khare et al. (Pub. No. US 11605021 B1), As shown in FIG. 4, at numeral 1, when a new model version has been created, the model training system 122 can update a model data store 400 indicating that the new model is available. In some embodiments, the new model can be added to the model data store. Alternatively, the model data store may be updated to include a pointer to the model in another storage location. A model manager 402 at the model hosting system 140 can poll the model data store at numeral 2 to determine whether a new model is available. When there is a new model available, the model manager 402 can load the new model into the model endpoint 404, as shown at numeral 3. While the model is being loaded, any new inference requests received by the endpoint can be added to cache 406 where they may be queued until the model is ready to receive requests. This enables the new model to be deployed in seconds rather than the several minutes that were previously required to provision new hosts for a new endpoint for the model. Once the new model is ready the queued messages are processed from the cache 406.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAN TRAN whose telephone number is (303)297-4266. The examiner can normally be reached on Monday - Thursday - 8:00 am - 5:00 pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAN H TRAN/Primary Examiner, Art Unit 2141
Prosecution Insights