Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/4/2026 has been entered.
Response to Amendment
(Submitted on 5/4/26)
The examiner notes that in response to applicant’s amendment made after FINAL REJECTION on 3/20/26, an advisory action was sent by the examiner on 5/4/26. Subsequently, the applicant submitted an RCE on 5/4/26 without any arguments. The examiner’s current office action was based on new grounds of rejection. The examiner notes that the applicant has CANCELLED the claims 9-10 and 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 11, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over
Adams et.al (hereinafter Adams) US 10901817 B1,
in view of Jeffrey Rapaport et.al (hereinafter Rap) US 2010/0205541 A1,
in view of Yaniv Gur et.al (hereinafter Gur) US 2021/0019665 A1.
Regarding claim 1. (Currently Amended)
Adams discloses:
- A method for classification prediction, performed by a terminal, comprising:
[Col 4, lines 26-29]:
The analytic server may have each model in an individual container. As a result, each model may execute individually. Multiple models may execute at the same time, with each model as a low balance circuit.
[Col 2, lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
(BRI: A classification model is a predictive analytics model that predicts a sample)
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 7-10]:
The users may initiate the electronic requests through web applications (e.g., comprising a hyperlink of a website) and/or mobile applications installed in the electronic user devices 140.
reading a configuration file and starting a classification prediction service
[Col 6, 17-19]:
The data scientist computing device 130 may be any computing device allowing a data scientist to interact with the analytic server 110.
[Col 6, 25-27]:
The data scientist computing device 130 may comprise any number of input and output devices supporting various types of data, such as text
[Col 6, 28-35]:
Each data scientist may train a predictive computer model for a certain prediction service on the data scientist computing device 130. The data scientist may also create a model specification file to describe the model based on a template provided by the analytic server 110. The model speciation file may include description on the service provided by the model, the expected inputs, the expected outputs.
[Col 6, 35-39]:
The data scientist may fill in configuration data for a set of fields in the template to describe the trained model, such as the data types and ranges of the inputs, the expected output, the provided service of the model,
[Col 6, 40-42]:
The analytic server 110 may receive the plurality of predictive computer models and their specification files from the data scientist computing devices 130.
[Col 6, 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
- wherein the configuration file comprises a prediction framework for performing batch prediction on classification prediction tasks,
[Col 2, Lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
[Col 2, Lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
- obtaining a classification prediction request, wherein the classification prediction request comprises a branch identifier.
[Col 2, lines 13-16]:
providing a system and method for efficiently processing electronic requests. Specifically, an analytic server may have a plurality of predictive computer models registered on a model-by-model basis,
[Col 2, lines 19-20]:
Based on the model specification file, the analytic server may extract a set of validation codes for each model,
[Col 3, lines 20-29]:
an electronic request to execute a predictive computer model within the plurality of predictive computer models using a set of inputs; identify, by the application programming interface layer of the server, a first predictive computer model corresponding to the received electronic request; validate, in the application programming interface layer, the electronic request by verifying that the set of inputs corresponds to the required input range and type in the set of validation codes of the first predictive computer model;
[BRI: the validation code is the branch identifier]
determining a service branch corresponding to the classification prediction request the classification prediction service according to the branch identifier,
[Col 2, lines 21-35]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model. If the electronic request is invalid, the analytic server may return an error message immediately with helpful information to the user. If the electronic request is valid, the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
- the service branch representing a model of the models supported by the classification prediction service;
[Col 2, lines 1-3]:
a system and method to execute different predictive computer models,.
[Col 2, lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
[Abstract]:
When the server receives an electronic request, the API layer of the server validates the request by verifying the inputs of the request satisfying the validation codes of the corresponding model,
- initializing a batch task generation method through the universal classification prediction interface, the batch task generation method comprising:
[ Col 8, lines 64-67]; [Col 9, line 1]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[BRI: the user’s initiation of a batch prediction request does represent initializing a batch task generation method, and the assigned ID (often accessible via a URL) is the mechanism to check the request’s status]
- in response to at least one classification prediction task comprising a prediction object to be classified and the prediction object to be classified represents a text content
[Col 11, lines 59-64]:
In addition to on-demand (synchronous) prediction requests, the analytic server may support batch requests with each batch containing dozens of prediction requests. As discussed above, the analytic server may put the batch requests in an in-memory data structure store (e.g., REDIS) queue
[Col 12, lines 6-12]:
For example, the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue. The analytic server may queue the batch requests based on the time they are received, and/or based on the level of priority.
[Col 10, lines 27-40]:
At step 306, the analytic server may receive an electronic request, in the API layer, to execute a predictive computer model using a set of inputs included in the electronic request. The electronic request may request a certain prediction service that requires a corresponding model to execute based on the set of inputs. The analytic server may identify the predictive computer model corresponding to the received electronic request. For example, a customer or a sales agent may issue a request to determine the customer's mortality score. The electronic request may comprise a set of inputs, such as the user age, heart condition, and the smoking status. The analytic server may receive the request and determine that the corresponding model is the mortality model.
[BRI: the “analytic server” (model serving) supports batch inference for text classification, allowing you to send dozens (or even thousands) of text prediction requests in a single asynchronous job]
- generating an identifier of a single task for at least one classification prediction task;
[Col 9, lines 4-5]:
a process may wake up to execute the models of the requests in the order the requests were queued,
[Col 7, lines 1-3]:
The analytic server 110 may extract a set of validation codes for each predictive computer model based on the model specification file.
[Col 7, lines 6-9]) the API layer of the analytic server 110 may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model.
[Col 8, lines 40-43]:
For efficient resource allocation, each predictive computer model may be scaled and/or load balanced independently, enabling more popular models to have the resources they need to appropriately meet demand.
[Col 2, lines 28-35]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
[Col 5, lines 53-55]:
The set of electronic user devices 140 may be any computing device allowing a user to interact with the analytic server 110,
[Col 5, lines 60-63]:
The electronic user devices 140 may comprise any number of input and output devices supporting various types of data, such as text, image, audio, video, and the like,
[Col 2, lines 21-25]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model,
- adding the identifier to a to-be-processed identifier list of a batch prediction task;
[Col 7, lines 24-28]:
the analytic server processing the electronic requests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212,
[Col 8, lines 64-67], [Col 9, lines 1-8]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed. The analytic server may update the request status in the URL whenever the status changes. During off-business hours or any other appropriate time, a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
(BRI: assigning resource locator is adding the identifier)
- traversing the to-be-processed identifier list to obtain the prediction object to be classified that needs to be processed by a classification prediction task corresponding to at least one item;
[Col 8, lines 25-31]:
The specification/service storage 208 may comprise a centralized fast, in-memory data store (e.g., REDIS) for the input validation configuration for each registered predictive computer model/service, the SWAGGER specification, a queue for asynchronous batch prediction requests as they await execution as well as the responses to the batch requests,
[Col 9, lines 4-8]:
a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
- initializing a self-defined variable of the classification prediction tasks of the model represented by the service branch through a self-defined classification prediction interface corresponding to the model;
[Col 10, lines 41-49]:
In some embodiments, the analytic server may receive a large number of electronic requests at the same time. The API layer of the analytic server may assign the requests to different predictive computer models. In the registration process, the analytic server may record a set of predictive computer models from different data scientists. When the analytic server receives the electronic requests in API layer, the analytic server may determine, in the API layer, the corresponding models and call the corresponding models.
[Col 11, lines 20-25]:
At step 312, upon the set of inputs satisfying the set of validation codes of the corresponding predictive computer model, the analytic server may execute the corresponding predictive computer model using the set of inputs in an independent environment to generate output results and return the results.
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
[Col 6, lines 11-16]:
The analytic server 110 may have a plurality of predictive computer models with each predictive computer model providing a different prediction service. The plurality of predictive computer models may be individual models developed by different data scientists.
[Col 12, lines 8-9]:
the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue.
[BRI: The API layer’s role in predictive analytics servers includes model selection, request routing, and parameter setup. Initializing a self-defined variable for the classification prediction task is a valid and common way to store the model’s configuration before execution, enabling the API to correctly assign and run the right predictive model for each request. This is a common pattern in ML serving and predictive scoring systems, ensuring that the model’s parameters are correctly set before inference. In this context, “initializing” a self-defined classification prediction task means setting up the model, defining its schema, and enabling it to run predictions through your own interface]
- and performing batch prediction on the plurality of prediction objects through the model represented by the service branch to output prediction results of the classification prediction tasks,
[Col 7, lines 22-28]:
FIG. 2 illustrates components 200 of the analytic server for processing electronic requests, according to an embodiment. The analytic server processing the electronic rquests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212.
[Col 8, lines 55-57]: [Col 9, line]
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
[Col 8, lines 64-67]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[Col 2, lines 43-48]:
This per-model containerized architecture may enable efficient resource allocation and per-model dependency management. For example, each model may be scaled and load balanced independently. Multiple models may execute at the same time.
[BRI: A per-model containerized architecture where each machine learning model runs inside its own container can enable efficient resource allocation and it supports multiple models executing in parallel for batch prediction]
- wherein each of the prediction results comprises a field indicating an actual name of a classification corresponding to the text content,
[Col 9, lines 53-57]:
When the analytic server receives a request, the analytic server may determine and call the appropriate model to satisfy the request. For example, the analytic server may feed the request inputs into the appropriate model, execute the model, and return the output results,
[Col 10, lines 9-13]:
The analytic server may parse the model specification file and extract input validation codes corresponding to the input rules included in the model specification file. The analytic server may extract the validation codes using natural language processing algorithms,
[Col 11, lines 43-49]:
The results may be eventually displayed on the user's electronic device. For example, the analytic server may display the results in a GUI on the user's electronic device. In some embodiments, the analytic server may transmit the results via an electronic message, such as an email, an SMS (short message service) message.
[BRI: an analytic server can display classification results in a GUI and send them via email or SMS that are text contents]
Adams does not explicitly disclose:
- acquiring, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
- and acquiring a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier.
- and generating a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
- determining the branch identifiers of the branch services respectively corresponding to the models as primary keys; based on definitions of the models,
- based on the definition of the models, determining invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
However, Rap discloses:
- acquiring, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
[0038]:
Automatically inferred guesses of what the local user's current interest is (physical focus spot plus mental topic selecting state) and/or what the local user's current conscious or subconscious voting intents are, are automatically generated and respectively recorded in uploaded data objects such as Current Focus identifying records (CFi ‘s) and Current vote indicating records (Cvi's). Incidentally, CFi ‘s do not have to be generated from one same client machine.
[0038]:
In one embodiment, the CFi data generated by each machine or device is accumulated in the local machine's buffer and held there for automated background gathering (e.g., uploading) by an on-network client watching service (e.g., an AARMS server) as bandwidth permits,
[0272]:
Step 3. The second AARMS then accesses its service directory data store (431') to determine if, given the client's preferences as indicated in the fetched CpCCp and DsCCp's, whether the client should be redirected to a different AARMS instance that is better suited to handle this particular client,
[0273] :
Option 3a. The third AARMS is geographically closer to the client machine and/or the third AARMS can offer a higher turn-around speed to the client machine (e.g., a roaming cell phone with computing capabilities) and the third AARMS has sufficient load capacity to take on the service needs of the user and/or his currently utilized client machine
[0274]:
Option 3b. The second AARMS is near its workload capacity and there are other MRMS instances available with spare capacity and with the ability to meet the client's bandwidth requirements (e.g., high ones if the client is a premium service user or lower ones if not subscribed to a premium service).
- and acquiring a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier;
[0013]:
The present disclosure of invention relates generally to real time social networks and to production of on-topic search results including that of automatically producing matches between currently online people for instant engagement in real time,
[0013]:
Another aspect of the disclosure relates to providing a real time changing indexing tree having hierarchically interconnected nodes representing topics, where the configuration of the tree changes dynamically in response to real time reactions of users to online content. In one embodiment, nodes of the tree correspond to topics in a dynamically changing topic space and the nodes link to on-topic chat rooms and/or to on-topic search results. Domain and topic nodes in the tree
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
[0215]:
The categorization location within a data center hierarchy tree (e.g., 401 of FIG. 4B) of a particular Domain/topic/subtopic node relative to other Domains/topics nodes with similar common CFi match values (e.g., 464, 465a, 465b, etc.) may be used to automatically create "related Domain/topic nodes" lists
- and generating a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
- determining the branch identifiers of the branch services respectively corresponding to the models as primary keys.
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: in DNS, each root and domain-level node is uniquely identified by its domain name, and the branch (zone) is identified by that domain and its delegation records. These identifiers map directly to the authoritative data center that hosts the zone’s records, effectively serving as the primary key for that branch in the DNS lookup process]
- based on the definition of the models, determining invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
[0194]:
Domains/Topics Lookup Service (DLUX) with a large number of topic domains and topics and subtopics dynamically re-assigned (migrated) to specific domains and/or topics, subtopics, etc. or newly created to serve such topic areas depending on user population interest in the respective topics
in many modern content and event systems, dynamic assignment of topics, subtopics, or domains after models are instantiated is a recognized pattern. This is often implemented as a dynamic loading or routing mechanism that reacts to user population, interest signals, or other contextual data]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, and Rap.
Adams teaches prediction framework , model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
One of ordinary skill would have motivation to combine Adams, and Rap that can provide corresponding models of the user's personality likes and dislikes are refined and improved over time (Rap[0129]).
Adams and Rap do not explicitly disclose:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
- wherein the method further comprises: initializing a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
- and storing the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair.
However, Gur discloses:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
[0020]:
The illustrative embodiments provide a ML framework comprising one or more universal ML application programming interfaces (APIs) and one or more databases of ML algorithms. The ML framework of the illustrative embodiments abstract the details of the ML algorithms through the one or more universal ML APIs and provides an engine to automatically handle all connectivity and format details of ML algorithms so that individual algorithm developers become users through the one or more ML APIs
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction,
[0021]:
The ML algorithm metadata model itself includes a description of the aggregate collections, a description of features generated from collections, a description of the machine learning model, provenance information about the ML algorithm. The ML algorithms themselves are stored in a file repository. Connection interfaces are provided for allowing the analytics runtime engines to run the various ML algorithms on different architectures, e.g., Apache SPARK, UIMA, etc.
[BRI: Based on the provided information, a machine learning framework that combines one or more universal ML APIs with one or more databases of ML algorithms can be considered a universal classification prediction interface as it is explicitly designed as an end-to-end, automated ML pipeline that supports multiple problem types, including classification]
- wherein the method further comprises: initializing a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
- and initializing a universal batch classification prediction method; instantiating the models, and
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction, and exposes standard ways to input features for ML algorithms. APIs are provided to support connection and data formatting to the ML algorithms.
[0044]:
The ML algorithm APIs 110-118 provide a standardized or universal mechanism for inputting data collections, features, and the like, for the ML algorithms
- storing the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair
[0036]:
The stored ML algorithms are stored for selection by developers (users) when a new trained machine learning model is needed to perform a task, such as a task that is part of an analytics pipeline. The ML index 132 provides indexes of the stored ML algorithms in the ML repository 130 such that the indexes may be searched based on developer (user) specified criteria for the task to be performed, and corresponding matching ML algorithms may be retrieved and utilized to train a ML model to perform the task. The trained ML model may then be registered, by the trained ML model registration logic 120, in a ML model repository 140, indexed by a ML model index 142, for use in analytics pipelines to perform corresponding tasks.
[BRI: the ML index is the key and the ML model registered (stored) in the repository is the value]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap and Gur.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
One of ordinary skill would be motivated to combine Adams, Rap and Gur to provide performance of the trained model (Gur [0048]).
Regarding claim 7. (Previously Presented)
Adams discloses:
- performing, in response to the service branch being idle, the classification prediction task based on the service branch
[Col 8, lines 55-57:
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
[Col 8, lines 64-67], Col 9, line 1]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
(BRI: waiting for execution is an “idle” status)
Regarding claim 11. (Currently Amended)
Adams discloses:
- An apparatus for classification prediction, for use in a terminal, comprising: one or more processors; and a non-transitory computer-readable storage medium for storing instructions executable by the one or more processors, wherein the one or more processors are configured to:
[Col 5, lines 53-63], In [Col 14, lines 12-30]:
[Col 4, lines 26-29]:
The analytic server may have each model in an individual container. As a result, each model may execute individually. Multiple models may execute at the same time, with each model as a low balance circuit.
[Col 2, lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 7-10]:
The users may initiate the electronic requests through web applications (e.g., comprising a hyperlink of a website) and/or mobile applications installed in the electronic user devices 140.
read a configuration file and starting a classification prediction service
[Col 6, 17-19]:
The data scientist computing device 130 may be any computing device allowing a data scientist to interact with the analytic server 110.
[Col 6, 25-27]:
The data scientist computing device 130 may comprise any number of input and output devices supporting various types of data, such as text
[Col 6, 28-35]:
Each data scientist may train a predictive computer model for a certain prediction service on the data scientist computing device 130. The data scientist may also create a model specification file to describe the model based on a template provided by the analytic server 110. The model speciation file may include description on the service provided by the model, the expected inputs, the expected outputs.
[Col 6, 35-39]:
The data scientist may fill in configuration data for a set of fields in the template to describe the trained model, such as the data types and ranges of the inputs, the expected output, the provided service of the model,
[Col 6, 40-42]:
The analytic server 110 may receive the plurality of predictive computer models and their specification files from the data scientist computing devices 130.
[Col 6, 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
- wherein the configuration file comprises a prediction framework for performing batch prediction on classification prediction tasks,
[Col 2, Lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
[Col 2, Lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
- obtain a classification prediction request, wherein the classification prediction request comprises a branch identifier.
[Col 2, lines 13-16]:
providing a system and method for efficiently processing electronic requests. Specifically, an analytic server may have a plurality of predictive computer models registered on a model-by-model basis,
[Col 2, lines 19-20]:
Based on the model specification file, the analytic server may extract a set of validation codes for each model,
[Col 3, lines 20-29]:
an electronic request to execute a predictive computer model within the plurality of predictive computer models using a set of inputs; identify, by the application programming interface layer of the server, a first predictive computer model corresponding to the received electronic request; validate, in the application programming interface layer, the electronic request by verifying that the set of inputs corresponds to the required input range and type in the set of validation codes of the first predictive computer model;
[BRI: the validation code is the branch identifier]
determine a service branch corresponding to the classification prediction request the classification prediction service according to the branch identifier,
[Col 2, lines 21-35]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model. If the electronic request is invalid, the analytic server may return an error message immediately with helpful information to the user. If the electronic request is valid, the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
- the service branch representing a model of the models supported by the classification prediction service;
[Col 2, lines 1-3]:
a system and method to execute different predictive computer models,.
[Col 2, lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
[Abstract]:
When the server receives an electronic request, the API layer of the server validates the request by verifying the inputs of the request satisfying the validation codes of the corresponding model,
- initialize a batch task generation method through the universal classification prediction interface, the batch task generation method comprising:
[ Col 8, lines 64-67]; [Col9, line 1]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[BRI: the user’s initiation of a batch prediction request does represent initializing a batch task generation method, and the assigned ID (often accessible via a URL) is the mechanism to check the request’s status]
- in response to at least one classification prediction task comprising a prediction object to be classified and the prediction object to be classified represents a text content
[Col 11, lines 59-64]:
In addition to on-demand (synchronous) prediction requests, the analytic server may support batch requests with each batch containing dozens of prediction requests. As discussed above, the analytic server may put the batch requests in an in-memory data structure store (e.g., REDIS) queue
[Col 12, lines 6-12]:
For example, the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue. The analytic server may queue the batch requests based on the time they are received, and/or based on the level of priority.
[Col 10, lines 27-40]:
At step 306, the analytic server may receive an electronic request, in the API layer, to execute a predictive computer model using a set of inputs included in the electronic request. The electronic request may request a certain prediction service that requires a corresponding model to execute based on the set of inputs. The analytic server may identify the predictive computer model corresponding to the received electronic request. For example, a customer or a sales agent may issue a request to determine the customer's mortality score. The electronic request may comprise a set of inputs, such as the user age, heart condition, and the smoking status. The analytic server may receive the request and determine that the corresponding model is the mortality model.
[BRI: the “analytic server” (model serving) supports batch inference for text classification, allowing you to send dozens (or even thousands) of text prediction requests in a single asynchronous job]
- generate an identifier of a single task for at least one classification prediction task;
[Col 9, lines 4-5]:
a process may wake up to execute the models of the requests in the order the requests were queued,
[Col 7, lines 1-3]:
The analytic server 110 may extract a set of validation codes for each predictive computer model based on the model specification file.
[Col 7, lines 6-9]) the API layer of the analytic server 110 may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model.
[Col 8, lines 40-43]:
For efficient resource allocation, each predictive computer model may be scaled and/or load balanced independently, enabling more popular models to have the resources they need to appropriately meet demand.
[Col 2, lines 28-35]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
[Col 5, lines 53-55]:
The set of electronic user devices 140 may be any computing device allowing a user to interact with the analytic server 110,
[Col 5, lines 60-63]:
The electronic user devices 140 may comprise any number of input and output devices supporting various types of data, such as text, image, audio, video, and the like,
[Col 2, lines 21-25]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model,
- add the identifier to a to-be-processed identifier list of a batch prediction task;
[Col 7, lines 24-28]:
the analytic server processing the electronic requests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212,
[Col 8, lines 64-67], [Col 9, lines 1-8]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed. The analytic server may update the request status in the URL whenever the status changes. During off-business hours or any other appropriate time, a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
(BRI: assigning resource locator is adding the identifier)
- traverse the to-be-processed identifier list to obtain the prediction object to be classified that needs to be processed by a classification prediction task corresponding to at least one item;
[Col 8, lines 25-31]:
The specification/service storage 208 may comprise a centralized fast, in-memory data store (e.g., REDIS) for the input validation configuration for each registered predictive computer model/service, the SWAGGER specification, a queue for asynchronous batch prediction requests as they await execution as well as the responses to the batch requests,
[Col 9, lines 4-8]:
a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
- initialize a self-defined variable of the classification prediction tasks of the model represented by the service branch through a self-defined classification prediction interface corresponding to the model;
[Col 10, lines 41-49]:
In some embodiments, the analytic server may receive a large number of electronic requests at the same time. The API layer of the analytic server may assign the requests to different predictive computer models. In the registration process, the analytic server may record a set of predictive computer models from different data scientists. When the analytic server receives the electronic requests in API layer, the analytic server may determine, in the API layer, the corresponding models and call the corresponding models.
[Col 11, lines 20-25]:
At step 312, upon the set of inputs satisfying the set of validation codes of the corresponding predictive computer model, the analytic server may execute the corresponding predictive computer model using the set of inputs in an independent environment to generate output results and return the results.
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
[Col 6, lines 11-16]:
The analytic server 110 may have a plurality of predictive computer models with each predictive computer model providing a different prediction service. The plurality of predictive computer models may be individual models developed by different data scientists.
[Col 12, lines 8-9]:
the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue.
[BRI: The API layer’s role in predictive analytics servers includes model selection, request routing, and parameter setup. Initializing a self-defined variable for the classification prediction task is a valid and common way to store the model’s configuration before execution, enabling the API to correctly assign and run the right predictive model for each request. This is a common pattern in ML serving and predictive scoring systems, ensuring that the model’s parameters are correctly set before inference. In this context, “initializing” a self-defined classification prediction task means setting up the model, defining its schema, and enabling it to run predictions through your own interface]
- and perform batch prediction on the plurality of prediction objects through the model represented by the service branch to output prediction results of the classification prediction tasks,
[Col 7, lines 22-28]:
FIG. 2 illustrates components 200 of the analytic server for processing electronic requests, according to an embodiment. The analytic server processing the electronic rquests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212.
[Col 8, lines 55-57]: [Col 9, line]
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
[Col 8, lines 64-67]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[Col 2, lines 43-48]:
This per-model containerized architecture may enable efficient resource allocation and per-model dependency management. For example, each model may be scaled and load balanced independently. Multiple models may execute at the same time.
[BRI: A per-model containerized architecture where each machine learning model runs inside its own container can enable efficient resource allocation and it supports multiple models executing in parallel for batch prediction]
- wherein each of the prediction results comprises a field indicating an actual name of a classification corresponding to the text content,
[Col 9, lines 53-57]:
When the analytic server receives a request, the analytic server may determine and call the appropriate model to satisfy the request. For example, the analytic server may feed the request inputs into the appropriate model, execute the model, and return the output results,
[Col 10, lines 9-13]:
The analytic server may parse the model specification file and extract input validation codes corresponding to the input rules included in the model specification file. The analytic server may extract the validation codes using natural language processing algorithms,
[Col 11, lines 43-49]:
The results may be eventually displayed on the user's electronic device. For example, the analytic server may display the results in a GUI on the user's electronic device. In some embodiments, the analytic server may transmit the results via an electronic message, such as an email, an SMS (short message service) message.
[BRI: an analytic server can display classification results in a GUI and send them via email or SMS that are text contents]
Adams does not explicitly disclose:
- acquire, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
- and acquire a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier.
- and generate a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
- determine the branch identifiers of the branch services respectively corresponding to the models as primary keys; based on definitions of the models,
- determining based on the definition of the models, invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
However, Rap discloses:
- acquire, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
[0038]:
Automatically inferred guesses of what the local user's current interest is (physical focus spot plus mental topic selecting state) and/or what the local user's current conscious or subconscious voting intents are, are automatically generated and respectively recorded in uploaded data objects such as Current Focus identifying records (CFi ‘s) and Current vote indicating records (Cvi's). Incidentally, CFi ‘s do not have to be generated from one same client machine.
[0038]:
In one embodiment, the CFi data generated by each machine or device is accumulated in the local machine's buffer and held there for automated background gathering (e.g., uploading) by an on-network client watching service (e.g., an AARMS server) as bandwidth permits,
[0272]:
Step 3. The second AARMS then accesses its service directory data store (431') to determine if, given the client's preferences as indicated in the fetched CpCCp and DsCCp's, whether the client should be redirected to a different AARMS instance that is better suited to handle this particular client,
[0273] :
Option 3a. The third AARMS is geographically closer to the client machine and/or the third AARMS can offer a higher turn-around speed to the client machine (e.g., a roaming cell phone with computing capabilities) and the third AARMS has sufficient load capacity to take on the service needs of the user and/or his currently utilized client machine
[0274]:
Option 3b. The second AARMS is near its workload capacity and there are other MRMS instances available with spare capacity and with the ability to meet the client's bandwidth requirements (e.g., high ones if the client is a premium service user or lower ones if not subscribed to a premium service).
- and acquire a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier;
[0013]:
The present disclosure of invention relates generally to real time social networks and to production of on-topic search results including that of automatically producing matches between currently online people for instant engagement in real time,
[0013]:
Another aspect of the disclosure relates to providing a real time changing indexing tree having hierarchically interconnected nodes representing topics, where the configuration of the tree changes dynamically in response to real time reactions of users to online content. In one embodiment, nodes of the tree correspond to topics in a dynamically changing topic space and the nodes link to on-topic chat rooms and/or to on-topic search results. Domain and topic nodes in the tree
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
[0215]:
The categorization location within a data center hierarchy tree (e.g., 401 of FIG. 4B) of a particular Domain/topic/subtopic node relative to other Domains/topics nodes with similar common CFi match values (e.g., 464, 465a, 465b, etc.) may be used to automatically create "related Domain/topic nodes" lists
- and generate a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
- determine the branch identifiers of the branch services respectively corresponding to the models as primary keys.
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: in DNS, each root and domain-level node is uniquely identified by its domain name, and the branch (zone) is identified by that domain and its delegation records. These identifiers map directly to the authoritative data center that hosts the zone’s records, effectively serving as the primary key for that branch in the DNS lookup process]
- determine based on the definition of the models, invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
[0194]:
Domains/Topics Lookup Service (DLUX) with a large number of topic domains and topics and subtopics dynamically re-assigned (migrated) to specific domains and/or topics, subtopics, etc. or newly created to serve such topic areas depending on user population interest in the respective topics
in many modern content and event systems, dynamic assignment of topics, subtopics, or domains after models are instantiated is a recognized pattern. This is often implemented as a dynamic loading or routing mechanism that reacts to user population, interest signals, or other contextual data]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, and Rap.
Adams teaches prediction framework , model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
One of ordinary skill would have motivation to combine Adams, and Rap that can provide corresponding models of the user's personality likes and dislikes are refined and improved over time (Rap[0129]).
Adams and Rap do not explicitly disclose:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
- wherein the method further comprises: initialize a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
- and store the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair.
However, Gur discloses:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
[0020]:
The illustrative embodiments provide a ML framework comprising one or more universal ML application programming interfaces (APIs) and one or more databases of ML algorithms. The ML framework of the illustrative embodiments abstract the details of the ML algorithms through the one or more universal ML APIs and provides an engine to automatically handle all connectivity and format details of ML algorithms so that individual algorithm developers become users through the one or more ML APIs
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction,
[0021]:
The ML algorithm metadata model itself includes a description of the aggregate collections, a description of features generated from collections, a description of the machine learning model, provenance information about the ML algorithm. The ML algorithms themselves are stored in a file repository. Connection interfaces are provided for allowing the analytics runtime engines to run the various ML algorithms on different architectures, e.g., Apache SPARK, UIMA, etc.
[BRI: Based on the provided information, a machine learning framework that combines one or more universal ML APIs with one or more databases of ML algorithms can be considered a universal classification prediction interface as it is explicitly designed as an end-to-end, automated ML pipeline that supports multiple problem types, including classification]
- wherein the method further comprises: initialize a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
- and initialize a universal batch classification prediction method; instantiate the models, and
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction, and exposes standard ways to input features for ML algorithms. APIs are provided to support connection and data formatting to the ML algorithms.
[0044]:
The ML algorithm APIs 110-118 provide a standardized or universal mechanism for inputting data collections, features, and the like, for the ML algorithms
- store the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair
[0036]:
The stored ML algorithms are stored for selection by developers (users) when a new trained machine learning model is needed to perform a task, such as a task that is part of an analytics pipeline. The ML index 132 provides indexes of the stored ML algorithms in the ML repository 130 such that the indexes may be searched based on developer (user) specified criteria for the task to be performed, and corresponding matching ML algorithms may be retrieved and utilized to train a ML model to perform the task. The trained ML model may then be registered, by the trained ML model registration logic 120, in a ML model repository 140, indexed by a ML model index 142, for use in analytics pipelines to perform corresponding tasks.
[BRI: the ML index is the key and the ML model registered (stored) in the repository is the value]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap and Gur.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
One of ordinary skill would be motivated to combine Adams, Rap and Gur to provide performance of the trained model (Gur [0048]).
Regarding claim 17. (Previously Presented)
Adams discloses:
- perform the classification prediction task through the service branch in response to the service branch being idle.
[Col 8, lines 55-57]:
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
in [Col 8, lines 64-67], Col 9, line 1]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
Regarding claim 20. :(Currently Amended)
Adams discloses:
- A non-transitory computer-readable storage medium having a plurality of programs for execution by a terminal having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the terminal to perform acts comprising:
[Col 5, lines 53-63], In [Col 14, lines 12-30]:
[Col 4, lines 26-29]:
The analytic server may have each model in an individual container. As a result, each model may execute individually. Multiple models may execute at the same time, with each model as a low balance circuit.
[Col 2, lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 7-10]:
The users may initiate the electronic requests through web applications (e.g., comprising a hyperlink of a website) and/or mobile applications installed in the electronic user devices 140.
reading a configuration file and starting a classification prediction service
[Col 6, 17-19]:
The data scientist computing device 130 may be any computing device allowing a data scientist to interact with the analytic server 110.
[Col 6, 25-27]:
The data scientist computing device 130 may comprise any number of input and output devices supporting various types of data, such as text
[Col 6, 28-35]:
Each data scientist may train a predictive computer model for a certain prediction service on the data scientist computing device 130. The data scientist may also create a model specification file to describe the model based on a template provided by the analytic server 110. The model speciation file may include description on the service provided by the model, the expected inputs, the expected outputs.
[Col 6, 35-39]:
The data scientist may fill in configuration data for a set of fields in the template to describe the trained model, such as the data types and ranges of the inputs, the expected output, the provided service of the model,
[Col 6, 40-42]:
The analytic server 110 may receive the plurality of predictive computer models and their specification files from the data scientist computing devices 130.
[Col 6, 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
- wherein the configuration file comprises a prediction framework for performing batch prediction on classification prediction tasks,
[Col 2, Lines 14-19]:
an analytic server may have a plurality of predictive computer models registered on a model-by-model basis. The analytic server may receive a model specification file for each model that defines the expected inputs and other description of the model.
[Col 2, Lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
- obtaining a classification prediction request, wherein the classification prediction request comprises a branch identifier.
[Col 2, lines 13-16]:
providing a system and method for efficiently processing electronic requests. Specifically, an analytic server may have a plurality of predictive computer models registered on a model-by-model basis,
[Col 2, lines 19-20]:
Based on the model specification file, the analytic server may extract a set of validation codes for each model,
[Col 3, lines 20-29]:
an electronic request to execute a predictive computer model within the plurality of predictive computer models using a set of inputs; identify, by the application programming interface layer of the server, a first predictive computer model corresponding to the received electronic request; validate, in the application programming interface layer, the electronic request by verifying that the set of inputs corresponds to the required input range and type in the set of validation codes of the first predictive computer model;
[BRI: the validation code is the branch identifier]
determining a service branch corresponding to the classification prediction request the classification prediction service according to the branch identifier,
[Col 2, lines 21-35]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model. If the electronic request is invalid, the analytic server may return an error message immediately with helpful information to the user. If the electronic request is valid, the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
- the service branch representing a model of the models supported by the classification prediction service;
[Col 2, lines 1-3]:
a system and method to execute different predictive computer models,.
[Col 2, lines 28-33]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results.
[Abstract]:
When the server receives an electronic request, the API layer of the server validates the request by verifying the inputs of the request satisfying the validation codes of the corresponding model,
- initializing a batch task generation method through the universal classification prediction interface, the batch task generation method comprising:
[ Col 8, lines 64-67]; [Col9, line 1]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[BRI: the user’s initiation of a batch prediction request does represent initializing a batch task generation method, and the assigned ID (often accessible via a URL) is the mechanism to check the request’s status]
- in response to at least one classification prediction task comprising a prediction object to be classified and the prediction object to be classified represents a text content
[Col 11, lines 59-64]:
In addition to on-demand (synchronous) prediction requests, the analytic server may support batch requests with each batch containing dozens of prediction requests. As discussed above, the analytic server may put the batch requests in an in-memory data structure store (e.g., REDIS) queue
[Col 12, lines 6-12]:
For example, the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue. The analytic server may queue the batch requests based on the time they are received, and/or based on the level of priority.
[Col 10, lines 27-40]:
At step 306, the analytic server may receive an electronic request, in the API layer, to execute a predictive computer model using a set of inputs included in the electronic request. The electronic request may request a certain prediction service that requires a corresponding model to execute based on the set of inputs. The analytic server may identify the predictive computer model corresponding to the received electronic request. For example, a customer or a sales agent may issue a request to determine the customer's mortality score. The electronic request may comprise a set of inputs, such as the user age, heart condition, and the smoking status. The analytic server may receive the request and determine that the corresponding model is the mortality model.
[BRI: the “analytic server” (model serving) supports batch inference for text classification, allowing you to send dozens (or even thousands) of text prediction requests in a single asynchronous job]
- generating an identifier of a single task for at least one classification prediction task;
[Col 9, lines 4-5]:
a process may wake up to execute the models of the requests in the order the requests were queued,
[Col 7, lines 1-3]:
The analytic server 110 may extract a set of validation codes for each predictive computer model based on the model specification file.
[Col 7, lines 6-9]) the API layer of the analytic server 110 may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model.
[Col 8, lines 40-43]:
For efficient resource allocation, each predictive computer model may be scaled and/or load balanced independently, enabling more popular models to have the resources they need to appropriately meet demand.
[Col 2, lines 28-35]:
the API layer of analytic server may send the electronic request to predictive model execution layer. Within the predictive model execution layer, the analytic server may execute the corresponding model based on the request inputs and generate output results. The predictive model execution layer may transmit the output results back to the API layer, which may transmit the output results to the user device.
[Col 5, lines 53-55]:
The set of electronic user devices 140 may be any computing device allowing a user to interact with the analytic server 110,
[Col 5, lines 60-63]:
The electronic user devices 140 may comprise any number of input and output devices supporting various types of data, such as text, image, audio, video, and the like,
[Col 2, lines 21-25]:
When the analytic server receives an electronic request to execute one of the predictive computer models, the API layer of the analytic server may validate the electronic request by verifying the inputs of the request satisfying the set validation codes of the corresponding model,
- adding the identifier to a to-be-processed identifier list of a batch prediction task;
[Col 7, lines 24-28]:
the analytic server processing the electronic requests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212,
[Col 8, lines 64-67], [Col 9, lines 1-8]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed. The analytic server may update the request status in the URL whenever the status changes. During off-business hours or any other appropriate time, a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
(BRI: assigning resource locator is adding the identifier)
- traversing the to-be-processed identifier list to obtain the prediction object to be classified that needs to be processed by a classification prediction task corresponding to at least one item;
[Col 8, lines 25-31]:
The specification/service storage 208 may comprise a centralized fast, in-memory data store (e.g., REDIS) for the input validation configuration for each registered predictive computer model/service, the SWAGGER specification, a queue for asynchronous batch prediction requests as they await execution as well as the responses to the batch requests,
[Col 9, lines 4-8]:
a process may wake up to execute the models of the requests in the order the requests were queued. When each job completes, the analytic server may post the results to the URL initially returned to the user (e.g., original requestor) and update the status of the request.
- initializing a self-defined variable of the classification prediction tasks of the model represented by the service branch through a self-defined classification prediction interface corresponding to the model;
[Col 10, lines 41-49]:
In some embodiments, the analytic server may receive a large number of electronic requests at the same time. The API layer of the analytic server may assign the requests to different predictive computer models. In the registration process, the analytic server may record a set of predictive computer models from different data scientists. When the analytic server receives the electronic requests in API layer, the analytic server may determine, in the API layer, the corresponding models and call the corresponding models.
[Col 11, lines 20-25]:
At step 312, upon the set of inputs satisfying the set of validation codes of the corresponding predictive computer model, the analytic server may execute the corresponding predictive computer model using the set of inputs in an independent environment to generate output results and return the results.
[Col 5, lines 64-65]:
A user operating the electronic user device 140 may issue an electronic request for a certain prediction service.
[Col 6, lines 2-7]:
The users may initiate the electronic requests by interacting with a graphical user interface (GUI) provided by the analytic server 110. For example, the users may enter a set of inputs in the GUI and request to execute a predictive computer model based on the set of inputs.
[Col 6, lines 11-16]:
The analytic server 110 may have a plurality of predictive computer models with each predictive computer model providing a different prediction service. The plurality of predictive computer models may be individual models developed by different data scientists.
[Col 12, lines 8-9]:
the analytic server may receive batch requests comprising a plurality of electronic requests with low priority. The analytic server may store the batch requests into a data store as a queue.
[BRI: The API layer’s role in predictive analytics servers includes model selection, request routing, and parameter setup. Initializing a self-defined variable for the classification prediction task is a valid and common way to store the model’s configuration before execution, enabling the API to correctly assign and run the right predictive model for each request. This is a common pattern in ML serving and predictive scoring systems, ensuring that the model’s parameters are correctly set before inference. In this context, “initializing” a self-defined classification prediction task means setting up the model, defining its schema, and enabling it to run predictions through your own interface]
- and performing batch prediction on the plurality of prediction objects through the model represented by the service branch to output prediction results of the classification prediction tasks,
[Col 7, lines 22-28]:
FIG. 2 illustrates components 200 of the analytic server for processing electronic requests, according to an embodiment. The analytic server processing the electronic rquests may comprise the following components: data power 202, token authentication 204. API layer 206, specification and service storage 208, predictive model execution layer 210, and batch/asynchronous queue worker 212.
[Col 8, lines 55-57]: [Col 9, line]
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
[Col 8, lines 64-67]:
When a user initiates batch prediction requests, the analytic server may assign each of the batch requests a URL (uniform resource locator) that can be used to check the status of the request, including awaiting execution, execution failed, and execution completed.
[Col 2, lines 43-48]:
This per-model containerized architecture may enable efficient resource allocation and per-model dependency management. For example, each model may be scaled and load balanced independently. Multiple models may execute at the same time.
[BRI: A per-model containerized architecture where each machine learning model runs inside its own container can enable efficient resource allocation and it supports multiple models executing in parallel for batch prediction]
- wherein each of the prediction results comprises a field indicating an actual name of a classification corresponding to the text content,
[Col 9, lines 53-57]:
When the analytic server receives a request, the analytic server may determine and call the appropriate model to satisfy the request. For example, the analytic server may feed the request inputs into the appropriate model, execute the model, and return the output results,
[Col 10, lines 9-13]:
The analytic server may parse the model specification file and extract input validation codes corresponding to the input rules included in the model specification file. The analytic server may extract the validation codes using natural language processing algorithms,
[Col 11, lines 43-49]:
The results may be eventually displayed on the user's electronic device. For example, the analytic server may display the results in a GUI on the user's electronic device. In some embodiments, the analytic server may transmit the results via an electronic message, such as an email, an SMS (short message service) message.
[BRI: an analytic server can display classification results in a GUI and send them via email or SMS that are text contents]
Adams does not explicitly disclose:
- acquiring, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
- and acquiring a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier.
- and generating a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
- determining the branch identifiers of the branch services respectively corresponding to the models as primary keys; based on definitions of the models,
- based on the definition of the models, determining invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
However, Rap discloses:
- acquiring, in response to a processing capacity of the service branch not being met, task loads of the classification prediction tasks with the branch identifier, until no classification prediction task with the branch identifier exists or the processing capacity of the service branch is met;
[0038]:
Automatically inferred guesses of what the local user's current interest is (physical focus spot plus mental topic selecting state) and/or what the local user's current conscious or subconscious voting intents are, are automatically generated and respectively recorded in uploaded data objects such as Current Focus identifying records (CFi ‘s) and Current vote indicating records (Cvi's). Incidentally, CFi ‘s do not have to be generated from one same client machine.
[0038]:
In one embodiment, the CFi data generated by each machine or device is accumulated in the local machine's buffer and held there for automated background gathering (e.g., uploading) by an on-network client watching service (e.g., an AARMS server) as bandwidth permits,
[0272]:
Step 3. The second AARMS then accesses its service directory data store (431') to determine if, given the client's preferences as indicated in the fetched CpCCp and DsCCp's, whether the client should be redirected to a different AARMS instance that is better suited to handle this particular client,
[0273] :
Option 3a. The third AARMS is geographically closer to the client machine and/or the third AARMS can offer a higher turn-around speed to the client machine (e.g., a roaming cell phone with computing capabilities) and the third AARMS has sufficient load capacity to take on the service needs of the user and/or his currently utilized client machine
[0274]:
Option 3b. The second AARMS is near its workload capacity and there are other MRMS instances available with spare capacity and with the ability to meet the client's bandwidth requirements (e.g., high ones if the client is a premium service user or lower ones if not subscribed to a premium service).
- and acquiring a plurality of prediction objects to be classified that respectively need to be processed by the classification prediction tasks with the branch identifier;
[0013]:
The present disclosure of invention relates generally to real time social networks and to production of on-topic search results including that of automatically producing matches between currently online people for instant engagement in real time,
[0013]:
Another aspect of the disclosure relates to providing a real time changing indexing tree having hierarchically interconnected nodes representing topics, where the configuration of the tree changes dynamically in response to real time reactions of users to online content. In one embodiment, nodes of the tree correspond to topics in a dynamically changing topic space and the nodes link to on-topic chat rooms and/or to on-topic search results. Domain and topic nodes in the tree
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
[0215]:
The categorization location within a data center hierarchy tree (e.g., 401 of FIG. 4B) of a particular Domain/topic/subtopic node relative to other Domains/topics nodes with similar common CFi match values (e.g., 464, 465a, 465b, etc.) may be used to automatically create "related Domain/topic nodes" lists
- and generating a model dictionary according to branch identifiers of branch services respectively corresponding to the models, the model dictionary representing a corresponding relationship between branch identifiers and corresponding model invoking interfaces, wherein generating the model dictionary according to the branch identifiers of the branch services respectively corresponding to the models comprises:
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: the data center LUT is the dictionary]
- determining the branch identifiers of the branch services respectively corresponding to the models as primary keys.
[0217]:
Referring to the specifics of FIG. 4B, one possible hierarchical organizing system 401 (as defined by stored data in a corresponding data center) is shown to include a global domains root 402 for its respective data center number, N. Each data center 410, 420, etc. will have its own root. Each root will having uniquely identified (e.g., numerically marked) branches 405, 407, etc. leading to respective domain level nodes 450, 470, etc. In one embodiment, the unique branch identifiers) of branches 405, 407, etc. correspond directly to or map through a data center lookup table (LUT, not shown) to the navigation type meta-tags that are usable in online content (e.g., as 317a of FIG. 3A) for determining topics of interest. Thus a corresponding meta-tag code provided in a user's uploaded CFi data (e.g., 118a of FIG. 1A) can be used by the matching service 150c of a given data center to hint at or directly address a corresponding node in the data center's domains configuration store 435 when performing a topic match.
[BRI: in DNS, each root and domain-level node is uniquely identified by its domain name, and the branch (zone) is identified by that domain and its delegation records. These identifiers map directly to the authoritative data center that hosts the zone’s records, effectively serving as the primary key for that branch in the DNS lookup process]
- based on the definition of the models, determining invoking interfaces for the models through a dynamic loading mechanism after the models are instantiated.
[0194]:
Domains/Topics Lookup Service (DLUX) with a large number of topic domains and topics and subtopics dynamically re-assigned (migrated) to specific domains and/or topics, subtopics, etc. or newly created to serve such topic areas depending on user population interest in the respective topics
[BRI: in many modern content and event systems, dynamic assignment of topics, subtopics, or domains after models are instantiated is a recognized pattern. This is often implemented as a dynamic loading or routing mechanism that reacts to user population, interest signals, or other contextual data]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, and Rap.
Adams teaches prediction framework , model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
One of ordinary skill would have motivation to combine Adams, and Rap that can provide corresponding models of the user's personality likes and dislikes are refined and improved over time (Rap[0129]).
Adams and Rap do not explicitly disclose:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
- wherein the method further comprises: initializing a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
- and storing the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair.
However, Gur discloses:
- and wherein the prediction framework at least comprises: a definition of a universal classification prediction interface, and definitions of self-defined classification prediction interfaces respectively corresponding to models with different types supported by the classification prediction service;
[0020]:
The illustrative embodiments provide a ML framework comprising one or more universal ML application programming interfaces (APIs) and one or more databases of ML algorithms. The ML framework of the illustrative embodiments abstract the details of the ML algorithms through the one or more universal ML APIs and provides an engine to automatically handle all connectivity and format details of ML algorithms so that individual algorithm developers become users through the one or more ML APIs
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction,
[0021]:
The ML algorithm metadata model itself includes a description of the aggregate collections, a description of features generated from collections, a description of the machine learning model, provenance information about the ML algorithm. The ML algorithms themselves are stored in a file repository. Connection interfaces are provided for allowing the analytics runtime engines to run the various ML algorithms on different architectures, e.g., Apache SPARK, UIMA, etc.
[BRI: Based on the provided information, a machine learning framework that combines one or more universal ML APIs with one or more databases of ML algorithms can be considered a universal classification prediction interface as it is explicitly designed as an end-to-end, automated ML pipeline that supports multiple problem types, including classification]
- wherein the method further comprises: initializing a universal variable of the models through the universal classification prediction interface, and performing corresponding startup setting;
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
- and initializing a universal batch classification prediction method; instantiating the models, and
[0043]:
The ML framework 100 may utilize the universal APIs 110-118 to train a ML model using a selected ML algorithm
[0042]:
A user selection of a ML algorithm to use to train a ML model to perform a specified task may be received via the user interface 150 to initiate a training operation to utilize the selected ML algorithm for training a corresponding ML model, after which the ML model may be registered in the trained model repository 140 and indexed in the trained model index 142, as well as deployed as part of an analytics pipeline.
[0021]:
The set of universal ML APIs enables common machine learning tasks, e.g., training, testing, prediction, and exposes standard ways to input features for ML algorithms. APIs are provided to support connection and data formatting to the ML algorithms.
[0044]:
The ML algorithm APIs 110-118 provide a standardized or universal mechanism for inputting data collections, features, and the like, for the ML algorithms
- storing the primary keys and the invoking interfaces as the model dictionary to a model prediction key value pair
[0036]:
The stored ML algorithms are stored for selection by developers (users) when a new trained machine learning model is needed to perform a task, such as a task that is part of an analytics pipeline. The ML index 132 provides indexes of the stored ML algorithms in the ML repository 130 such that the indexes may be searched based on developer (user) specified criteria for the task to be performed, and corresponding matching ML algorithms may be retrieved and utilized to train a ML model to perform the task. The trained ML model may then be registered, by the trained ML model registration logic 120, in a ML model repository 140, indexed by a ML model index 142, for use in analytics pipelines to perform corresponding tasks.
[BRI: the ML index is the key and the ML model registered (stored) in the repository is the value]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap and Gur.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
One of ordinary skill would be motivated to combine Adams, Rap and Gur to provide performance of the trained model (Gur [0048]).
Claims 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over
Adams et.al (hereinafter Adams) US 10901817 B1,
in view of Jeffrey Rapaport et.al (hereinafter Rap) US 2010/0205541 A1,
in view of Yaniv Gur et.al (hereinafter Gur) US 2021/0019665 A1.
further in view of Jorey Ramer et.al (hereinafter Ramer) US 2011/0258049 A1.
Regarding claim 22: (Previously Presented)
Adams discloses:
and in response to a type of the model represented by the service branch being "PytextPredictor", the self defined variable corresponding to the type comprises "predictor" and “max_seq_ln”
[Col 7, lines 55-60]:
The API layer 206 may be written in Python and may leverage a lightweight toolkit for building efficient asynchronous web services (e.g., STARLETTE). The API layer 206 may provide the following functionalities: prediction execution service registration, input validation, clean separation, error handling, and SWAGGER specification.
[Col 2, lines 48-51]:
Each model may have their own set of third-party libraries. So that each model can leverage whatever version of R/Python (and any associated third-party dependencies) they require without breaking the functionality of any other registered model.
3])
Adams, Rap and Gur do not explicitly disclose:
- wherein in response to a type of the model represented by the service branch being
"ERNIEPredictor", the self-defined variable corresponding to the type comprises” “path”, “ vocab_path”, “map_json_file”and "use_gpu";
However, Ramer discloses:
- wherein in response to a type of the model represented by the service branch being
"ERNIEPredictor", the self-defined variable corresponding to the type comprises” “path”, “ vocab_path”, “map_json_file”and "use_gpu";
[0869]:
Reports may also be delivered by FTP, where the wireless provider 108 provides a host name, user name, password, and directory name. Reports are then delivered as file drops into the given directory. The filenames include the name of the report, a timestamp and sequence number,
[1202]:
The monetization platform may enable supporting directory services as a site
Inventory within the network hierarchy,
[2068]:
The processor may be or include a signal processor, digital processor, embedded processor, microprocessor or any variant such as a co-processor (math co-processor, graphic co-processor,
[0222]:
A mobile content relating to a ringtone derived from a popular Chinese-language song may receive a higher relevancy score for the contextual information of "Server Host=China" than for "Server Host=Argentina."
[0427]:
a generalized process may be used for disambiguating a user query entry 120 by pairing the query information with at least one element from the mobile subscriber characteristics 112 database 112.
[1755]:
Contextual Analysis: URL. fwdarw. Keywords & Categories
[1756]:
Basic contextual analysis,
[1757]:
NLP powered keyword aggregation (synonyms) [1758] NLP powered micro categories (concepts), smaller than categories bigger than keywords [1759] Good entity lists plus patterns, train with NLP marked up text (entities, verbs).fwdarw.meaningful patterns,
[1892]:
An ad network may be an organization or an entity that may connect entities that want to host advertisements with advertisers who want to present sponsored content,
[1965]:
Support vector machines may be used in machine learning to detect and exploit complex patterns in data by clustering, classifying and ranking the data,
[0054]:
FIG. 23 illustrates a generalized method for mobile user profile category creation based on user browse behaviors,
[0132]:
the search engine might be a search facility that is based on Boolean search logic, categories of results, term frequencies, document frequencies, documents selected by other users who have entered similar queries, link structures of possible results, or any other known search facilities using any other known search algorithm,
[0170] :
The algorithm facility 144 may contain a collaborative filtering protocol, category filtering, a recommendation system and/or other process facilities for analyzing, refining, or filtering user input and/or search results,
[0220]:
Automatic syndication of mobile content to a website may be based at least in part on using contextual information associated with the website in order to determine the relevancy of mobile content that is available for syndication. Contextual information that may be associated with a website may include keywords, terms, or phases located on the website,
[1963]:
Linear regression modeling may be used to analyze the relationship between a response, or dependent, variable and a set of independent or predictor variables and predicts the response variable as a linear function of the parameters,
[1964]:
time series modeling may be used for predicting or forecasting the future behavior of variables and account for the fact that data points taken over time may have an internal structure that should be considered.
[1965]:
Geospatial predictive modeling techniques may be used to describe constraints and influences by spatially correlating occurrences of historical geospatial locations with environmental factors
(BRI: the specification [0099] recites that the “ERNIEPredictor” is an example)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap , Gur and Ramer.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Ramer teaches ERNIEPREDICTOR.
One of ordinary skill would be motivated to combine Adams, Rap, Gur and Ramer to improve the search results relevancy score (Ramer [0343])
Regarding claim 24 (Previously Presented)
Adams discloses:
- and in response to a type of the model represented by the service branch being "PytextPredictor", the self defined variable corresponding to the type comprises "predictor" and “max_seq_ln”
[Col 7, lines 55-60]:
The API layer 206 may be written in Python and may leverage a lightweight toolkit for building efficient asynchronous web services (e.g., STARLETTE). The API layer 206 may provide the following functionalities: prediction execution service registration, input validation, clean separation, error handling, and SWAGGER specification.
[Col 2, lines 48-51]:
Each model may have their own set of third-party libraries. So that each model can leverage whatever version of R/Python (and any associated third-party dependencies) they require without breaking the functionality of any other registered model.
Adams, Rap and Gur do not explicitly disclose:
- wherein in response to a type of the model represented by the service branch being
"ERNIEPredictor", the self-defined variable corresponding to the type comprises” “path”, “ vocab_path”, “map_json_file”and "use_gpu";
However, Ramer discloses:
- wherein in response to a type of the model represented by the service branch being
"ERNIEPredictor", the self-defined variable corresponding to the type comprises” “path”, “ vocab_path”, “map_json_file”and "use_gpu";
[0869]:
Reports may also be delivered by FTP, where the wireless provider 108 provides a host name, user name, password, and directory name. Reports are then delivered as file drops into the given directory. The filenames include the name of the report, a timestamp and sequence number,
[1202]:
The monetization platform may enable supporting directory services as a site
Inventory within the network hierarchy,
[2068]:
The processor may be or include a signal processor, digital processor, embedded processor, microprocessor or any variant such as a co-processor (math co-processor, graphic co-processor,
In [0222]:
A mobile content relating to a ringtone derived from a popular Chinese-language song may receive a higher relevancy score for the contextual information of "Server Host=China" than for "Server Host=Argentina."
[0427]:
a generalized process may be used for disambiguating a user query entry 120 by pairing the query information with at least one element from the mobile subscriber characteristics 112 database 112.
[1755]:
Contextual Analysis: URL.fwdarw.Keywords & Categories
[1756]:
Basic contextual analysis,
[1757]:
NLP powered keyword aggregation (synonyms) [1758] NLP powered micro categories (concepts), smaller than categories bigger than keywords [1759] Good entity lists plus patterns, train with NLP marked up text (entities, verbs).fwdarw.meaningful patterns,
[1892]:
An ad network may be an organization or an entity that may connect entities that want to host advertisements with advertisers who want to present sponsored content,
[1965]:
Support vector machines may be used in machine learning to detect and exploit complex patterns in data by clustering, classifying and ranking the data,
[0054]:
FIG. 23 illustrates a generalized method for mobile user profile category creation based on user browse behaviors,
[0132]:
the search engine might be a search facility that is based on Boolean search logic, categories of results, term frequencies, document frequencies, documents selected by other users who have entered similar queries, link structures of possible results, or any other known search facilities using any other known search algorithm,
[0170] :
The algorithm facility 144 may contain a collaborative filtering protocol, category filtering, a recommendation system and/or other process facilities for analyzing, refining, or filtering user input and/or search results,
[0220]:
Automatic syndication of mobile content to a website may be based at least in part on using contextual information associated with the website in order to determine the relevancy of mobile content that is available for syndication. Contextual information that may be associated with a website may include keywords, terms, or phases located on the website,
[1963]:
Linear regression modeling may be used to analyze the relationship between a response, or dependent, variable and a set of independent or predictor variables and predicts the response variable as a linear function of the parameters,
[1964]:
time series modeling may be used for predicting or forecasting the future behavior of variables and account for the fact that data points taken over time may have an internal structure that should be considered.
[1965]:
Geospatial predictive modeling techniques may be used to describe constraints and influences by spatially correlating occurrences of historical geospatial locations with environmental factors
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap, Gur and Ramer.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Ramer teaches ERNIEPREDICTOR.
One of ordinary skill would have motivation to combine Adams, Rap, Gur and Ramer to improve the search results relevancy score ( Ramer [0343])
Claims 21 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over
Adams et.al (hereinafter Adams) US 10901817 B1,
in view of Jeffrey Rapaport et.al (hereinafter Rap) US 2010/0205541 A1.
in view of Yaniv Gur et.al (hereinafter Gur) US 2021/0019665 A1.
further in view Diao et.al (hereinafter Diao) US 7756535 B1,
further in view MENG et.al (hereinafter MENG ) US 2021/0312135 A1 (Foreign Priority CN 201910452828.3 Filed: 2019-05-28)
Regarding claim 21: (Previously Presented)
Adams, Rap and Gur do not explicitly disclose:
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified, and converting a word segmentation result into an input characteristic supported by a type of the classification prediction task;
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified,
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic; and predicting the batch processing characteristic based on the service branch.
However, Diao discloses:
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified, and converting a word segmentation result into an input characteristic supported by a type of the classification prediction task;
[Col 2, lines 62-67], [Col 3, lines 1-10]:
The conversion module 121 may comprise computer-readable program code for converting an incoming text message into a form suitable for processing by the prediction module 122. In one embodiment, the conversion module 121 parses a text message for presence of features included in the feature list 123. The features may comprise words or phrases indicative of a particular class (e.g., phishing, spam, porn, legitimate) of text messages. The conversion module 121 may identify features in the text message and convert the text message into a vector with the identified features as fundamental units (also referred to as "items"). Identification of features in the text message may be performed by string matching. Depending on what language the text message is in, the conversion module 121 may have to first segment the text message into individual words or phrases before parsing the text message for particular features.
- and converting a word segmentation result into an input characteristic supported by the model corresponding to a type of the classification prediction task;
[Col 3, lines 55-59]:
The word segmentation module 151 may comprise computer-readable program code for parsing sample text messages for the presence of particular words or phrases listed in the dictionary 155. The dictionary 155 may include words or phrases typically found in different classes of text messages.
[Col 3, lines 25-31]:
The representative vectors may each represent a typical text message of a particular class of text messages. For example, the learning model 124 may include a representative vector for a typical spam text message, another representative vector for a typical phishing text message, another representative vector for a legitimate message, and so on.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap, Gur and Diao.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Diao teaches word segmentation.
One of ordinary skills would be motivated to combine Adams, Rap, Gur and Diao to reduce the model file because of storing only the non-zero items (Diao [Col 9, lines 16-22])
Adams, Rap, Gur and Diao do not explicitly disclose:
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic; and predicting the batch processing characteristic based on the service branch.
However, MENG discloses:
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic
[0059]:
The server obtains a sentence, a voice, or a text to be subjected to sentiment analysis, that is, obtains an object (for example, a text) to be predicted and analyzed, i.e., the source data.
[0037]:
As shown in FIG. 1, a deep convolutional neural network (CNN) architecture is used for handling fine-grained sentiment analysis work, and local signs of a window section of Word Embeddings (e.g., text “sushi rolls are great”) are extracted. For the same local features extracted in different batches, some local features 1 are directly subjected to linear transformation 3, and the remaining local features 2 and a target feature vector, i.
[0037] :
After the encoding features of each local feature segment of Word Embeddings are screened 6, sentiment classification prediction is then performed through normalization and fully connected layer processing.
[0099]:
the server obtains the first forward hidden feature vector to the N.sup.th forward hidden feature vector; when t is not N, the server obtains a (t+1).sup.th reverse hidden feature vector in the source data, encodes the t.sup.th piece of sub-data based on the (t+1).sup.th reverse hidden feature vector, the target word feature vector, and the preset gated nonlinear transformation model to obtain a t.sup.th reverse hidden feature vector corresponding to the t.sup.th piece of sub-data,
[0099]:
the server respectively splices the first forward hidden feature vector to the N.sup.th forward hidden feature vector and the first reverse hidden feature vector to the N.sup.th reverse hidden feature vector in a one-to-one correspondence (i.e., the first forward hidden feature vector is spliced with the first reverse hidden feature vector, and the N.sup.th forward hidden feature vector is spliced with the N.sup.th reverse hidden feature vector), to obtain a first sub-feature vector to an N.sup.th sub-feature vector; and the server determines the first sub-feature vector to the N.sup.th sub-feature vector as the sub-feature vectors corresponding to the sub-data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap, Gur, Diao and Meng.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Diao teaches segmentation of text into words.
MENG teaches splicing input characteristics.
One of ordinary skill would be motivated to combine Adams, Rap, Gur, Diao and MENG to improve the quality of the word feature vector (MENG [0108])
Regarding claim 23: (Previously Presented)
Adams discloses:
- predicting the batch processing characteristic based on the service branch.
[Col 2, lines 38-45]:
The analytic server may separate the predictive model execution layer from the API layer and allocate system resources based on per-model request volume. Within the predictive model execution layer, each predictive computer model may reside in its own separate, containerized environment. This per-model containerized architecture may enable efficient resource allocation and per-model dependency management
[Col 8, lines 55-67]:
The batch/asynchronous queue worker 212 may support batch requests with each batch containing dozens of prediction requests.
[Col 12, lines 9-16]:
The analytic server may queue the batch requests based on the time they are received, and/or based on the level of priority. The analytic server may execute a plurality of predictive computer models corresponding to the plurality of electronic requests in the queue during off-business hours. The off-business hours may be the time when the workload of the analytic server is less than a threshold.
Adams, Rap, Gur do not explicitly disclose:
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified, and converting a word segmentation result into an input characteristic supported by a type of the classification prediction task;
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified,
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic; and predicting the batch processing characteristic based on the service branch.
However, Diao disclose:
- performing word segmentation respectively on text contents corresponding to the plurality of prediction objects to be classified, and converting a word segmentation result into an input characteristic supported by a type of the classification prediction task;
[Col 2, lines 62-67], [Col 3, lines 1-10]:
The conversion module 121 may comprise computer-readable program code for converting an incoming text message into a form suitable for processing by the prediction module 122. In one embodiment, the conversion module 121 parses a text message for presence of features included in the feature list 123. The features may comprise words or phrases indicative of a particular class (e.g., phishing, spam, porn, legitimate) of text messages. The conversion module 121 may identify features in the text message and convert the text message into a vector with the identified features as fundamental units (also referred to as "items"). Identification of features in the text message may be performed by string matching. Depending on what language the text message is in, the conversion module 121 may have to first segment the text message into individual words or phrases before parsing the text message for particular features.
- and converting a word segmentation result into an input characteristic supported by the model corresponding to a type of the classification prediction task;
[Col 3, lines 55-59]:
The word segmentation module 151 may comprise computer-readable program code for parsing sample text messages for the presence of particular words or phrases listed in the dictionary 155. The dictionary 155 may include words or phrases typically found in different classes of text messages.
[Col 3, lines 25-31]:
The representative vectors may each represent a typical text message of a particular class of text messages. For example, the learning model 124 may include a representative vector for a typical spam text message, another representative vector for a typical phishing text message, another representative vector for a legitimate message, and so on.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap , Gur and Diao..
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Dao teaches word segmenrtation.
One of ordinary skill would have motivation to combine Adams , Rap, Gur and Diao to reduce the model file as a result of storing only the non-zero items (Diao [Col 9, lines 16-22])
Adams, Rap, Gur and Diao do not explicitly disclose:
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic; and predicting the batch processing characteristic based on the service branch.
However, MENG discloses:
- splicing input characteristics respectively corresponding to the plurality of prediction objects to be classified to obtain a batch processing characteristic
[0059]:
The server obtains a sentence, a voice, or a text to be subjected to sentiment analysis, that is, obtains an object (for example, a text) to be predicted and analyzed, i.e., the source data.
[0037]:
As shown in FIG. 1, a deep convolutional neural network (CNN) architecture is used for handling fine-grained sentiment analysis work, and local signs of a window section of Word Embeddings (e.g., text “sushi rolls are great”) are extracted. For the same local features extracted in different batches, some local features 1 are directly subjected to linear transformation 3, and the remaining local features 2 and a target feature vector, i.
[0037]:
After the encoding features of each local feature segment of Word Embeddings are screened 6, sentiment classification prediction is then performed through normalization and fully connected layer processing.
[0099]:
the server obtains the first forward hidden feature vector to the N.sup.th forward hidden feature vector; when t is not N, the server obtains a (t+1).sup.th reverse hidden feature vector in the source data, encodes the t.sup.th piece of sub-data based on the (t+1).sup.th reverse hidden feature vector, the target word feature vector, and the preset gated nonlinear transformation model to obtain a t.sup.th reverse hidden feature vector corresponding to the t.sup.th piece of sub-data,
[0099]:
the server respectively splices the first forward hidden feature vector to the N.sup.th forward hidden feature vector and the first reverse hidden feature vector to the N.sup.th reverse hidden feature vector in a one-to-one correspondence (i.e., the first forward hidden feature vector is spliced with the first reverse hidden feature vector, and the N.sup.th forward hidden feature vector is spliced with the N.sup.th reverse hidden feature vector), to obtain a first sub-feature vector to an N.sup.th sub-feature vector; and the server determines the first sub-feature vector to the N.sup.th sub-feature vector as the sub-feature vectors corresponding to the sub-data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Adams, Rap, Gur, Diao and MENG.
Adams teaches prediction framework, model support for service, plurality of prediction object, generating an identifier for a task and self-defined variable prediction interface.
Rap teaches processing capacity for the task, and branch identifier.
Gur teaches initializing a universal batch classification prediction and storing primary keys.
Diao teaches segmentation of text into words.
MENG teaches splicing input characteristics.
One of ordinary skills would be to combine Adams, Rap, Gur, Diao and MENG to improve the quality of the word feature vector (MENG [0108]).
Conclusion
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to TIRUMALE KRISHNASWAMY RAMESH whose telephone number is (571)272-4605. The examiner can normally be reached by phone.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on phone (571-272-3768). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIRUMALE K RAMESH/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
