DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-13 are pending.
Drawings
The drawings remain objected to because replacement Fig. 3 is not is not of high enough quality to be scanned, copied, or printed properly. The image is low resolution/contrast, and the text is illegible. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application.
Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more.
Step 1 (The Statutory Categories): Is the claim to a process, machine, manufacture, or composition of matter? MPEP 2106.03.
Per Step 1, claim 1 is to a method (i.e., a process), claim 7 to a system (i.e., a machine). Thus, the claims are directed to statutory categories of invention. However, the claims are rejected under 35 U.S.C. 101 because they are directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The analysis proceeds to Step 2A Prong One.
Step 2A Prong One: Does the claim recite an abstract idea, law of nature, or natural phenomenon? MPEP 2106.04.
The abstract idea of claims 1 and 7 is (claim 1 being representative):
receiving at least one first dataset, wherein the at least one first dataset comprises at least one labeled dataset and at least one unlabeled dataset;
processing each of a plurality of first sub-datasets included in the at least one labeled dataset to generate at least one first meta feature from the at least one labeled dataset, wherein the at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets;
correlating the at least one first meta feature with a prebuilt model comprising a plurality of classification models, wherein the prebuilt model further comprises at least one mapping function for mapping at least one pre-calculated meta feature with a plurality of pre-calculated classification performance scores corresponding to the plurality of classification models;
estimating a classification performance score of each of the plurality of classification models for the at least one labeled dataset, based on corelating the at least one first meta feature with the prebuilt model;
generating a list comprising the plurality of classification models arranged in descending order of the estimated classification performance scores; and
selecting a predefined number of top classification models from the list to build an ensemble classification model for classifying the at least one unlabeled dataset.
Steps a-f are those which could be performed mentally, including with pen and paper. The steps describe a multi-stage process for selecting the best classification models by analyzing data characteristics, estimating performance via pre-calculated mapping, and ranking models for an ensemble approach. The steps focus on abstract calculations, mental comparisons, and decision-making for model selection, i.e., mental processes. These steps mirror cognitive tasks, such as analyzing, comparing, and selecting. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, including observations, evaluations, judgements, and/or opinions, then it falls within the Mental Processes – Concepts Performed in the Human Mind grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Additionally and alternatively, steps a-f describe the mathematical calculations that comprise algorithmic, statistical, and numerical manipulations. Specifically, the steps describe calculating meta features as cluster indices, correlating them with a prebuilt model via mapping functions, and applying regression or ranking algorithms to estimate model performance scores. If a claim limitation, under its broadest reasonable interpretation, covers mathematical concepts, including mathematical relationships, mathematical formulas or equations, mathematical calculations, then it falls within the Mathematical Concepts grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application? MPEP 2106.04.
This judicial exception is not integrated into a practical application because the additional elements are merely instructions to apply the abstract idea to a computer, as described in MPEP 2106.05(f).
Claim 1 recites no additional elements.
Claim 7 recites the following additional elements: a memory; at least one processor communicatively coupled with the memory, wherein the at least one processor is configured to.
These elements are merely instructions to apply the abstract idea to a computer, per MPEP 2106.05(f). Applicant has only described generic computing elements in their specification, as seen in [0054]-[0060] of applicant’s specification as filed, for example. (Paragraph starting with “Now, Figure 1 is explained in conjunction with Figure 2, which is a block diagram 200 of the communication system 100, in accordance with some embodiments of the present disclosure.”)
Further, the combination of these elements is nothing more than a generic computing system applied to the tasks of the abstract idea. Because the additional elements are merely instructions to apply the abstract idea to a generic computing system, they do not integrate the abstract idea into a practical application, when viewed in combination. See MPEP 2106.05(f).
Therefore, per Step 2A Prong Two, the additional elements, alone and in combination, do not integrate the judicial exception into a practical application. The claim is directed to an abstract idea.
Step 2B (The Inventive Concept): Does the claim recite additional elements that amount to significantly more than the judicial exception? MPEP 2106.05.
Step 2B involves evaluating the additional elements to determine whether they amount to significantly more than the judicial exception itself.
The examination process involves carrying over identification of the additional element(s) in the claim from Step 2A Prong Two and carrying over conclusions from Step 2A Prong Two pertaining to MPEP 2106.05(f).
The additional elements and their analysis are therefore carried over: applicant has merely recited elements that facilitate the tasks of the abstract idea, as described in MPEP 2106.05(f).
Further, the combination of these elements is nothing more than a generic computing system applied to the tasks of the abstract idea. When the claim elements above are considered, alone and in combination, they do not amount to significantly more.
Therefore, per Step 2B, the additional elements, alone and in combination, are not significantly more. The claims are not patent eligible.
The analysis takes into consideration all dependent claims as well:
Dependent claims 2-6 and 8-13 further narrow the abstract idea with additional steps and/or information. This narrowing of the abstract idea doesn’t integrate it into practical application and/or add significantly more. Some of the dependent claims recite further additional elements, beyond those highlighted above:
Claims 2 and 8: using the ensemble classification model.
Claims 3 and 9: using at least one clustering model.
Claims 6 and 12: by tuning one or more hyper parameters corresponding to the plurality of classification models.
Claim 13: provide a Machine Learning as a service (MLaaS) platform for data classification and classification model selection.
Similar to above, these are generic computing elements and processes, claimed in a results-oriented manner, which do no more than facilitate the tasks of the narrowed abstract idea. Whether viewed alone or in combination, they do not integrate the abstract idea into practical application and/or add significantly more. See MPEP 2106.05(f).
Accordingly, claims 1-13 are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Response to Arguments
Applicant’s arguments filed 8/5/26 have been carefully considered. Examiner’s response follows.
I. Objection to the Drawings
Applicant’s figure 3 remains objected to for the reasons set forth above.
II. Rejection of Claims 1-13 under 35 U.S.C. §101
Applicant offers:
Claims 1-13 have been rejected under 35 U.S.C. §101 because the claimed invention is purportedly directed to an abstract idea without significantly more. Applicant traverses this rejection for the following reasons.
The USPTO approach to analyzing the Supreme Court's Alice test splits the first step of the Alice test into two prongs. Under the first prong, "any judicial exceptions, including certain groupings of abstract ideas (i.e., mathematical concepts, certain methods of organizing human activity such as a fundamental economic practice, or mental processes)" are identified. Under the second prong, the remaining/additional elements are analyzed to determine if the claim integrates those elements identified under prong 1 into a "practical application." See MPEP § 2106.05(a)-(c), (e)-(h).
The Office Action rejects claims 1-13 under 35 U.S.C. §101 because the claimed invention is purportedly directed to an abstract idea of Mental Concept under Step 2A, Prong One, alleging that the claim features covers performance of the features in the mind but for the recitation of computer components (Office Action, page 4, lines 13-21). However, the claims cannot recite a mental process when the claim features cannot be practically performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations. See MPEP 2106.04(a)(2) citing SRI Int'l, Inc. v. Cisco Systems, Inc., 930 F.3d-8-1295, 1304 (Fed. Cir. 2019). Here, claim 1 is directed to a solution for conventional techniques for selecting a classification model. The problem of the conventional techniques is described with respect to paragraph [0048] as published in corresponding U.S. Patent Publication No. 2024/0242131 (hereinafter "the specification"). Specifically, the conventional techniques are time consuming and resource intensive and it is difficult to perform accurate classification of a huge dataset in real time using the conventional techniques.
The solution recited in claim 1 is computer based since the problem is that human operators cannot mentally process the massive amounts of data. For example, to overcome this and other problems, claim 1 recites the use of clustering indices. The clustering indices are used for automatically selecting one or more classification models from a plurality of available classification models to form an ensemble classification model (Specification, [0049]). The ensemble classification model may be used for accurately classifying a given dataset (Id.). The present disclosure uses clustering indices as meta features for selecting best classification models to build the ensemble classification model without fitting/training the plurality of classification models over the dataset (Id.). Thus, claim 1 is firmly rooted in computer technologies to perform computer-based functions and cannot be practically performed in the human mind. It is precisely because humans cannot cope with the ever-increasing amount of data that are being continuously generated, using mental processes that the claims recite a computer-based solution, the claimed functions cannot be considered as a mental process. For at least these reasons, independent claim 1 is not directed to a mental process and is not directed to an abstract idea.
The Examiner additionally asserts that the claims are directed to mathematical concepts (Office Action, page 5, lines 2-5). Accordingly, the Examiner has dissected the claim to assert multiple abstract ideas. While an Examiner may identify the claim as reciting both a mental process and a mathematical concept for Step 2A Prong One to make the analysis clear on the record, "the examiner should consider the limitations together as a single abstract idea for Step 2A Prong Two and Step 2B (if necessary) rather than as a plurality of separate abstract ideas to be analyzed individually" (See MPEP 21-6.04, subsection II.B.).
Here, the claim does not specify any mathematical formula, calculations, or relationships. Instead, claim 1 recites receiving at least one first dataset and processing each of a plurality of first sub-datasets included in the at least one labeled dataset to generate at least one first meta feature from the at least one labeled dataset. The at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets. The at least one first meta feature is correlated with a prebuilt model comprising a plurality of classification models. The prebuilt model includes at least one mapping function for mapping at least one pre-calculated meta feature with a plurality of pre-calculated classification performance scores corresponding to the plurality of classification models. A classification performance score of each of the plurality of classification models is estimated for the at least one labeled dataset, based on corelating the at least one first meta feature with the prebuilt model. A list comprising the plurality of classification models arranged in descending order of the estimated classification performance scores is generated. A predefined number of top classification models is selected from the list to build an ensemble classification model for classifying the at least one unlabeled dataset. The method recited in claim 1 is not a method of organizing human activity because it does not fall within the enumerated sub-groupings of fundamental economic principles or practices, commercial or legal interactions, and managing personal behavior and relationships or interactions between people. While the processing and estimating may involve a mathematical calculation, the claim does not specify any mathematical formula, calculations, or relationships. Therefore, these features of claim 1 are neither mathematical concepts nor mental processes.
While well taken, examiner remains unpersuaded by these remarks. In particular, examiner disagrees with applicant’s conclusion, i.e., “Thus, claim 1 is firmly rooted in computer technologies to perform computer-based functions and cannot be practically performed in the human mind. It is precisely because humans cannot cope with the ever-increasing amount of data that are being continuously generated, using mental processes that the claims recite a computer-based solution, the claimed functions cannot be considered as a mental process.”
It's important to consider the actual language of the abstract idea, which includes:
receiving at least one first dataset, wherein the at least one first dataset comprises at least one labeled dataset and at least one unlabeled dataset;
processing each of a plurality of first sub-datasets included in the at least one labeled dataset to generate at least one first meta feature from the at least one labeled dataset, wherein the at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets;
correlating the at least one first meta feature with a prebuilt model comprising a plurality of classification models, wherein the prebuilt model further comprises at least one mapping function for mapping at least one pre-calculated meta feature with a plurality of pre-calculated classification performance scores corresponding to the plurality of classification models;
estimating a classification performance score of each of the plurality of classification models for the at least one labeled dataset, based on corelating the at least one first meta feature with the prebuilt model;
generating a list comprising the plurality of classification models arranged in descending order of the estimated classification performance scores; and
selecting a predefined number of top classification models from the list to build an ensemble classification model for classifying the at least one unlabeled dataset.
These steps are not necessarily technical. An administrator could perform these tasks, including processing, correlating, and estimating, with pen and paper. Applicant seems to suggest that the tasks would be too cumbersome for an individual to perform either mentally and/or with pen and paper; however, this is not the standard for eligibility. Instead, the fundamental question being asked is do the claims articulate a technological solution to a technological problem reflected in the specification? The problem here, if one exists, pertains to selecting a predefined number of top classification models, i.e., an abstract idea. Applicant’s claim does not recite, in any detail, the steps pertaining to the actual construction of an ensemble classification model and/or its training, contrary to any assertions.
The Mathematical Concepts grouping also appears relevant, at least in view of steps b-d, which describe: calculating at least one first cluster index; correlating the at least one first meta feature with a prebuilt model comprising a plurality of classification models; at least one mapping function; and estimating a classification performance score. The use of mathematical calculations to accomplish these steps is self-evident.
Applicant continues:
The Examiner asserts on page 5 of the Office Action that the claimed invention is directed to an abstract idea by alleging the claims do not integrate the abstract idea into a practical application (Office Action, lines 8-10). In the present application, the features of the amended claim 1 are integrated into a practical application by providing an improvement on computer-based ensemble classification model. Claim 1 does not merely collect and compare data. The system and methods claimed in the present application provide a solution to the problems of the conventional technology and by using clustering indices as meta features for selecting best classification models to build the ensemble classification model without fitting/training the plurality of classification models over the dataset (Specification, [0049]).
Claim 1 has been amended to further clarify the improvement. Support for the amendment can be found in at least paragraphs [0044], [0066], [0076]-[0085], [0104], and [0105] of the specification. No new matter has been added. Amended claim 1 recites that each of a plurality of first sub-datasets are processed and the at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets. It is these use clustering indices that are used for automatically selecting one or more classification models from a plurality of available classification models to form an ensemble classification model. Accordingly, the claims are directed to a technical improvement to conventional technologies.
MPEP 2106.05(a)(III) states:
If it is asserted that the invention improves upon conventional functioning of a computer, or upon conventional technology or technological processes, a technical explanation as to how to implement the invention should be present in the specification. That is, the disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. An indication that the claimed invention provides an improvement can include a discussion in the specification that identifies a technical problem and explains the details of an unconventional technical solution expressed in the claim, or identifies technical improvements realized by the claim over the prior art. For example, in McRO, the court relied on the specification's explanation of how the particular rules recited in the claim enabled the automation of specific animation tasks that previously could only be performed subjectively by humans, when determining that the claims were directed to improvements in computer animation instead of an abstract idea. McRO, 837 F.3d at 1313-14, 120 USPQ2d at 1100-01. In contrast, the court in Affinity Labs of Tex. v. DirecTV, LLC relied on the specification's failure to provide details regarding the manner in which the invention accomplished the alleged improvement when holding the claimed methods of delivering broadcast content to cellphones ineligible. 838 F.3d 1253, 1263-64, 120 USPQ2d 1201, 1207-08 (Fed. Cir. 2016). (Emphasis added).
The specification describes a problem with conventional detection technology and an improvement to this problem, and the claims recite the improvement. Claim 1 thus recites an improvement in the technological field and should be deemed patent eligible for providing a practical application under Step 2A, Prong 2. Accordingly, even assuming arguendo that the present claims recite an abstract idea that falls within one of the enumerated subject matter groupings (a position that is not conceded), the claimed invention of claim 1 is an improvement to technology or technical field of computer-based technology. The features of claim 1 ground the claims into a practical application such that the claim are not directed to the judicial exception and independent claim 1 is patent eligible. Claims 2-8 depend from independent claim 1 and are patent eligible by virtue of their dependence from patent eligible independent claim 1.
Claim 7 has been amended in a similar manner as amended claim 1. For at least the same reasons as set forth above, it is respectfully submitted that the independent claim 7 recites patent eligible subject matter under 35 U.S.C. § 101 and requests withdrawal of the rejection to these claims. Claims 8-13 depend from independent claim 7 and are patent eligible by virtue of their dependence from a patent eligible independent claim. In view of the forgoing, claims 1-13 recite patent-eligible subject matter pursuant to 35 U.S.C. §101. Accordingly, withdrawal of this rejection is respectfully requested.
Examiner disagrees. At Step 2A Prong Two and Step 2B, the questions being asked are 1) do the additional elements, alone or in combination, integrate the abstract idea into practical application; and 2) do the additional elements, alone or in combination, add significantly more to the abstract idea?
In this instance, the additional elements (e.g., memory; at least one processor communicatively coupled to the memory, wherein the at least one processor is configured to) are merely instructions to apply the abstract idea to a computer, as described in MPEP 2106.05(f). Applicant has only described generic computing elements in their specification, as seen in [0054]-[0060] of applicant’s specification as filed, for example. (Paragraph starting with “Now, Figure 1 is explained in conjunction with Figure 2, which is a block diagram 200 of the communication system 100, in accordance with some embodiments of the present disclosure.”) MPEP 2106.05(f) is clear that the application of generic computing devices and/or machinery do not integrate the abstract idea into practical application or add significantly more. Thus, contrary to applicant’s assertion, the claimed invention does not represent an improvement to technology.
Accordingly, examiner maintains the rejections under 35 U.S.C. §101.
III. Rejection of Claims 1-4, 7-10, and 13 under 35 U.S.C. §103
Regarding the rejections under 35 U.S.C. §103, applicant’s amendments and clarifying remarks are persuasive. These rejections are withdrawn.
In particular, Vilalta does not teach or suggest processing each of a plurality of first sub-datasets to generate at least one first meta feature that is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets, as recited in amended claim 1. Instead, in Vilalta, records 305-320 correspond to different object in an image of the dataset and the features 350-365 would correspond to a different object parameters, such as the amount of luminosity, shape or size (Col. 5, lines 50-55). Statistics are extracted from the dataset 300 regarding the objects in the images to generate the meta-features (Col. 6, lines 9- 12). The meta-features measure the degree of variation between the objects (Col. 6, lines 18-29). Thus, in Vilalta, all of the features 350-365 are generated based on the objects identified in the domain dataset 300. Accordingly, Vilalta merely teaches a single dataset. Vilalta does not teach or suggest processing each of a plurality of first sub-datasets included in the at least one labeled dataset to generate at least one first meta feature from the at least one labeled dataset.
Furthermore, the features disclosed in Vilalta are merely objects (e.g., stars, galaxies, etc.) of the data. For example, if the domain dataset 300 were to correspond to astronomical images to be classified as either stars or galaxies, then each record 305-320 would correspond to a different object in the image, and each field 350-365 would correspond to a different feature such as the amount of luminosity, shape or size (Vilalta, col. 5, lines 47-53). Therefore, the features of Vilalta are not meta features which are characteristics about the data. Instead, the features identify specific objects in the dataset which are then labeled in the class field 370 (Col. 5, lines 53-54). Accordingly, the features of Vilalta do not teach or suggest at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets. Instead, the features define specific objects in the dataset and the meta-features are calculated for those specific objects.
Agrawal and Givental do not cure the deficiencies of Vilalta. Agrawal is cited for teaching the selection of an algorithm (Office Action, page 10, lines 3-8). However, Agrawal does not teach or suggest at least one first meta feature is at least one first cluster index calculated based on a plurality of sub-cluster indexes corresponding to each of the plurality of first sub-datasets. Instead, Agrawal is directed to the selection of machine learning algorithms based on performance predictions by trained algorithm-specific regressors (Agrawal, [0001]).
In an updated search, examiner identified the following references, which, while generally relevant to the field of endeavor, stop short of the specificity required by the claim:
US 20140188768, which teaches: [0088] The above-described systems and methods provide a way to create customize model ensembles on demand. The embodiments described herein allow for selecting a customized set of models from a database of models. The database of models also includes metadata about the models. The metadata relating to the models includes information clarifying appropriateness of each particular model to a given query such that, at the time of the query, each model's applicability may be weighed against that exact query. Models are selected based on the query, i.e., local models within the query's feature space are used in order to increase the accuracy of each model's predictions. The individual results of each model within the model ensemble are combined, creating an aggregate result from multiple models rather than relying on the best single model. Metadata regarding each model's applicability to the particular query is again used during the combination of the individual results, both in determining the amount of bias for which to compensate, as well as in weighing each individual model's result, i.e., based on that particular model's individual applicability to the query.
US 20170161613, which teaches: [0032] As can be seen in FIG. 2, the auto-classification system includes a variety of computing components that work together. The structure and function of the elements shown in FIG. 2 will be discussed in greater detail below. However, in brief, a model orchestrator 202 segments the incoming dataset into processing buckets based on input data quality of various records, invokes the models from model registry in order (e.g., highest to lowest data quality) and performs adaptive ensembling on the results of the model runs to get to the final outcome. A model repository includes a mapping between given model stack configuration, applicable level of information uncertainty and data quality, performance baselines in this context, as well as run-time estimates for the same. The model stacks 204a-204n are the actual classification and enrichment models that work on the input data (which, in this example scenario, is spend data). The model repository thus includes a model registry 206 that stores information about the model stacks 204a-204n, as well as a store 208 for data and/or metadata about the model training, baselining, and model refinement. A reference data universe 210 is used for model training and refinement. This includes, for example, an item data store 210a, a supplier store 210b, a context store 210c, a transaction (or spend) store 210d, and a store or interface for receiving a feed of third-party data 210e.
US 20190197357, which teaches: [0053] In one embodiment, as will be further described in this document, these policy/parameter thresholds are used as the base for mapping training data to inference uses, as facilitated by trust mapping engine 203. Any discrepancies between the training data and the inference uses are analyzed and classified as one or more of minor misuses, major misuses, errors, exceptions, exemptions, etc., as facilitated by analysis and classification logic 213. Any data relating to misuses is then forwarded on to misuse index estimation logic 215 to estimate an index according to the severity and/or importance of misuses so that appropriate actions may be recommended and/or taken to stop a current misuse, prevent a future misuse, etc.
Accordingly, the rejections under 35 U.S.C. §103 are withdrawn.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20140188768, which teaches: [0088] The above-described systems and methods provide a way to create customize model ensembles on demand. The embodiments described herein allow for selecting a customized set of models from a database of models. The database of models also includes metadata about the models. The metadata relating to the models includes information clarifying appropriateness of each particular model to a given query such that, at the time of the query, each model's applicability may be weighed against that exact query. Models are selected based on the query, i.e., local models within the query's feature space are used in order to increase the accuracy of each model's predictions. The individual results of each model within the model ensemble are combined, creating an aggregate result from multiple models rather than relying on the best single model. Metadata regarding each model's applicability to the particular query is again used during the combination of the individual results, both in determining the amount of bias for which to compensate, as well as in weighing each individual model's result, i.e., based on that particular model's individual applicability to the query.
US 20170161613, which teaches: [0032] As can be seen in FIG. 2, the auto-classification system includes a variety of computing components that work together. The structure and function of the elements shown in FIG. 2 will be discussed in greater detail below. However, in brief, a model orchestrator 202 segments the incoming dataset into processing buckets based on input data quality of various records, invokes the models from model registry in order (e.g., highest to lowest data quality) and performs adaptive ensembling on the results of the model runs to get to the final outcome. A model repository includes a mapping between given model stack configuration, applicable level of information uncertainty and data quality, performance baselines in this context, as well as run-time estimates for the same. The model stacks 204a-204n are the actual classification and enrichment models that work on the input data (which, in this example scenario, is spend data). The model repository thus includes a model registry 206 that stores information about the model stacks 204a-204n, as well as a store 208 for data and/or metadata about the model training, baselining, and model refinement. A reference data universe 210 is used for model training and refinement. This includes, for example, an item data store 210a, a supplier store 210b, a context store 210c, a transaction (or spend) store 210d, and a store or interface for receiving a feed of third-party data 210e.
US 20190197357, which teaches: [0053] In one embodiment, as will be further described in this document, these policy/parameter thresholds are used as the base for mapping training data to inference uses, as facilitated by trust mapping engine 203. Any discrepancies between the training data and the inference uses are analyzed and classified as one or more of minor misuses, major misuses, errors, exceptions, exemptions, etc., as facilitated by analysis and classification logic 213. Any data relating to misuses is then forwarded on to misuse index estimation logic 215 to estimate an index according to the severity and/or importance of misuses so that appropriate actions may be recommended and/or taken to stop a current misuse, prevent a future misuse, etc.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN SAMUEL WASAFF whose telephone number is (571)270-5091. The examiner can normally be reached Monday through Friday 8:00 am to 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SARAH MONFELDT can be reached at (571) 270-1833. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JOHN SAMUEL WASAFF
Primary Examiner
Art Unit 3629
/JOHN S. WASAFF/ Primary Examiner, Art Unit 3629