Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the following limitations:
1. A computer-implemented method for monitoring performance of a trained machine learning model, the method comprising:
computing a data drift score based on the at least one of the one or more first inputs or the one or more first outputs, the at least one of the one or more second inputs or the one or more second outputs, and a predefined policy (computing a data drift score in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper);
The claim recites an abstract idea.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application. The claim recites the following additional elements:
receiving at least one of one or more first inputs or one or more first outputs of the trained machine learning model during a first time period (amounts to mere data gathering, an insignificant extra-solution activity as discussed in MPEP 2106.05(g), which is well understood, routine and convention activity of receiving or gathering data as identified by the court in MPEP 2106.05(d));
receiving at least one of one or more second inputs or one or more second outputs of the trained machine learning model during a second time period (amounts to mere data gathering, an insignificant extra-solution activity as discussed in MPEP 2106.05(g), which is well understood, routine and convention activity of receiving or gathering data as identified by the court in MPEP 2106.05(d)); and
Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Step 2B: As shown above, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claim 2: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
2. The computer-implemented method of claim 1, wherein computing the data drift score comprises:
computing a first set of features based on the at least one of the one or more first inputs or the one or more first outputs (computing a first set of features in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper);
computing a second set of features based on the at least one of the one or more second inputs or the one or more second outputs (computing a second set of features in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper);
for each feature included in the first set of features, computing an intermediate data drift score based on the feature and a correspond feature included in the second set of features (computing an intermediate data drift score in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper); and
computing the data drift score based on the intermediate data drift scores and one or more weights defined in the predefined policy (computing data drift score in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites no additional element:
Step 2B: As shown above, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claims 3-5: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites additional element(s):
However, these elements amount to generally linking the abstract ideas to the technological environment or field of use as discussed in in MPEP 2106.05(h)).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claim 6: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites additional element(s):
6. The computer-implemented method of claim 1, further comprising generating one or more alerts based on the data drift score and a threshold defined in the predefined policy (amounts to mere insignificant application, an insignificant extra-solution activity as discussed in MPEP 2106.05(g), which is extra-solution activity of well, understood routine and conventional operation of applying it under MPEP 2106.05(d)).
Step 2B: As shown above, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claims 7-8: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites additional element(s):
However, these elements amount to generally linking the abstract ideas to the technological environment or field of use as discussed in in MPEP 2106.05(h)).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claim 9: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
9. The computer-implemented method of claim 1, wherein the data drift score is further computed based on data used to train the trained machine learning model (computing data drift score in high level is an observation, evaluation, judgment, opinion mental process which can reasonably be performed in one’s mind with the aid of pencil and paper).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites no additional element:
Step 2B: As shown above, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claim 10: Step 1: the claim is directed to statuary category.
Step 2A Prong 1: The claim recites the abstract idea of parent claim.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim recites additional element(s):
10. The computer-implemented method of claim 1, further comprising generating a user interface based on the data drift score (amounts to mere insignificant application, an insignificant extra-solution activity as discussed in MPEP 2106.05(g), which is extra-solution activity of well, understood routine and conventional operation of presentation of offer or statistics under MPEP 2106.05(d)).
Step 2B: As shown above, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The judicial exceptions are not integrated into a practical application.
The claim is not patent eligible.
Claims 11-19 are non-transitory computer readable storage medium claims having similar limitation as claims 1-9 and are rejected under the same rationale. The additional elements in claim 11 is One or more non-transitory computer-readable media storing program instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of (amounts to performing generic function of execution of stored instructions (MPEP 2106.05(f)). Accordingly, the additional elements do not integrate the abstract into practical application and are not sufficient to amount to significant more than the abstract idea. Therefore, the claims are an abstract idea.
Claim 20 is system claims having similar limitation as claim 1 and is rejected under the same rationale. The additional elements in claim 20 is A system, comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to (amounts to performing generic function of execution of stored instructions (MPEP 2106.05(f)). Accordingly, the additional elements do not integrate the abstract into practical application and are not sufficient to amount to significant more than the abstract idea. Therefore, the claims are an abstract idea.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-7, 9-12, 14-17, 19-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ganapavarapu et al (US 20230376825 A1)
1. A computer-implemented method for monitoring performance of a trained machine learning model ([0001] The present invention relates generally to artificial intelligence models, and more particularly to adaptive retraining of an artificial intelligence model by detecting a data drift, a concept drift, and a model drift.), the method comprising:
receiving at least one of one or more first inputs or one or more first outputs of the trained machine learning model during a first time period (Fig. 3-303. [0030] The disclosed system and method may replace the previously deployed AI model with the new AI model. Alternatively, the disclosed system and method may compare both the previously deployed AI model with the new AI model against each other using drift results over future period of data, and then the disclosed system and method select a winner as the production model.);
receiving at least one of one or more second inputs or one or more second outputs of the trained machine learning model during a second time period Fig. 3-302. [0027] In phase 2 (identifying required retraining data and creating a new AI model) mentioned above, in response to determining that the data drift is present, the disclosed system and method select new training data from the drifted period using known sampling techniques. In response to determining that the data drift is not present, the disclosed system and method use the original training data. Different strategies can be used to prepare new training data from the drifted period.);and
computing a data drift score based on the at least one of the one or more first inputs or the one or more first outputs, the at least one of the one or more second inputs or the one or more second outputs, and a predefined policy (Fig. 3-304. [0009] FIG. 4 illustrates an example of relationships between a data drift score, a concept drift score, a model drift score, and an overall drift score, in accordance with another embodiment of the present invention. Examiner Note: policy not further defined, reads on any policy/rules/computation that output the drift score. See [0033]).
2. The computer-implemented method of claim 1, wherein computing the data drift score comprises:
computing a first set of features based on the at least one of the one or more first inputs or the one or more first outputs ([0016] An AI system includes input data, predictive model and parameters, model predictions, ground truth, and various types of drift functions to analyze the data elements and drift thresholds. FIG. 1 is a diagram illustrating a problem setting of predictive model 110, in accordance with one embodiment of the present invention. Input sensor data (X) 120 is generated in periodic interval from multiple sensors installed in various assets and operations. Feature extracted data (X′) 130 is prepared from timeseries of input sensor data (X) 120 using feature extraction techniques. Predictive model or AI model (M) 110 may use either input sensor data (X) 120 or feature extracted data (X′) 130 for training. Predictive model or AI model (M) 110 is trained on input sensor data (X) 120 or feature extracted data (X′) 130 and outputs predictions (Ŷ) 140 in example 1 and/or predictions (Ŷ) 150 in example 2 for all the subsequent input data. In example 1, predictions (Ŷ) 140 is classification output; in example 2, predictions (Ŷ) 150 is regression output. In example 1, ground truth (Y) 160 is available for predictive model (M) 110; in example 2, ground truth (Y) 170 is available for predictive model (M) 110.);
computing a second set of features based on the at least one of the one or more second inputs or the one or more second outputs ([0016] An AI system includes input data, predictive model and parameters, model predictions, ground truth, and various types of drift functions to analyze the data elements and drift thresholds. FIG. 1 is a diagram illustrating a problem setting of predictive model 110, in accordance with one embodiment of the present invention. Input sensor data (X) 120 is generated in periodic interval from multiple sensors installed in various assets and operations. Feature extracted data (X′) 130 is prepared from timeseries of input sensor data (X) 120 using feature extraction techniques. Predictive model or AI model (M) 110 may use either input sensor data (X) 120 or feature extracted data (X′) 130 for training. Predictive model or AI model (M) 110 is trained on input sensor data (X) 120 or feature extracted data (X′) 130 and outputs predictions (Ŷ) 140 in example 1 and/or predictions (Ŷ) 150 in example 2 for all the subsequent input data. In example 1, predictions (Ŷ) 140 is classification output; in example 2, predictions (Ŷ) 150 is regression output. In example 1, ground truth (Y) 160 is available for predictive model (M) 110; in example 2, ground truth (Y) 170 is available for predictive model (M) 110.;
for each feature included in the first set of features, computing an intermediate data drift score based on the feature and a correspond feature included in the second set of features ([0003] In one aspect, a computer-implemented method for adaptive retraining of an artificial intelligence model is provided. The computer-implemented method includes computing drift magnitude scores for respective drift functions. The computer-implemented method further includes computing an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift. The computer-implemented method includes computing an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold. The computer-implemented method includes determining whether retraining of the artificial intelligence model is required, based on the overall drift score. The computer-implemented method includes performing the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required.); and
computing the data drift score based on the intermediate data drift scores and one or more weights defined in the predefined policy ([0003] In one aspect, a computer-implemented method for adaptive retraining of an artificial intelligence model is provided. The computer-implemented method includes computing drift magnitude scores for respective drift functions. The computer-implemented method further includes computing an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift. The computer-implemented method includes computing an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold. The computer-implemented method includes determining whether retraining of the artificial intelligence model is required, based on the overall drift score. The computer-implemented method includes performing the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required. See [0033], Fig. 7 on weights).
4. The computer-implemented method of claim 1, wherein the predefined policy specifies at least one of a set of features to compute based on the at least one of the one or more first inputs or the one or more first outputs and the at least one of the one or more second inputs or the one or more second outputs, an alert threshold, a frequency with which the data drift score is computed, or a portion of data for which the data drift score is computed ([0023] In embodiments of the present invention, any algorithm which is used to identify a drift in input data, a drift in ground truth, a drift in model predictions, or relationship among them is called a drift function. Based on the type of data that the algorithm considers, the drift function is categorized into one of the categories: the data drift functions, the concept drift functions, and the model drift functions. The proposed system and method re-categorize all the available drift functions ƒ.sub.1, ƒ.sub.2, ƒ.sub.3, . . . ƒ.sub.n into data drift functions, concept drift functions, and model drift functions as per drift definitions. Drift thresholds, including a drift threshold (τ.sub.ƒ.sub.i) for a drift function ƒ.sub.i, a data drift threshold (τ.sub.dd), a concept drift threshold (τ.sub.cd), and a model drift threshold (τ.sub.md), can be set either by predefined values or by users. Examiner Note: each threshold is considered an alert threshold).
5. The computer-implemented method of claim 1, wherein the at least one of the one or more first inputs or the one or more first outputs includes a user-specified portion of input into the machine learning model or output of the machine learning model during the first time period ([0023] In embodiments of the present invention, any algorithm which is used to identify a drift in input data, a drift in ground truth, a drift in model predictions, or relationship among them is called a drift function. Based on the type of data that the algorithm considers, the drift function is categorized into one of the categories: the data drift functions, the concept drift functions, and the model drift functions. The proposed system and method re-categorize all the available drift functions ƒ.sub.1, ƒ.sub.2, ƒ.sub.3, . . . ƒ.sub.n into data drift functions, concept drift functions, and model drift functions as per drift definitions. Drift thresholds, including a drift threshold (τ.sub.ƒ.sub.i) for a drift function ƒ.sub.i, a data drift threshold (τ.sub.dd), a concept drift threshold (τ.sub.cd), and a model drift threshold (τ.sub.md), can be set either by predefined values or by users).
6. The computer-implemented method of claim 1, further comprising generating one or more alerts based on the data drift score and a threshold defined in the predefined policy ([0023] In embodiments of the present invention, any algorithm which is used to identify a drift in input data, a drift in ground truth, a drift in model predictions, or relationship among them is called a drift function. Based on the type of data that the algorithm considers, the drift function is categorized into one of the categories: the data drift functions, the concept drift functions, and the model drift functions. The proposed system and method re-categorize all the available drift functions ƒ.sub.1, ƒ.sub.2, ƒ.sub.3, . . . ƒ.sub.n into data drift functions, concept drift functions, and model drift functions as per drift definitions. Drift thresholds, including a drift threshold (τ.sub.ƒ.sub.i) for a drift function ƒ.sub.i, a data drift threshold (τ.sub.dd), a concept drift threshold (τ.sub.cd), and a model drift threshold (τ.sub.md), can be set either by predefined values or by users. [0019] Embodiments of the present invention disclose a system and method for quantifying different types of drifts in training artificial intelligence (AI) models, e.g., the data drift, the concept drift, and the model drift, using proper drift functions/algorithms. The disclosed system and method detect the drifts and notify users to retrain the AI models when required, using combined analysis of the above drifts. The disclosed system and method select appropriate data required for retraining using the above analysis and perform retraining of the AI model. Examiner Note: the notification is an alert).
7. The computer-implemented method of claim 6, wherein the threshold is set based on a number of alerts that are expected to be generated using the threshold ([0023] In embodiments of the present invention, any algorithm which is used to identify a drift in input data, a drift in ground truth, a drift in model predictions, or relationship among them is called a drift function. Based on the type of data that the algorithm considers, the drift function is categorized into one of the categories: the data drift functions, the concept drift functions, and the model drift functions. The proposed system and method re-categorize all the available drift functions ƒ.sub.1, ƒ.sub.2, ƒ.sub.3, . . . ƒ.sub.n into data drift functions, concept drift functions, and model drift functions as per drift definitions. Drift thresholds, including a drift threshold (τ.sub.ƒ.sub.i) for a drift function ƒ.sub.i, a data drift threshold (τ.sub.dd), a concept drift threshold (τ.sub.cd), and a model drift threshold (τ.sub.md), can be set either by predefined values or by users.).
9. The computer-implemented method of claim 1, wherein the data drift score is further computed based on data used to train the trained machine learning model (Fig.2 -203, [0003] In one aspect, a computer-implemented method for adaptive retraining of an artificial intelligence model is provided. The computer-implemented method includes computing drift magnitude scores for respective drift functions. The computer-implemented method further includes computing an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift. The computer-implemented method includes computing an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold. The computer-implemented method includes determining whether retraining of the artificial intelligence model is required, based on the overall drift score. The computer-implemented method includes performing the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required.
10. The computer-implemented method of claim 1, further comprising generating a user interface based on the data drift score [0084] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.).
Claims 11-12, 14-17, 19 are non-transitory computer readable storage medium claims having similar limitation as claims 1-2, 4-7, 9 and are rejected under the same rationale. The additional elements in claim 11 is One or more non-transitory computer-readable media storing program instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of ([0004] In another aspect, a computer program product for adaptive retraining of an artificial intelligence model is provided. The computer program product comprises a computer readable storage medium having program instructions embodied therewith, and the program instructions are executable by one or more processors. The program instructions are executable to: compute drift magnitude scores for respective drift functions; compute an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift; compute an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold; determine whether retraining of the artificial intelligence model is required, based on the overall drift score; and perform the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required.)
Claim 20 is system claims having similar limitation as claim 1 and is rejected under the same rationale. The additional elements in claim 20 is A system, comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to ([0005] In yet another aspect, a computer system for adaptive retraining of an artificial intelligence model is provided. The computer system comprises one or more processors, one or more computer readable tangible storage devices, and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors. The program instructions are executable to compute drift magnitude scores for respective drift functions. The program instructions are further executable to compute an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift. The program instructions are further executable to compute an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold. The program instructions are further executable to determine whether retraining of the artificial intelligence model is required, based on the overall drift score. The program instructions are further executable to perform the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 3, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ganapavarapu et al (US 20230376825 A1) in view of Galle et al (US 20230109734 A1)
Claims 3 and 13. While Ganapavarapu disclose drift detection, Ganapavarapu fails to apply it to features includes at least one of a sentiment of a question, a length of a question, a complexity of an answer, a cluster of embeddings of questions or answers, or a user behavior.
However, Galle disclose drift detection (thereby in the same field of endeavor), and apply it to features includes at least one of a sentiment of a question, a length of a question, a complexity of an answer, a cluster of embeddings of questions or answers, or a user behavior ([0033] The drift detection engine 122 combines several metrics that capture linguistic drift, including the change in the distribution over predictions that the model makes, the change in the number of articles tagged positively, the rate of incidence in positively-labeled documents of the top (e.g., 50) words or phrases that the concept model associates with the concept, and other factors that determine whether the language associated with the concept has changed significantly since the time that the model was initially trained. If the drift metric exceeds a threshold, retraining is triggered. As an example, the drift detection engine 122 can trigger a retraining when the number of articles tagged positively for a concept has moved above or below two standard deviations (calculated over the year before the classifier was trained) for 10 days in the last month. As another example, the drift detection engine 122 may trigger a retraining of the present model when the Kullback-Leibler divergence between the distribution of predictions the present model made for the year before the classifier was trained and the distribution of predictions the present model made in the last month is above a specified threshold. [0044] The process continues by scoring new documents 320. The term “drift” and the term “difference” are used interchangeably herein and can describe a divergence between a model's linguistic calibration for a particular concept (learned at training time) and the way that the concept is discussed in current documents. As noted above with reference to FIG. 1, the system can include a threshold for drift, which triggers a concept model retraining if exceeded, and can be called a drift detection model. Some reasons for drift might be: Cyclical/seasonal effects—the language model (order and frequency of words) associated with a concept may change cyclically. For example, articles about the economy might be written differently depending on whether the economy is experiencing a bull market or a recession; articles about national environmental issues might differ based on whether there is a Democratic or Republican president; and articles about politics may differ based on whether the country is involved in a presidential election cycle.)
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to extend the application of drift detection of Ganapavarapu to incorporate the application into the field of language model or text classification.
Given the fact that language model or text classification is just one of the many application one can apply drift detection to, one having ordinary skill in the art would have been motivated to make this obvious modification with predictable result of wherein the first set of features includes at least one of a sentiment of a question, a length of a question, a complexity of an answer, a cluster of embeddings of questions or answers, or a user behavior.
Claim(s) 8, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ganapavarapu et al (US 20230376825 A1) in view of Umesh et al (US 20230124621 A1)
Claims 8 and 18. While Ganapavarapu disclose drift detection over time period, Ganapavarapu fails to explicitly disclose time period is a week-to-date, a month-to-date, a quarter-to-date, or a year-to-date time period, and the second time period is a previous week, a previous month, a previous quarter, or a previous year time period.
However, Umesh disclose drift detection (thereby in the same field of endeavor), and disclose time period is a week-to-date, a month-to-date, a quarter-to-date, or a year-to-date time period, and the second time period is a previous week, a previous month, a previous quarter, or a previous year time period ([0070] Additionally, or alternatively, S210 may function to identify an anomalous behavior of a threat scoring ensemble based on an evaluation involving a trend test (e.g., a Mann-Kendall test or the like) of threat score values predicted by the threat scoring ensemble over a selected period. In one example, S210 may function to collect daily threat score distributions of a target threat score ensemble over a period (e.g., a target week, a target month, a target quarter, or any suitable time span) and S210 may function to identify whether one or more trends (e.g., upward trend, downward trend, etc.) in the movement of the threat score exist in the threat score distribution data associated with the target threat score ensemble. Accordingly, if a trend (upward, downward, etc.) is detected within the threat score distribution data of the target score ensemble, S210 may function to identify the threat scoring ensemble as an anomalous ensemble, a drift-experiencing machine learning-based ensemble, or the like.).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that time period can be any of week/month/quarter/year/etc.
Given the fact time period can be any of week/month/quarter/year/etc., one having ordinary skill in the art would have been motivated to make this obvious modification with predictable result of wherein the first time period is a week-to-date, a month-to-date, a quarter-to-date, or a year-to-date time period, and the second time period is a previous week, a previous month, a previous quarter, or a previous year time period.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Naili et al (US 20240171979 A1) disclose data drift score. See [0056].
Ackerman et al (“Automatically detecting data drift in machine learning classifiers” 2021) disclose automatically detecting data drift in machine learning classifiers. See abstract.
Suárez-Cetrulo et al (“A survey on machine learning for recurring concept drifting data streams” 2022) disclose a survey on machine learning for recurring concept drifting data streams. See abstract.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUT WONG whose telephone number is (571)270-1123. The examiner can normally be reached M-F 10am-6pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at 5712703169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUT WONG/Primary Examiner, Art Unit 2127
Prosecution Insights