DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on June 22, 2023. Claims 1-20 are pending in the case. Claims 1, 10, and 17 are the independent claims.
This action is non-final.
Claim Objections
Claims 16 is objected to because of the following informalities: Claim 16 (a method claim) recites dependency upon claim 9 (a system claim) where a recitation on claim 10 (a method claim) was perhaps intended. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: an extraction component in claim 2, a trust monitor component in claim 7, a trust model manager component in claim 8. Examiner notes that claim 1 additionally recites computer executable components which comprise multiple additional components. However, the various components are all recited as being part of the computer executable components, which are stored in memory and executed by a processor. Therefore, the components of claim 1 are considered to be recited with sufficient structure to not invoke 35 USC 112(f). However, the components of claims 2, 7, and 8 discussed above, are only recited as being components of the system, and not as being included within the computer executable components. Therefore, these are not considered to be recited with sufficient structure to avoid invoking 35 USC 112(f).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental steps) without significantly more. This judicial exception is not integrated into a practical application because any additional elements amount to implementing the abstract idea on a generic computer. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding independent claims 1, 10, and 17, and relying on the evaluation flowchart in MPEP 2106:
Step 1 (Is the claim to a process, machine, manufacture, or composition of matter?): Yes. Claim 1 is a system (machine). Claim 10 is a method (process). Claim 17 is a computer program product comprising a storage medium (article of manufacture).
Step 2a Prong One (Does the claim recite an abstract idea?): Yes. Claims 1, 10, and 17 recite:
compute a first ratio indicative of inequity of the at least one feature (a mental process of determination, including using a mathematical computation performable mentally, such as computation of a ratio);
compares/comparing/compare…runtime data of the AI model with the ingested data to identify one or more matching records and generate a list comprising at least the one or more matching records (a mental process of determination, including comparing data to determine a list of matching records);
predict a status of a database management system (a mental process of evaluation, such as performing a mental prediction).
Under the broadest reasonable interpretation, these steps may be performed mentally, using mental observation and mental determination, including by a human using a physical aid such as pen and paper, including a human mentally performing observations and mentally performing mathematical calculations, and therefore correspond to the Mental Processes grouping.
Step 2a Prong Two (Does the claim recite additional elements that integrate the judicial exception into a practical application?): No. Claims 1, 10, and 17 additionally recite:
(in claim 1) a memory that stores computer executable components; and a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise: a data ingestion component that…a runtime deviation check component that…a training component that… (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
(in claim 1) uses testing data of an artificial intelligence (AI) model to generate ingested data by randomly changing one or more records of at least one feature comprised in the testing data based on a specific rule, wherein the ingested data is used to… (insignificant extra-solution activity as discussed in MPEP 2106.05(g))
(in claims 10 and 17) generating/generate…ingested data by randomly changing one or more records of at least one feature comprised in testing data of an AI model based on a specific rule, wherein the ingested data is used to…(insignificant extra-solution activity as discussed in MPEP 2106.05(g))
(in claim 10) the method is computer-implemented and the generating…comparing…training are performed by a system operatively coupled to a processor (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
(in claim 17) the computer program product is for training an AI model to improve robustness of the AI model to training inequity and runtime deviation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to…. and the generate…compare…train are performed by the processor (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
trains/training the AI model based on an influence factor of the at least one feature, the first ratio of the at least one feature and the list (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as disclosed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components.
Step 2b (Does the claim recite additional elements that amount to siqnificantly more than the judicial exception): No. Relying on the same analysis as Step 2a Prong Two (see MPEP 2106.05.I.A: Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:…Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP 2106.05(f));…Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception...; Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g);…)), claims 1, 10, and 17 do not recite any additional elements that amount to significantly more than the abstract idea. As discussed above, Claims 1, 10, and 17 recite:
(in claim 1) a memory that stores computer executable components; and a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise: a data ingestion component that…a runtime deviation check component that…a training component that… (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
(in claim 1) uses testing data of an artificial intelligence (AI) model to generate ingested data by randomly changing one or more records of at least one feature comprised in the testing data based on a specific rule, wherein the ingested data is used to… (insignificant extra-solution activity as discussed in MPEP 2106.05(g) such as data gathering and outputting and/or selecting a particular data source/type to be manipulated, where the insignificant extra-solution activity can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of performing repetitive calculations, electronic recordkeeping, and/or storing and retrieving information in memory)
(in claims 10 and 17) generating/generate…ingested data by randomly changing one or more records of at least one feature comprised in testing data of an AI model based on a specific rule, wherein the ingested data is used to…(insignificant extra-solution activity as discussed in MPEP 2106.05(g) such as data gathering and outputting and/or selecting a particular data source/type to be manipulated, where the insignificant extra-solution activity can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of performing repetitive calculations, electronic recordkeeping, and/or storing and retrieving information in memory)
(in claim 10) the method is computer-implemented and the generating…comparing…training are performed by a system operatively coupled to a processor (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
(in claim 17) the computer program product is for training an AI model to improve robustness of the AI model to training inequity and runtime deviation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to…. and the generate…compare…train are performed by the processor (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
trains/training the AI model based on an influence factor of the at least one feature, the first ratio of the at least one feature and the list (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea.
Regarding dependent claims 2, 11, and 18:
Step 2a Prong One: incorporates the rejection of claims 1, 10, and 17. The claims additionally recite extracts/extracting/extract one or more influence factors for one or more respective features of the testing data, wherein the at least one feature is selected for generating the ingested data based on the influence factor being greater than a first threshold (a mental determination, such as mentally determining an influence factor of a feature and mentally selecting the feature based on comparison of the determined influence factor to a threshold).
Step 2a Prong Two: the claims additionally recite an extraction component (claim 2), that the extracting is performed by the system (claim 11), and that extract is performed by the processor (claim 18) (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Step 2b: the claims additionally recite an extraction component (claim 2), that the extracting is performed by the system (claim 11), and that extract is performed by the processor (claim 18) (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Regarding dependent claims 3, 12, and 19:
Step 2a Prong One: incorporates the rejection of claims 1, 10, and 17
Step 2a Prong Two: the claims further recite wherein generating the ingested data comprises generating one or more ingested records for the at least one feature based on one or more existing records for the at least one feature in the testing data (insignificant extra-solution activity as discussed in MPEP 2106.05(g)).
Step 2b: the claims further recite wherein generating the ingested data comprises generating one or more ingested records for the at least one feature based on one or more existing records for the at least one feature in the testing data (insignificant extra-solution activity as discussed in MPEP 2106.05(g) such as data gathering and outputting and/or selecting a particular data source/type to be manipulated, where the insignificant extra-solution activity can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of performing repetitive calculations, electronic recordkeeping, and/or storing and retrieving information in memory).
Regarding dependent claims 4, 13, and 20:
Step 2a Prong One: incorporates the rejection of claims 1, 10, and 17. The claims further recite wherein the first ratio is generated based on an amount of records comprised in the ingested data and an inequity count of the at least one feature determined using the AI model (a mental process of determination, such as a human mentally determining/calculating a ratio based on an amount of records and an inequity count).
Step 2a Prong Two: the claims do not recite any additional limitations.
Step 2b: the claims do not recite any additional limitations.
Regarding dependent claims 5 and 14:
Step 2a Prong One: incorporates the rejection of claims 4 and 13; the claims further recite wherein the inequity count is determined by comparing prediction results generated by the AI model based on processing the one or more records in testing data of the AI model and based on processing one or more ingested records (a mental process of evaluation, such as a human mentally observing the prediction results generated by the AI model and mentally determining the inequity count based on the mental observation).
Step 2a Prong Two: the claims do not recite any additional limitations.
Step 2b: the claims do not recite any additional limitations.
Regarding dependent claims 6 and 15:
Step 2a Prong One: incorporates the rejection of claims 1 and 10; the claims further recite uses the prediction results/the prediction results are used to determine runtime deviation of the AI model (a mental process of evaluation, such as a human mentally determining a runtime deviation of an AI model based on prediction results).
Step 2a Prong Two: the claims additionally recite wherein the list further comprises prediction results generated by the AI model based on processing the one or more matching records (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the AI model generating the prediction results based on processing matching records and a field of use and technological environment as discussed in MPEP 2106.05(h) with respect to the list comprising prediction results generated by the AI model).
Step 2b: the claims additionally recite wherein the list further comprises prediction results generated by the AI model based on processing the one or more matching records (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the AI model generating the prediction results based on processing matching records and a field of use and technological environment as discussed in MPEP 2106.05(h) with respect to the list comprising prediction results generated by the AI model).
Regarding dependent claim 7:
Step 2a Prong One: incorporates the rejection of claim 1; the claim further recites monitors the influence factor for the at least one feature, the first ratio of the at least one feature and the list to generate a second ratio (a mental process of evaluation, such as a human mentally observing the influence factor, first ratio, and list and mentally determining/calculating a second ratio).
Step 2a Prong Two: the claims additionally recite a trust monitor component (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Step 2b: the claims additionally recite a trust monitor component (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Regarding dependent claim 8:
Step 2a Prong One: incorporates the rejection of claim 7. The claim additionally recites generates feedback for training the AI model based on at least the first ratio being greater than a threshold or the second ratio being lower than a second threshold (a mental process of determination, such as a human mentally determining whether the first ratio is greater than a threshold or the second ratio is lower than a second threshold, and then mentally determining feedback; Examiner notes that the claim does not require actually providing the feedback).
Step 2a Prong Two: the claim additionally recites a trust model manager component and that the feedback is for training the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Step 2b: the claim additionally recites a trust model manager component and that the feedback is for training the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Regarding dependent claim 9:
Step 2a Prong One: incorporates the rejection of claim 1. The claim additionally recites predict the status of the database management system while maintaining inequity and runtime deviation…below respective thresholds (a mental process of determination, such as a human mentally predicting DBMS status while maintaining the inequity and deviation).
Step 2a Prong Two: the claim additionally recite wherein the AI model is trained to perform the prediction, and that the runtime deviation is of the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Step 2b: the claim additionally recite wherein the AI model is trained to perform the prediction, and that the runtime deviation is of the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Regarding dependent claim 16:
Step 2a Prong One: incorporates the rejection of claim 9; the claim further recites:
monitoring…the influence factor for the at least one feature, the first ratio of the at least one feature and the list to generate a second ratio (a mental process of evaluation, such as a human mentally observing the influence factor, first ratio, and list and mentally determining/calculating a second ratio)
generating…feedback for training the AI model based on at least the first ratio being greater than a threshold or the second ratio being lower than a second threshold (a mental process of determination, such as a human mentally determining whether the first ratio is greater than a threshold or the second ratio is lower than a second threshold, and then mentally determining feedback; Examiner notes that the claim does not require actually providing the feedback).
Step 2a Prong Two: the claims additionally recite that the monitoring and generating are performed by the system and that the feedback is for training the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Step 2b: the claims additionally recite the claims additionally recite that the monitoring and generating are performed by the system and that the feedback is for training the AI model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) and a field of use and technological environment as discussed in MPEP 2106.05(h)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as recited in the dependent claims discussed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components, and limitations describing a field of use or technological environment. The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea, and limitations describing a field of use or technological environment.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102€, (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-3, 6-12, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over by Jia et al. (US 20210097329 A1) in view of Liu et al. (US 20260030557 A1), further in view of Donthireddy et al. (US 20230359600 A1).
With respect to claims 1, 10, and 17, Jia teaches a system, comprising: a memory that stores computer executable components; and a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise a data ingestion component, a runtime deviation check component, and a training component that perform respective method steps; a computer program product for training an AI model to improve robustness of the AI model to training inequity and runtime deviation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform the method; and the computer-implemented method, comprising:
generating, by the data ingestion component that uses testing data of an artificial intelligence (AI) model/by a system operatively coupled to a processor/by the processor, ingested data by randomly changing one or more records of at least one feature comprised in testing data of an AI model based on a specific rule, wherein the ingested data is used to compute a first ratio indicative of inequity of the at least one feature (e.g. paragraph 0029-0030, determining importance metrics for machine learning features of a machine learning model, such as by using a modified version of a test dataset with alternated values for a specified feature (i.e. randomized value, etc.); paragraph 0038, selecting features for evaluation randomly; paragraph 0040, values of selected feature in test dataset modified, such as by replacing the values with randomly generated values, etc.; paragraph 0042, feature importance value can be a percentage change value between base performance and modified performance of the machine learning model for the selected feature; i.e. an importance metric indicates that a given feature is more or less important than other features, and is therefore indicative of an inequity/bias with respect to that feature; compare with specification of the instant application at paragraph 0026, which indicates that the inequity ratio corresponds to the high influence factor);
training, by the training component/by the system/by the processor, the AI model based on an influence factor of the at least one feature and the first ratio of the at least one feature (e.g. paragraph 0031-0032, based on the importance metrics, managing machine learning model features, including by removing the feature from the model (when the importance is below a threshold) by retraining the model, or retraining the model using a modified feature, etc.).
Jia does not explicitly disclose:
comparing, by the runtime deviation check component/by the system/by the processor, runtime data of the AI model with the ingested data to identify one or more matching records and generate a list comprising at least the one or more matching records; and
training the AI model based on the list.
However, Liu teaches:
comparing, by the runtime deviation check component/by the system/by the processor, runtime data of the AI model with the ingested data to identify one or more matching records and generate a list comprising at least the one or more matching records (e.g. paragraph 0108, indicating that phrases describing similarity between datasets are equivalent to phrases describing matching between datasets; paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determining matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0211, model monitoring method in which device determines similarity between input data set and training data set of a model, and determines the performance of the model according to the similarity comparison result; i.e. the system compares an input data set for a model, analogous to runtime data, with training data of the model, analogous to ingested data, to determine a subset of the data items/records in each data set which match, analogous to generating a list comprising the matching records); and
training the AI model based on the list (e.g. paragraph 0135, model switching refers to switching to a model that is retrained; paragraph 0212, updating the first model via model fine-tuning or model switching; paragraph 0215, where similarity comparison result includes second matching degree between the data in the input data set and the data in the training data set, updating the first model; paragraph 0216, tuning model parameters, or switching the model; paragraph 0217, tuning or fine-tuning the model parameters or model structure; paragraph 0218, model switching; i.e. based on the similarity/matching degree between the input and training data sets, which is determined based on determining a set of elements/records (i.e. list) of match between the data sets, the system trains/retrains the model, such as via tuning/fine-tuning).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia and Liu in front of him to have modified the teachings of Jia (directed to managing machine learning features), to incorporate the teachings of Liu (directed to model performance monitoring) to include the capability to determine a set/list of records that match between an input/runtime dataset and a training/ingested dataset of a model, and to train/retrain/fine-tune the model based on the determined set/list of matching records. One of ordinary skill would have been motivated to perform such a modification in order to improve the feasibility and flexibility of model monitoring as described in Liu (paragraph 0211).
Jia and Gong do not explicitly disclose that the AI model is trained to predict a status of a database management system. However, Donthireddy teaches that the AI model is trained to predict a status of a database management system (e.g. paragraph 0067-0074, Fig. 4, training machine learning model to detect events in monitoring data for databases; model predicting likelihood that database will experience an event in the future, such as an outage, slow performance, or other adverse event).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Liu, and Donthireddy in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Liu (directed to model performance monitoring), to incorporate the teachings of Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status) to include the capability to train the model to predict a status of a database management system. One of ordinary skill would have been motivated to perform such a modification in order to provide improved and early detection of and correction for events that can negatively impact database performance, generate and provision databases based on recommendations, and provision and manage databases via a single control plane, resulting in efficiencies and faster time to market as described in Donthireddy (paragraph 0015).
With respect to claims 2, 11, and 18, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claims 1, 10, and 17 as previously discussed, and Jia further teaches the method further comprising: extracting, by an extraction component/by the system/by the processor, one or more influence factors for one or more respective features of the testing data, wherein the at least one feature is selected for generating the ingested data based on the influence factor being greater than a first threshold (e.g. paragraph 0025, evaluating features to determine their importance such as using an importance metric; paragraph 0026, during subsequent iteration of the process of Fig. 2, modifying specification of data to be collected based on importance of machine learning features; features determined not to meet threshold of importance no longer utilized and data of these dropped features is no longer identified as to be collected; paragraph 0030, importance metric for feature determined by comparing base performance of the model with new performance of the model for modified dataset with alternated values for the specified feature, including a removed value; compare with specification of the instant application at paragraph 0024, indicating that an influence factor for a feature is extracted by computing a difference between an original score for a model based on unaltered testing data and a new score for the model having changed feature values; i.e. only features having an importance greater than a threshold amount are collected and used for training the model).
With respect to claims 3, 12, and 19, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claims 1, 10, and 17 as previously discussed, and Jia further teaches wherein generating the ingested data comprises generating one or more ingested records for the at least one feature based on one or more existing records for the at least one feature in the testing data (e.g. paragraph 0029-0030, determining importance metrics for machine learning features of a machine learning model, such as by using a modified version of a test dataset with alternated values for a specified feature (i.e. randomized value, etc.); paragraph 00387, selecting features for evaluation randomly; paragraph 0040, values of selected feature in test dataset modified, such as by replacing the values with randomly generated values, etc.; paragraph 0042, feature importance value can be a percentage change value between base performance and modified performance of the machine learning model for the selected feature).
With respect to claims 6 and 15, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claims 1 and 10 as previously discussed, and Liu further teaches wherein the list further comprises prediction results generated by the AI model based on processing the one or more matching records, and wherein the runtime deviation check component uses the prediction results to determine runtime deviation of the AI model (e.g. paragraph 0078, the input data set may be understood as the data set which is predicted by using the model; input data set processed by model to obtain prediction result; paragraph 0086, monitoring model performance to determine whether the performance meets requirements; paragraph 0106, monitoring model performance based on comparison of input data set and training data set; if the datasets are similar, model output with high accuracy may be obtained and model performance is good; if the datasets are not similar, the model output with high accuracy cannot be obtained and the performance of the model is poor; paragraph 0108, indicating that phrases describing similarity between datasets are equivalent to phrases describing matching between datasets; paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determining matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0211, model monitoring method in which device determines similarity between input data set and training data set of a model, and determines the performance of the model according to the similarity comparison result; i.e. the system compares an input data set for a model, analogous to runtime data, with training data of the model, analogous to ingested data, to determine a subset of the data items/records in each data set which match, analogous to generating a list comprising the matching records, where at least the input data includes prediction results generated by the model (and therefore the set of matches comprises at least these), and the comparison is used as part of a model monitoring process to determine a deviation (i.e. in accuracy) of the model).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Donthireddy, and Liu in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status), to incorporate the teachings of Liu (directed to model performance monitoring) to include the capability to determine a set/list of records that match between an input/runtime dataset and a training/ingested dataset of a model, and to train/retrain/fine-tune the model based on the determined set/list of matching records. One of ordinary skill would have been motivated to perform such a modification in order to improve the feasibility and flexibility of model monitoring as described in Liu (paragraph 0211).
With respect to claim 7, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claim 1 as previously discussed, and Jia further teaches the system further comprising: a trust monitor component that monitors the influence factor for the at least one feature and the first ratio of the at least one feature to generate a second ratio (e.g. paragraph 0026, model deployed for production use in servers to perform inference work associated with a service provided to end users; collecting additional training data during deployment and utilizing it to retrain the machine learning model, such as by repeating the process of Fig. 2; during subsequent iteration of the process the specification of the data collected may be modified based on importance of the machine learning features determined during model training, such as by dropping/no longer utilizing features not meeting threshold of importance; paragraph 0027, indicating that process of Fig. 3 is included in 208 of the process of Fig. 2; paragraph 0029-0030, Fig. 3, determining importance metrics for machine learning features of a machine learning model; paragraph 0036, indicating that process of Fig. 4 is included in processes of Figs. 2/3; paragraph 0042, Fig. 4, step 410, feature importance value can be a percentage change value between base performance and modified performance of the machine learning model for the selected feature; i.e. while the model is deployed in a production environment, the system continues to perform additional iterations of the process of Fig. 2 (which includes processes of Figs. 3 and 4, including steps related to determining importance/influence metrics for individual features such as relevant percentages/ratios), such that the importance/influence factor including the associated percentage/ratio are continuously evaluated and updated, analogous to monitoring the influence factor/ratio of the at least one feature; moreover, since the process is repeated/iterated and includes calculating a new percentage/ratio indicative of importance (percentage change value in model performance) each time, a subsequent calculation of this percentage/ratio is analogous to generation of a second ratio).
Liu further teaches the system further comprising: a trust monitor component that monitors the list to generate a second ratio (e.g. paragraph 0108, indicating that phrases describing similarity between datasets are equivalent to phrases describing matching between datasets; paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determining matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0211, implementing model monitoring to flexibly determine the similarity comparison between the input data set and the training data set; paragraph 0242, when monitoring the model, comparing the similarity between the input data set and the training dataset and determining performance; i.e. the system performs monitoring which includes comparing an input data set for a model, analogous to runtime data, with training data of the model, analogous to ingested data, to determine a subset of the data items/records in each data set which match, analogous to generating a list comprising the matching records (and therefore monitoring that list), where this also includes determining a ratio, such as a ratio based on the total number of items/records in the input dataset and the number of items/records in the input dataset that are identical to items/records in the training dataset, analogous to a second ratio).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Donthireddy, and Liu in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status), to incorporate the teachings of Liu (directed to model performance monitoring) to include the capability to perform model monitoring, including by determining/monitoring a set/list of records that match between an input/runtime dataset and a training/ingested dataset of a model, and determining corresponding ratios. One of ordinary skill would have been motivated to perform such a modification in order to improve the feasibility and flexibility of model monitoring as described in Liu (paragraph 0211).
With respect to claim 8, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claim 7 as previously discussed, and Jia further teaches the system further comprising: a trust model manager component that generates feedback for training the AI model based on at least the first ratio being greater than a threshold (e.g. paragraph 0029, determining importance metrics for machine learning features; paragraphs 0031-0032, based on the importance metrics, managing the machine learning features, such as by removing/dropping the feature, modifying the feature, etc.; paragraph 00345, generating new version of the model based on the management of the machine learning features, such as generating/retraining the model with training data that does not include removed features or with training data that includes new/transformed/modified features, etc.; paragraph 0042, determining feature importance metric which is percentage change value; i.e. where the importance metric is a percentage/ratio, and is used to determine how to manage the features, and the result of this determination indicates how the model is to be trained/retrained, this is analogous to generating feedback for training the AI model based on the first ratio).
Liu teaches the system further comprising: a trust model manager component that generated feedback for training the AI model based on the second ratio being lower than a second threshold (e.g. paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determinine matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0212, updating the first model via model fine-tuning or model switching; paragraph 0213, model switching refers to switching to model that is retrained; paragraph 0215, where similarity comparison result includes second matching degree between the data in the input data set and the data in the training data set, updating the first model; paragraph 0216, tuning model parameters, or switching the model based on matching degrees being less than threshold; paragraph 0217, tuning or fine-tuning the model parameters or model structure based on comparison of matching degree to threshold; paragraph 0218, model switching based on matching degree below threshold; i.e. after determining the ratio/degree of matching, this is compared to a threshold and the result of this comparison is used as feedback for determining how to train the model, such as by fine-tuning, or retraining the model).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Donthireddy, and Liu in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status), to incorporate the teachings of Liu (directed to model performance monitoring) to include the capability to compare the ratio/degree of matching between the datasets and, where this ratio/degree is below a threshold amount, utilize this as feedback/guidance for determining how to further train the model, such as by fine-tuning the model, retraining the model, etc.. One of ordinary skill would have been motivated to perform such a modification in order to improve the feasibility and flexibility of model monitoring as described in Liu (paragraph 0211).
With respect to claim 16, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claim 9, and Jia additionally teaches the method further comprising:
monitoring, by the system, the influence factor for the at least one feature and the first ratio of the at least one feature (e.g. paragraph 0026, model deployed for production use in servers to perform inference work associated with a service provided to end users; collecting additional training data during deployment and utilizing it to retrain the machine learning model, such as by repeating the process of Fig. 2; during subsequent iteration of the process the specification of the data collected may be modified based on importance of the machine learning features determined during model training, such as by dropping/no longer utilizing features not meeting threshold of importance; paragraph 0027, indicating that process of Fig. 3 is included in 208 of the process of Fig. 2; paragraph 0029-0030, Fig. 3, determining importance metrics for machine learning features of a machine learning model; paragraph 0036, indicating that process of Fig. 4 is included in processes of Figs. 2/3; paragraph 0042, Fig. 4, step 410, feature importance value can be a percentage change value between base performance and modified performance of the machine learning model for the selected feature; i.e. while the model is deployed in a production environment, the system continues to perform additional iterations of the process of Fig. 2 (which includes processes of Figs. 3 and 4, including steps related to determining importance/influence metrics for individual features such as relevant percentages/ratios), such that the importance/influence factor including the associated percentage/ratio are continuously evaluated and updated, analogous to monitoring the influence factor/ratio of the at least one feature; moreover, since the process is repeated/iterated and includes calculating a new percentage/ratio indicative of importance (percentage change value in model performance) each time, a subsequent calculation of this percentage/ratio is analogous to generation of a second ratio).; and
generating, by the system, feedback for training the AI model based on at least the first ratio being greater than a threshold (e.g. paragraph 0029, determining importance metrics for machine learning features; paragraphs 0031-0032, based on the importance metrics, managing the machine learning features, such as by removing/dropping the feature, modifying the feature, etc.; paragraph 00345, generating new version of the model based on the management of the machine learning features, such as generating/retraining the model with training data that does not include removed features or with training data that includes new/transformed/modified features, etc.; paragraph 0042, determining feature importance metric which is percentage change value; i.e. where the importance metric is a percentage/ratio, and is used to determine how to manage the features, and the result of this determination indicates how the model is to be trained/retrained, this is analogous to generating feedback for training the AI model based on the first ratio).
Liu further teaches the method further comprising:
monitoring, by the system, the list to generate a second ratio (e.g. paragraph 0108, indicating that phrases describing similarity between datasets are equivalent to phrases describing matching between datasets; paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determining matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0211, implementing model monitoring to flexibly determine the similarity comparison between the input data set and the training data set; paragraph 0242, when monitoring the model, comparing the similarity between the input data set and the training dataset and determining performance; i.e. the system performs monitoring which includes comparing an input data set for a model, analogous to runtime data, with training data of the model, analogous to ingested data, to determine a subset of the data items/records in each data set which match, analogous to generating a list comprising the matching records (and therefore monitoring that list), where this also includes determining a ratio, such as a ratio based on the total number of items/records in the input dataset and the number of items/records in the input dataset that are identical to items/records in the training dataset, analogous to a second ratio); and
generating, by the system, feedback for training the AI model based on the second ratio being lower than a second threshold (e.g. paragraph 0199, determining number of input data in the input data set that are identical to the training data in the training data set relative to a threshold to determine a matching determination result indicating matching or mismatching; paragraph 0200, determining ratio of number of input data in the input data set that are identical to training data in the training data set to the total number of input data relative to threshold to determine matching or mismatching; paragraph 0201, for each input data in the input data set, comparing input data with each training data in the training data set and determining whether the data in the two data sets matches according to a number/proportion of identical data in the two data sets; paragraph 0202, matching degree with respect to a threshold; paragraph 0212, updating the first model via model fine-tuning or model switching; paragraph 0213, model switching refers to switching to model that is retrained; paragraph 0215, where similarity comparison result includes second matching degree between the data in the input data set and the data in the training data set, updating the first model; paragraph 0216, tuning model parameters, or switching the model based on matching degrees being less than threshold; paragraph 0217, tuning or fine-tuning the model parameters or model structure based on comparison of matching degree to threshold; paragraph 0218, model switching based on matching degree below threshold; i.e. after determining the ratio/degree of matching, this is compared to a threshold and the result of this comparison is used as feedback for determining how to train the model, such as by fine-tuning, or retraining the model).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Donthireddy, and Liu in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status), to incorporate the teachings of Liu (directed to model performance monitoring) to include the capability to compare the ratio/degree of matching between the datasets and, where this ratio/degree is below a threshold amount, utilize this as feedback/guidance for determining how to further train the model, such as by fine-tuning the model, retraining the model, etc.. One of ordinary skill would have been motivated to perform such a modification in order to improve the feasibility and flexibility of model monitoring as described in Liu (paragraph 0211).
With respect to claim 9, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claim 1 as previously discussed, and Jia and Liu further teach wherein the AI model is trained to make predictions while maintaining inequity and runtime deviation of the AI model below respective thresholds (e.g. Jia paragraph 0026, trained machine learning model deployed for production use to perform inference work; additional training data collected during deployment and utilized to retraining the machine learning model, such as repeating process of Fig. 2 (which, as previously cited, can include determining feature importance metrics and appropriately training/retraining the model based on the determined feature importance); Liu paragraph 0083, monitoring ML model to monitor accuracy of output data of the model; paragraph 0086, monitoring performance of the model at device where the model is deployed, determining whether performance of the model meets actual requirement, and updating the first model in time when the performance of the model does not meet the actual requirement; paragraph 0142, when monitoring performance of the first model, acquiring training and input data sets and comparing them to obtain similarity comparison result; paragraphs 0199-0216, determining whether training and input datasets are matching or mismatched and/or a degree of matching according to thresholds, and fine-tuning, switching/retraining, etc. the model based on the comparison to the relevant threshold; i.e. the model is deployed for performing predictions, and the feature importance (i.e. indicative of inequity to the extent that a given feature may be more or less important/influential than other features) and accuracy (i.e. level of deviation) are monitored, and the model may be updated in order to keep these within acceptable levels (i.e. below respective thresholds)).
Donthireddy teaches wherein the AI model is trained to predict the status of the database management system (e.g. paragraph 0067-0074, Fig. 4, training machine learning model to detect events in monitoring data for databases; model predicting likelihood that database will experience an event in the future, such as an outage, slow performance, or other adverse event).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Jia, Liu, and Donthireddy in front of him to have modified the teachings of Jia (directed to managing machine learning features) and Liu (directed to model performance monitoring), to incorporate the teachings of Donthireddy (directed to database provisioning and management systems and methods, including training models to predict database status) to include the capability to train the model to predict a status of a database management system. One of ordinary skill would have been motivated to perform such a modification in order to provide improved and early detection of and correction for events that can negatively impact database performance, generate and provision databases based on recommendations, and provision and manage databases via a single control plane, resulting in efficiencies and faster time to market as described in Donthireddy (paragraph 0015).
Claims 4, 5, 13, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over by Jia in view of Liu, further in view of Donthireddy, further in view of Weider et al. (US 20200380398 A1).
With respect to claims 4, 13, and 20, Jia in view of Liu, further in view of Donthireddy teaches all of the limitations of claims 1, 10, and 17 as previously discussed. Jia, Liu, and Donthireddy do not explicitly disclose wherein the first ratio is generated based on an amount of records comprised in the ingested data and an inequity count of the at least one feature determined using the AI model.
However, Weider teaches wherein the first ratio is generated based on an amount of records comprised in the ingested data and an inequity count of the at least one feature determined using the AI model (e.g. paragraph 0058, accessing datasets and examining them to identify bias and data imbalance; determining number of datapoints corresponding to each category available for a feature; comparing the number of each category to the total number of datapoints to calculate a percentage of the datapoints representing each category; the identified distribution is examined to determine whether the feature is balanced within the desired threshold; machine learning model used to perform the statistical analysis; i.e. a bias/imbalance ratio/percentage is generated based on a total number of datapoints in the dataset (analogous to an amount of records comprised in the ingested data) and a number of datapoints for each category of the features (analogous to an inequity count of the at least one feature)).
With respect to claims 5 and 14, Jia in view of Liu, further in view of Donthireddy, further in view of Weider teaches all of the limitations of claims 4 and 13 as previously discussed, and Jia further teaches wherein the inequity count is determined by comparing prediction results generated by the AI model based on processing one or more records in testing data of the AI model and based on processing one or more ingested records (e.g. paragraph 0029-0030, determining importance metrics for machine learning features of a machine learning model, such as by using a modified version of a test dataset with alternated values for a specified feature (i.e. randomized value, etc.); to establish baseline performance, each entry of the test dataset is provided as an input to the model and a corresponding inference result of the model is compared with corresponding known correct result to determine accuracy/performance of the model; paragraph 0038, selecting features for evaluation randomly; paragraph 0040, values of selected feature in test dataset modified, such as by replacing the values with randomly generated values, etc.; paragraph 0041, each entry of the modified test dataset is provided as an input to the model and a corresponding inference result of the model is compared with corresponding correct result to determine accuracy/performance of the model (i.e. with modified dataset); paragraph 0042, feature importance value can be a percentage change value between base performance and modified performance of the machine learning model for the selected feature; i.e. the importance metric indicates that a given feature is more or less important than other features, and is therefore indicative of an inequity/bias with respect to that feature, and is determined by comparing base model performance (i.e. using the original test dataset, analogous to prediction results generated by the AI model and based on processing or more records in testing data) to modified model performance (i.e. using the modified dataset, analogous to prediction results generated by the AI model and based on processing one or more ingested records) to determine a percentage change in performance in the model based on the test dataset and the modified dataset).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JEREMY L STANLEY/
Primary Examiner, Art Unit 2127
Prosecution Insights