DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on 09/06/2024. Claims 1-20 are pending in the case.
Priority
The present application claims priority under 35 U.S.C. §119 to Chinese Patent Application No. 202210224481.9, filed March 7, 2022, and Chinese Patent Application No. 202210958867.2, filed August 9, 2022. Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119 and/or 35 U.S.C. 120 is acknowledged.
Information Disclosure Statement
4. As required by MPEP 609 (c), the Applicants’ submission of the Information Disclosure Statement(s) filed on 10/29/2025 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now
Claim Rejections - 35 USC § 101
5. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed towards an abstract idea, without significantly more.
Step 1
According to the first part of the analysis, in the instant case, claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter)
Regarding Claim 1,
At step 2A, Prong 1, Does the claim recite a judicial exception?
Claim 1 recites the steps of:
performing, by a first network element, inference on a task based on a first model (This step involves mathematical operation to produce an inference result which falls under the mathematical concepts abstract idea group);
determining, by the first network element, first accuracy corresponding to the first model, wherein the first accuracy is used to indicate accuracy of the first model on an inference result of the task (This step involves an evaluating model’s result by determining an accuracy value is understood to be a mathematical operation which falls under the mathematical concepts abstract idea group).
The claim recites a judicial exception, a mathematical concept that evaluate model inference accuracy and determining whether to report an unacceptable result. The accuracy determination and the comparison to preset condition recite mathematical operation which falls within the “Mathematical Concept” groupings of abstract ideas. Accordingly, the claims recite an abstract idea.
Step 2A prong 2: Does the claim recite additional elements? Do those additional elements, individually and in combination, integrate the judicial exception into a practical application?
Further, the claim does not recite any additional element which could integrate this abstract idea into a practical application, because the additional elements recited of consist of:
a processor and a memory, wherein a program or instructions are stored in the memory and executable on the processor, and the program or instructions, when executed by the processor, cause the first network element (claim 19), … (a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));)
processor and a memory, wherein a program or instructions are stored in the memory and executable on the processor, and when the program or instructions are executed by the processor (claim 20) … (a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));)
in a case that the first accuracy reaches a preset condition, sending, by the first network element, first information to a second network element, wherein the first information is used to indicate that accuracy of the first model does not meet an accuracy requirement or decreases (specifying what type of data is gathered limits the claim to a particular field of use or data source, MPEP 2106.05(h)),
The additional elements as disclosed above alone or in combination do not
integrate the judicial exception into practical application. The additional elements do no more than gathering data, apply the abstract idea on a generic computer and display the result. They are generic computer functions in combination with limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use that are implemented to perform the disclosed abstract idea above. Thus, the claim is directed towards the abstract idea.
Step 2B: Do the additional elements, considered individually and in combination, amount to significantly more than the judicial exception?
No, as shown above with respect to integration of the abstract idea into a practical application, the additional element of:
a processor and a memory, wherein a program or instructions are stored in the memory and executable on the processor, and the program or instructions, when executed by the processor, cause the first network element ( claim 19), … (a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f))
processor and a memory, wherein a program or instructions are stored in the memory and executable on the processor, and when the program or instructions are executed by the processor ( claim 20) … (a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));)
in a case that the first accuracy reaches a preset condition, sending, by the first network element, first information to a second network element, wherein the first information is used to indicate that accuracy of the first model does not meet an accuracy requirement or decreases (specifying what type of data is gathered limits the claim to a particular field of use or data source, MPEP 2106.05(h)),
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are generic computer functions in combination with limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use that are implemented to perform the disclosed abstract idea above. Thus, the claim is directed towards the abstract idea
Thus, the claims are not patent eligible. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Neither can insignificant extra-solution activity. All of these additional elements as generically claimed are thus considered well-understood, routine, and conventional. Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea.
Thus, these independent claims are not patent eligible.
The dependent claims respectively recite a judicial exception in limitations of: “wherein before the sending, by the first network element, first information to a second network element, the method further comprises: receiving, by the first network element, a model performance subscription request from the second network element, wherein the model performance subscription request is used to request the first network element to monitor the accuracy of the first model.”(claims 2/15), “wherein after the sending the first information to the second network element, the method further comprises: receiving, by the first network element, second information from the second network element, wherein the second information comprises information of a retrained first model.”(claims 3/16), “wherein the determining, by the first network element, first accuracy corresponding to the first model comprises: obtaining, by the first network element based on the first model, inference result data corresponding to the task; obtaining, by the first network element, tag data corresponding to the inference result data; and calculating, by the first network element, the first accuracy of the first model based on the inference result data and the tag data.(claims 4), “wherein the first accuracy can be used to indicate at least one of the following: correctness of the inference result of the task; or deviation of the inference result of the task.”(claim 5/18), “wherein the first information comprises at least one of the following: identification information of the first model; identification information of the task; constraint condition information of the task; information indicating that the accuracy of the first model does not meet the accuracy requirement or decreases; the first accuracy; request indication information for retraining the first model; indication information for re-requesting a model, used to request to obtain a model corresponding to the task; first data of the task, wherein the first data is used to retrain the first model; or information of a fourth network element, wherein the fourth network element is a network element that receives and stores the first data from the first network element.”(claims 6/17), “wherein the second information further comprises at least one of the following: condition information applicable to the retrained first model; or third accuracy of the retrained first model, wherein the third accuracy is used to indicate accuracy of a model output result presented by the retrained first model in a training phase or a testing phase.”(claims 7), “wherein the obtaining, by the first network element, tag data corresponding to the inference result data comprises: determining, by the first network element, a source device of tag data corresponding to the task; and obtaining, by the first network element, the tag data from the source device.”(Claim 8), “wherein after the obtaining, based on the first model, the inference result data corresponding to the task, the method further comprises: sending, by the first network element, the inference result data to a third network element, wherein the third network element is a network element that triggers the task; wherein the third network element comprises a consumer network element.”(claim 9), “wherein the preset condition comprises at least one of the following conditions that: the first accuracy is lower than a first threshold; the first accuracy is lower than second accuracy; or the first accuracy is lower than the second accuracy, and a difference between the first accuracy and the second accuracy is greater than a second threshold, wherein the second accuracy is used to indicate accuracy of a model output result presented by the first model in a training phase or a testing phase.”(claim 10), “wherein in the case that the first accuracy reaches the preset condition, the method further comprises: requesting, by the first network element, to obtain a second model from a fifth network element, wherein the second model is a model provided by the fifth network element and used for the task; and performing, by the first network element, inference on the task based on the second model; wherein the fifth network element comprises a model training function network element.(claim 11), “wherein in the case that the first accuracy reaches the preset condition, the method further comprises: sending, by the first network element, fourth information to a third network element, wherein the fourth information is used to indicate that the accuracy of the first model does not meet the accuracy requirement or decreases; wherein the fourth information comprises at least one of the following: all or a part of description information of the task; information indicating that the accuracy of the first model does not meet the accuracy requirement or decreases; the first accuracy; recommended operation information; or waiting time information, wherein the waiting time information is used to indicate a time required for the first network element to resume inference on the task.” (claim 12), “herein the first network element comprises a model inference function network element; or the second network element comprises a model training function network element.”(claim 13). These additional limitations (in claims 2-13 and 15-18) also constitute mathematical concepts which fall within the “Mental Processes”, Mathematical concept and of abstract ideas.
This judicial exception is not integrated into a practical application. Additional elements “computer readable medium comprising: computer program code (in claims 2-13 and 15-18) all amount to no more than adding insignificant extra-solution activity/specifications related to data gathering, data input, or data transmittal. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of non-transitory computer readable medium comprising: computer program code are again insignificant extra-solution activity steps that cannot provide an inventive concept. All of these additional elements as generically claimed are considered well-understood, routine, and conventional.
Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea. Thus, all of the dependent claims are also not patent eligible.
Examiner Comments
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (US 20220351081 A1, 2022-11-03) in view of HUANG (Pub. No.: US 20230224752 A1, Pub. Date: 2023-07-13)
Regarding independent Claim 1,
Liu teaches a model accuracy determining method (see Liu: Fig.2, [0063], illustrating The model evaluation module 202 is configured to evaluate the training model based on a model evaluation metric and an inference result), comprising:
performing, by a first network element, inference on a task based on a first model (see Liu: Fig.6, [0109], “S602: The first server inputs input data into the first training model for model inference, to obtain an inference result.”);
determining, by the first network element, first accuracy corresponding to the first model (see Liu: Fig.6, [0118], “S603: The first server evaluates the first training model based on the inference result and a model evaluation metric, to obtain an evaluation result of the model evaluation metric.”… Fig.3, [0068]), “The model evaluation metric assessment module 2023 is further configured to determine, by using the input data and the inference result that are used for model evaluation and based on the at least one model evaluation metric used to evaluate the training model and the evaluation rule of the model evaluation metric, an evaluation result of the at least one model evaluation metric used to evaluate the training model,”), wherein the first accuracy is used to indicate accuracy of the first model on an inference result of the task (see Liu: Fig.3, [0010], “the model evaluation metric includes at least one of the following: accuracy of the inference result, precision of the inference result, recall of the inference result, F1-score (F1-Score) of the inference result, and an area under a receiver operating characteristic ROC curve AUC of the inference result.”); and
in a case that the first accuracy reaches a preset condition (see Liu: Fig.6, [0129], “S604: If an evaluation result of at least one model evaluation metric does not exceed a preset threshold corresponding to the model evaluation metric, the first server sends a retraining instruction for the first training model.”), sending, by the first network element, first information to a second network element (see Liu: Fig.3, [0069], “the retraining trigger policy for the training model is as follows: If an evaluation result of the at least one model evaluation metric used to evaluate the training model does not exceed a preset threshold corresponding to the model evaluation metric, the model evaluation metric assessment module 2023 determines that the training model has a poor inference effect, and sends the retraining instruction for the training model to the model retraining management module 201”),
wherein the second network element is a network element that provides the first model (see Liu: Fig.6, [0104], “The first training model is obtained by performing model training on a preset model by the second server. The preset model may be a model pre-stored in the system, or may be a model determined based on a requirement of an application scenario of the required first training model.”)
As shown above, Liu teaches all the limitations of Claim1. Liu does not teach the method wherein the first information is used to indicate that accuracy of the first model does not meet an accuracy requirement or decreases.
However, Huang teaches the method wherein the first information is used to indicate that accuracy of the first model does not meet an accuracy requirement or decreases (see Huang: Fig.13, [0614], “The first information includes the performance report of the model, and the performance report of the model indicates the performance evaluation result of the model, or the performance report of the model indicates that the performance evaluation result of the model does not meet a requirement for the performance indicator of the model.”)
Because both Liu and Huang are in the same/similar field of endeavor of monitoring machine learning model performance, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify teaching of Liu’s retraining communication to include Huang’s performance report indicating that the model’s performance evaluation result does not meet it performance requirement. After modification of Liu, the model providing server would receive an indication that the deployed model performance failed its requirement as taught by Huang. One would have been motivated to make such a combination to help the training server recognize a degraded model performance and initiate a timely retaining so that the use of inaccurate model is reduced.
Regarding Claim 2,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu teaches the method wherein:
before the sending, by the first network element, first information to a second network element (see Huang: Fig.8, [0400], “Step 803a: The inference NWDAF sends, to the training NWDAF, a model request that carries the analytics ID.”) the method further comprises:
receiving, by the first network element, a model performance subscription request from the second network element wherein the model performance subscription request is used to request the first network element to monitor the accuracy of the first mode (see Huang: Fig.8, [0400 “Step 804a: The training NWDAF sends, to the inference NWDAF, a model performance information subscription request that carries the analytics ID, a model performance indicator (for example, precision (Precision), accuracy (Accuracy), error rate (Error Rate), recall rate (Recall), F1 score (F-Score), mean squared error (Mean Squared Error, MSE), root mean squared error (Root Mean Squared Error, RMSE), root mean squared logarithmic error (Root Mean Squared Logarithmic Error, RMSLE), mean absolute error (Mean Absolute Error, MAE), model inference duration, model robustness, model expandability, or model interpretability), and a reporting periodicity.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention, to modify teaching of Liu’s retraining communication to include Huang’s a model performance information subscription request and a performance report indicating that the model’s performance evaluation result does not meet it performance requirement. After modification of Liu, the model providing server would receive an indication that the deployed model performance failed its requirement as taught by Huang. One would have been motivated to make such a combination to help the training server recognize a degraded model performance and initiate a timely retaining so that the use of inaccurate model is reduced.
Regarding Claim 3,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu teaches the method wherein:
after the sending the first information to the second network element (see Liu: Fig.3, [0069], “the retraining trigger policy for the training model is as follows: If an evaluation result of the at least one model evaluation metric used to evaluate the training model does not exceed a preset threshold corresponding to the model evaluation metric, the model evaluation metric assessment module 2023 determines that the training model has a poor inference effect, and sends the retraining instruction for the training model to the model retraining management module 201”), the method further comprises:
receiving, by the first network element, second information from the second network element (see Liu: Fig.6, [0145], “S606: The second server determines a retraining sample set based on the input data and the inference result.”), wherein the second information comprises information of a retrained first model (see Liu: Fig.6, [0146],, “In response to the received retraining instruction for the first training model, the second server adds, to the retraining sample set used for model retraining, the input data and the inference result corresponding to the input data that are of the first training model and that are obtained by the second server from the first server. The retraining sample set stores training data used for model retraining.”)
Regarding Claim 4,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu further teaches the method wherein:
the determining, by the first network element, first accuracy corresponding to the first model (see Liu: Fig.3, [0068], “The model evaluation metric assessment module 2023 is further configured to determine, by using the input data and the inference result that are used for model evaluation and based on the at least one model evaluation metric used to evaluate the training model and the evaluation rule of the model evaluation metric, an evaluation result of the at least one model evaluation metric used to evaluate the training model,”), comprises:
obtaining, by the first network element based on the first model, inference result data corresponding to the task (see Liu: Fig.8, [0160], “The evaluation unit 803 is configured to evaluate the first training model based on the inference result and a model evaluation metric, to obtain an evaluation result of the model evaluation metric.”)
obtaining, by the first network element, tag data corresponding to the inference result data (see Liu: Fig.8, [0161], “] The model evaluation metric includes at least one of the following: accuracy of the inference result, precision of the inference result, recall of the inference result, F1-score (F1-Score) of the inference result, and an area under a receiver operating characteristic ROC curve AUC of the inference result.”); and
calculating, by the first network element, the first accuracy of the first model based on the inference result data and the tag data (see Liu: Fig.8, [0160], “The evaluation unit 803 is configured to evaluate the first training model based on the inference result and a model evaluation metric, to obtain an evaluation result of the model evaluation metric.”)
Regarding Claim 5,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu teaches the method wherein:
the first accuracy can be used to indicate at least one of the following: correctness of the inference result of the task; or deviation of the inference result of the task (see Liu: Fig.1, [0021], “the processing unit is configured to: if the inference result is a correct inference result, reserve, by the second server, the inference result and input data corresponding to the inference result; or if the inference result is an incorrect inference result, delete, by the second server, the inference result and input data corresponding to the inference result, or replace, by the second server, the inference result with a correct inference result corresponding to the input data.”)
Regarding Claim 6,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu teaches the method wherein:
the first information (see Liu: Fig.3, [0048], the model training module 102 is configured to obtain a preset model used for model training from the first model storage module 103.), comprises at least one of the following:
identification information of the first model; identification information of the task; constraint condition information of the task; information indicating that the accuracy of the first model does not meet the accuracy requirement or decreases; the first accuracy; request indication information for retraining the first model; indication information for re-requesting a model, used to request to obtain a model corresponding to the task; first data of the task, wherein the first data is used to retrain the first model; or information of a fourth network element, wherein the fourth network element is a network element that receives and stores the first data from the first network element (see Liu: Fig.3, [0049], The first model storage module 103 is configured to obtain a trained training model from the model training module 102, and store the training model obtained by the first model storage module 103. The first model storage module 103 is further configured to store description information of the training model. The description information of the training model includes a name, usage, life cycle, and the like of the training model. In addition, the first model storage module 103 is configured to modify the description information of the training model.)
Regarding Claim 7,
As shown above, Liu and Huang teach all the limitations of Claim 3. Liu teaches the method wherein:
the second information (see Liu: Fig.3, [0053], The second model storage module 104 is configured to obtain and store the training model sent by the first model storage module 103. Optionally, the second model storage module 104 is further configured to obtain and store the description information that is of the training model and that is sent by the first model storage module 103. For the description information, refer to the foregoing content. Details are not described herein again) the model training module 102 is configured to obtain a preset model used for model training from the first model storage module 103.), further comprises at least one of the following:
condition information applicable to the retrained first model; or third accuracy of the retrained first model, wherein the third accuracy is used to indicate accuracy of a model output result presented by the retrained first model in a training phase or a testing phase (see Liu: Fig.3, [0058], “If the second server 20 is a server in the public cloud, the data collection module 101, the model training module 102, the first model storage module 103, and the model retraining management module 201 are located on the second server 20. If the second server 20 is a server cluster including a plurality of servers in the private cloud, the data collection module 101, the model training module 102, the first model storage module 103, and the model retraining management module 201 are located on a same server in the server cluster, or on different servers in the server cluster.”)
Regarding Claim 8,
As shown above, Liu and Huang teach all the limitations of Claim 4. Huang further teaches the method wherein:
the obtaining, by the first network element, tag data corresponding to the inference result data (see Huang: Fig.3, [147], “Inference NWDAF: The inference NWDAF is responsible for model inference, performs data analysis based on an inference result, and outputs a data analysis result.”), comprises:
determining, by the first network element, a source device of tag data corresponding to the task (see Huang: Fig.3, [147],“The analytics type identifier indicates a specific analytics type provided by the training NWDAF, which, for example, may be service experience, network performance, or UE mobility.”; and
obtaining, by the first network element, the tag data from the source device (see Huang: Fig.3, “The training NWDAF sends, to the inference NWDAF, a model response that carries model information.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention, to modify teaching of Liu’s retraining communication to include Huang’s source device of tag data corresponding to the task determination and the tag data from the source device . After modification of Liu, the model providing server would receive an indication that the deployed model performance failed its requirement as taught by Huang. One would have been motivated to make such a combination to help the training server recognize a degraded model performance and initiate a timely retaining so that the use of inaccurate model is reduced.
Regarding Claim 9,
As shown above, Liu and Huang teach all the limitations of Claim 4. Huang further teaches the method wherein:
after the obtaining, based on the first model, the inference result data corresponding to the task ( see Huang: Fig.3, [0147], “Inference NWDAF: The inference NWDAF is responsible for model inference, performs data analysis based on an inference result, and outputs a data analysis result..”), the method further comprises:
sending, by the first network element, the inference result data to a third network element, wherein the third network element is a network element that triggers the task (see Huang: Fig.5, [197], “Model performance monitoring and feedback: Model status information is added to registration information of an inference NWDAF at the NRF, and the inference NWDAF performs model performance monitoring. When determining that a model needs to be retrained, the inference NWDAF updates the model status information at the NRF. The NRF notifies a training NWDAF of the model status update, to trigger the training NWDAF to retrain the model.”), wherein the third network element comprises a consumer network element (see Huang: Fig.1, [0116], “a data analytics network element may be the NWDAF network element shown in FIG. 1, or may be another network element that is in a future communication system and that has a function of the NWDAF network element. A network repository network element may be the NRF network element.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention, to modify teaching of Liu’s retraining communication to include Huang’s source device of tag data corresponding to the task determination and the tag data from the source device . After modification of Liu, the model providing server would receive an indication that the deployed model performance failed its requirement as taught by Huang. One would have been motivated to make such a combination to help the training server recognize a degraded model performance and initiate a timely retaining so that the use of inaccurate model is reduced.
Regarding Claim 10,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu further teaches the method wherein:
the preset condition comprises at least one of the following conditions that:
the first accuracy is lower than a first threshold; the first accuracy is lower than second accuracy; or the first accuracy is lower than the second accuracy, and a difference between the first accuracy and the second accuracy is greater than a second threshold, wherein the second accuracy is used to indicate accuracy of a model output result presented by the first model in a training phase or a testing phase (see Liu: Fig.2, [0066], “The evaluation policy configuration module 2021 is used to configure an evaluation policy of the training model, including configuring at least one model evaluation metric used to evaluate the training model, an evaluation rule of the model evaluation metric, a preset threshold corresponding to the model evaluation metric, a selection rule of input data and an inference result that are used to evaluate the model evaluation metric, and a retraining trigger policy for the training model.”
Regarding Claim 11,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu further teaches the method wherein:
in the case that the first accuracy reaches the preset condition (see Huang: Fig.8, [0414], “Based on step 804a′ and step 804b′, if determining that a value corresponding to the model performance indicator reaches the performance threshold.”), the method further comprises:
requesting, by the first network element, to obtain a second model from a fifth network element, wherein the second model is a model provided by the fifth network element and used for the task (see Huang: Fig.8, [0417], “he training NWDAF sends, to the inference NWDAF, a model performance information request that carries the analytics ID and a model performance indicator (for example, precision, accuracy, error rate, recall rate, F1 score, mean squared error, root mean squared error, root mean squared logarithmic error, mean absolute error, model inference duration, model robustness, model expandability, or model interpretability).”) and
performing, by the first network element, inference on the task based on the second model (see Huang: Fig.8, [0442], “if step 804a′″ and step 804b′″ or step 804a″″ and step 804b″″ are performed, when determining, based on the model performance evaluation reference information, that the model performance reaches the performance threshold preset by the training NWDAF or does not meet the model performance requirement of the inference NWDAF, the training NWDAF determines to start model retraining.”); wherein the fifth network element comprises a model training function network element see Huang: Fig.1, [0116], “if step 804a′″ and step 804b′″ or step 804a″″ and step 804b″″ are performed, when determining, based on the model performance evaluation reference information, that the model performance reaches the performance threshold preset by the training NWDAF or does not meet the model performance requirement of the inference NWDAF, the training NWDAF determines to start model retraining.”)
See Claim 4 above for motivation to combine Liu and Haung.
Regarding Claim 12,
As shown above, Liu and Huang teach all the limitations of Claim 1. Huang further teaches the method wherein:
in the case that the first accuracy reaches the preset condition (see Huang: Fig.8, [0439], “if step 804a and step 804b are performed, when determining that the value corresponding to the model performance indicator reaches the performance threshold preset by the training NWDAF or does not meet the model performance requirement of the inference NWDAF, the training NWDAF determines to start model retraining.”), the method further comprises:
sending, by the first network element, fourth information to a third network element, wherein the fourth information is used to indicate that the accuracy of the first model does not meet the accuracy requirement or decreases (see Huang: Fig.13, [0614], “The first information includes the performance report of the model, and the performance report of the model indicates the performance evaluation result of the model, or the performance report of the model indicates that the performance evaluation result of the model does not meet a requirement for the performance indicator of the model.”), wherein the fourth information comprises at least one of the following:
all or a part of description information of the task; information indicating that the accuracy of the first model does not meet the accuracy requirement or decreases; the first accuracy; recommended operation information; or waiting time information, wherein the waiting time information is used to indicate a time required for the first network element to resume inference on the task (see Huang: Fig.5, [0230], “A determining basis may be that a performance evaluation result of the model does not meet a model performance requirement (for example, model precision decreases to less than 80%, where 80% is a model precision requirement), or a service key performance indicator (Key Performance Indicator, KPI) reported by the NF does not meet a KPI requirement (for example, the KPI decreases to below the KPI requirement).”
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention, to modify teaching of Liu’s retraining communication to include Huang’s sending of information indicating that the accuracy of the first model does not meet the accuracy requirement or decreases in the case that the first accuracy reaches the preset condition. After modification of Liu, the model providing server would receive an indication that the deployed model performance failed its requirement as taught by Huang. One would have been motivated to make such a combination to help the training server recognize a degraded model performance and initiate a timely retaining so that the use of inaccurate model is reduced.
Regarding Claim 13,
As shown above, Liu and Huang teach all the limitations of Claim 1. Liu further teaches the method wherein:
the first network element comprises a model inference function network element; or, the second network element comprises a model training function network element (see Liu: Fig.2, [0044], “modules are divided by function, the system used for AI model training and inference in the hybrid cloud scenario shown in FIG. 1 may include a data collection module 101, a model training module 102, a first model storage module 103, a second model storage module 104, and a model inference management module 105.”)
Regarding independent Claim 14,
Claim 14 is directed to a method claim and has similar/same claim limitation as Claim 1 and is rejected under the same rationale.
Regarding Claim 15,
Claim 15 is directed to a method claim and has similar/same claim limitation as Claim 2 and is rejected under the same rationale.
Regarding Claim 16,
Claim 16 is directed to a method claim and has similar/same claim limitation as Claim 3 and is rejected under the same rationale.
Regarding Claim 17,
Claim 17 is directed to a method claim and has similar/same claim limitation as Claim 6 and is rejected under the same rationale.
Regarding Claim 18,
Claim 18 is directed to a method claim and has similar/same claim limitation as Claim 5 and is rejected under the same rationale.
Regarding Claim 19,
Claim 19 is directed to a method claim and has similar/same claim limitation as Claim 1 and is rejected under the same rationale.
Regarding Claim 20,
Claim 20 is directed to an apparatus claim and has similar/same claim limitation as Claim 1 and is rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 20240281719 A1
DUAN; Xiaoyan
Title: MACHINE LEARNING MODEL PROCESSING METHOD AND APPARATUS, AND STORAGE MEDIUM
Description: The present disclosure relates to the field of communication technology, and in particular, to a machine learning model processing method and apparatus, and a storage medium.
US 20230100253 A1
ZHU; Xipeng
Title: NETWORK-BASED ARTIFICIAL INTELLIGENCE (AI) MODEL CONFIGURATION
Description: The present disclosure relates generally to wireless communications, and more specifically to configuration of a network-based artificial intelligence or machine learning (AI/ML) model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights