Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant's election with traverse of Group I (claims 49-57 and 72) in the reply filed on 11/21/2025 is acknowledged. The traversal is on the ground that the groups of invention all share the same or corresponding technical feature. This is not found persuasive.
According to MPEP 1850, unity of invention exists a priori when the multiple claimed inventions all share the same or corresponding technical feature. In this instant, no technical feature is shared across all of claimed inventions of claims 49 (group I), 58 (group II), and 69 (group III). Specifically, claim 49 recites, "receiving, from a model training logical function (MTLF) of the NWDAF or of a second NWDAF, a subscription request for drift monitoring notification associated with a machine learning (ML) model used by an analytics logical function (AnLF) of the NWDAF"; claim 58 recites, "sending, to an analytics logical function (AnLF) of the NWDAF or of a second NWDAF, a subscription request for drift monitoring notifications associated with a machine learning (ML) model used by the AnLF of the NWDAF or of the second NWDAF"; and claim 69 recites, sending, to a model training logical function (MTLF) of the NWDAF, a subscription request for notification associated with the ML model". The technical features in the inventions of claims 49 and 58 include "of a second NWDAF", which is not shared in the invention of claim 69. Furthermore, claim 69 recites the technical feature of "receiving a notification to terminate use of the ML model from the MTLF based on the subscription request", which is not shared in the invention of claim 49.
The requirement is still deemed proper and is therefore made FINAL.
Clams 49-57 and 72 have been examined.
Claims 58-59, 61-71 and 73-74 have been withdrawn from consideration.
Allowable Subject Matter
Claims 52 and 55-56 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reason for Allowance
The following is an examiner’s statement of reasons for allowance: The cited prior arts fail to teach Applicant’s invention performed by a network data analytics function (NWDAF) of a communication network for monitoring drift associated with a machine learning (ML) model, comprising: receiving, from a model training logical function (MTLF) of the NWDAF or of a second NWDAF, a subscription request for drift monitoring notifications associated with a machine learning (ML) model used by an analytics logical function (AnLF) of the NWDAF; monitoring for drift associated with the ML model, based on metadata associated with the ML model; and based on the monitoring meeting one or more criteria included in the metadata, sending one or more drift monitoring notifications to the MTLF in accordance with the subscription; wherein: the monitoring is performed on one or more monitoring objects; and each monitoring object is associated with at least one of the following types of data: raw data acquired by the AnLF for input to the ML model; feature vectors computed by the AnLF based on the raw data; predictions based on the ML model; and actual values corresponding to the predictions based on the ML model, when such actual values are available; wherein: the metadata associated with the ML model includes a drift detection configuration; and the drift detection configuration includes one or more of the following for each particular one of the monitoring objects: access information for data to be used for the particular monitoring object; specific attributes and/or subsets of data to be used for the particular monitoring object; and size, duration, and/or sampling ratio of data to be used for the particular monitoring object; and wherein the drift detection configuration also includes: for each particular monitoring object associated with raw data or feature vectors, identification of supported data drift tests; and for each particular monitoring object associated with predictions based on the ML model, identification of relevant performance metrics and one or more thresholds for each relevant performance metric.
The cited prior arts fail to teach Applicant’s invention performed by a network data analytics function (NWDAF) of a communication network for monitoring drift associated with a machine learning (ML) model, comprising: receiving, from a model training logical function (MTLF) of the NWDAF or of a second NWDAF, a subscription request for drift monitoring notifications associated with a machine learning (ML) model used by an analytics logical function (AnLF) of the NWDAF; monitoring for drift associated with the ML model, based on metadata associated with the ML model; and based on the monitoring meeting one or more criteria included in the metadata, sending one or more drift monitoring notifications to the MTLF in accordance with the subscription; wherein the metadata includes one or more of the following: a monitoring or notification period, and one or more performance thresholds; and each of the one or more drift monitoring notifications is sent to the MTLF based on the monitoring meeting one or more of the following criteria related to the metadata: periodic availability of one or more monitored performance metrics according to the monitoring or notification period, or a relation between one or more monitored performance metrics and corresponding ones of the performance thresholds, indicating that drift has occurred; and wherein monitoring for drift is performed on one or more monitoring objects and comprises, at each monitoring or notification period: evaluating respective performance metrics for the monitoring objects; determining respective first relations between the respective performance metrics for the monitoring objects and respective performance thresholds associated with the monitoring objects; and determining whether drift has occurred based on a second relation among the respective first relations; and when it is determined that drift has occurred, determining whether the drift is severe based on a termination threshold.
Claim Rejections – 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 49-51, 53-54, 57 and 72 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Marquezan et al, WO 2023/006212 (hereinafter Marquezan).
As per claim 49, Marquezan teaches the invention as claimed performed by a network data analytics function (NWDAF) of a communication network for monitoring drift associated with a machine learning (ML) model (page. 46, line 28-page 47, lines 12), the method comprising:
receiving, from a model training logical function (MTLF) of the NWDAF or of a second NWDAF, a subscription request for drift monitoring notifications associated with a machine learning (ML) model used by an analytics logical function (AnLF) of the NWDAF (page 41, lines 1-7; page 57, lines 8-9; page 58, lines 11-21, e.g., receiving, from NWDAF (MTLF) subscription to receive analytic notifications associated with ML model used by inference entity/AnLF of NWDAF);
monitoring for drift associated with the ML model, based on metadata associated with the ML model (page 24, lines 29-page 25,lines 30; page 34, line 25-page 35, line 24, e.g., monitoring unstable analytics associated with ML model based on subscription ML metadata); and
based on the monitoring meeting one or more criteria included in the metadata, sending one or more drift monitoring notifications to the MTLF in accordance with the subscription (page 28, lines 6-17; page 30, lines 17-27; page 36, lines 22-26; page 57, lines 15-16, e.g., based on the monitoring of ML output crossing a threshold, sending notifications to MTLF in accordance with the subscription).
As per claim 50, Marquezan teaches the invention as claimed in claim 49. Marquezan further teach wherein: the monitoring is performed on one or more monitoring objects (page 19, lines 24-32; page 20, lines 26-30; page 21, line 32-page 22, line 10); and
each monitoring object is associated with at least one of the following types of data:
raw data acquired by the AnLF for input to the ML model;
feature vectors computed by the AnLF based on the raw data;
predictions based on the ML model; and
actual values corresponding to the predictions based on the ML model, when such actual values are available (page 19, lines 24-32; page 20, lines 26-30; page 21, line 32-page 22, line 10).
As per claim 51, Marquezan teaches the invention as claimed in claim 50. Marquezan further teach wherein:
the metadata associated with the ML model includes a drift detection configuration (page 48, lines 22-32; page 57, lines 7-27; page 58, lines 24-27; page 59, lines 6-15); and
the drift detection configuration includes one or more of the following for each particular one of the monitoring objects:
access information for data to be used for the particular monitoring object;
specific attributes and/or subsets of data to be used for the particular monitoring object; and
size, duration, and/or sampling ratio of data to be used for the particular monitoring object (page 48, lines 22-32; page 57, lines 7-27; page 58, lines 24-27; page 59, lines 6-15).
As per claim 53, Marquezan teaches the invention as claimed in claim 49. Marquezan further teach the metadata includes one or more of the following: a monitoring or notification period, and one or more performance thresholds (page 57, lines 8-9; page 24, line 29-page 25, line 20); and
each of the one or more drift monitoring notifications is sent to the MTLF based on the monitoring meeting one or more of the following criteria related to the metadata (page 57, lines 8-9; page 24, line 29-page 25, line 20):
periodic availability of one or more monitored performance metrics according to the monitoring or notification period, or
a relation between one or more monitored performance metrics and corresponding ones of the performance thresholds, indicating that drift has occurred (page 57, lines 8-9; page 24, line 29-page 25, line 20).
As per claim 54, Marquezan teaches the invention as claimed in claim 53. Marquezan further teach wherein:
monitoring for drift is performed on one or more monitoring objects (page 19, lines 24-32; page 20, lines 26-30; page 21, line 32-page 22, line 10); and
each drift monitoring notification includes one or more the following:
an identifier of the ML model or of analytics associated with the drift monitoring,
a timestamp,
observed drift levels associated with each monitoring object,
an identifier of a data drift test or performance metric used for each monitoring object, and
a value of a monitored performance metric for the type of data associated with each monitoring object (page 25, lines 10-20; page 27, line 25-page 28, line 17; page 40, lines 19-32; page 57, lines 23-25).
As per claim 57, Marquezan teaches the invention as claimed in claim 49. Marquezan further teach wherein the subscription request includes one or more of the following:
the metadata associated with the ML model;
one or more analytics identifiers associated with the drift monitoring;
one or more ML model identifiers associated with the drift monitoring;
an address to send drift monitoring notifications;
a timestamp of the subscription request; and
a duration of validity for the subscription request (page 48, lines 22-32; page 57, lines 7-27; page 58, lines 24-27; page 59, lines 6-15).
As per claim 72, Marquezan teaches network equipment configured to implement a drift detection logical function (DDLF) of a network data analytics function (NWDAF) of a communication network (page 46, lines 28-32), wherein:
the network equipment comprises communication interface circuitry and processing circuitry that are operably coupled (fig. 3; page 29, line 7-page 30, line 8); and
the processing circuitry and the communication interface circuitry are configured to perform the method of claim 49 (see citations and explanation set forth in the rejection of claim 49 above).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should
be directed to Philip Lee whose telephone number is (571)272-3967. The examiner can normally be
reached on 6a-3p M-F.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
Glenton Burgess can be reached on 571-272-3949. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application
Information Retrieval (PAIR) system. Status information for published applications may be obtained
from either Private PAIR or Public PAIR. Status information for unpublished applications is available
through Private PAIR only. For more information about the PAIR system, see http://pair- direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer
Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR
CANADA) or 571-272-1000.
/PHILIP C LEE/Primary Examiner, Art Unit 2454