MODEL LEARNING METHOD, MODEL LEARNING APPARATUS, AND STORAGE MEDIUM

Application No. 18/560,372

• Filed: Nov 10, 2023
• Priority: May 14, 2021 — nonprovisional of PCTCN2021093927
• Examiner: WIDHALM DE RODRIG, ANGELA MARIE
• Art Unit: 2443
• Tech Center: 2400 — Computer Networks
• Assignee: (Beijing University Of Posts And Telecommunications)
• OA Round: 1 (Non-Final)

Office Action

DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .

This is a non-final office action in response to Application Number 18/560,372 filed on 10 November 2023; a preliminary amendment was also filed on 10 November 2023 in which claims 1-2, 4, 6-7, 9, 12, 15-16, and 19-20 were amended, claims 17-18 were cancelled, and claims 21-22 were added. The claims 1-16 and 19-22 are pending.
The instant application is a 371 of PCT/CN2021/093927 filed on 14 May 2021.
The applicants of record are Beijing Xiaomi Mobile Software Co., Ltd., and Beijing University of Posts and Telecommunications. The inventors of record are Qin Mu, Wei Hong, Zhongyuan Zhao, and Yifan Cai.

Information Disclosure Statement
The information disclosure statements (IDS) submitted on 10 November 2023 and 11 September 2025 were filed on or after the filing date of the instant application on 10 November 2023 and before the mailing date of the first office action on the merits. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.

Claim Interpretation
The claims have been considered according to the latest Patent Eligibility Guidelines and are considered eligible.

Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –

(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.

Claims 8, 15-16, and 21-22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yang et al. (CN 112668128 A)

Regarding claim 8, Yang disclosed a method for model learning, performed by a micro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers.” | [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model.”), and comprising:
receiving a model training request sent by a macro base station (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”); and
sending the model training request to a terminal (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”),
wherein a number of micro base stations receiving the model training request is a first number (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”); and
a communication coverage range of the first number of micro base stations is within a communication coverage range of the macro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers. The first MEC server within the macro base station has powerful computing and communication resources. Let Z represent the set of second MEC servers within a micro base station. Each second MEC server z∈Z has corresponding computing power and covers several terminal devices through the base station connected to it….”).

Regarding claim 15, Yang disclosed the method for model learning according to claim 8, further comprising:
sending terminal switching information, the terminal switching information comprising information of a terminal that exits model training and a target micro base station that the terminal re-accesses (see Yang [n0075]: “In the specific implementation process, for a federated learning task i∈I, the node selection problem can be summarized as selecting a selection set Z_i∈Z in each iteration to minimize the loss function of this training while keeping the terminal energy consumption within a preset range.”; [n0076]: “Step 102: Obtain the current environment state data corresponding to each federated learning iteration, and input the current environment state data into the terminal device node selection model to obtain the terminal device node selection strategy output by the terminal device node selection model corresponding to the test accuracy and the constraints.”; [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”; [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”), and the terminal switching information being used for redetermining, by the macro base station, a terminal that executes a model training task (see Yang [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”); and
redetermining the terminal that executes the model training task (see Yang [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”) on condition that terminal information sent by the macro base station is received (see Yang [n0075]: “In the specific implementation process, for a federated learning task i∈I, the node selection problem can be summarized as selecting a selection set Z_i∈Z in each iteration to minimize the loss function of this training while keeping the terminal energy consumption within a preset range.”; [n0076]: “Step 102: Obtain the current environment state data corresponding to each federated learning iteration, and input the current environment state data into the terminal device node selection model to obtain the terminal device node selection strategy output by the terminal device node selection model corresponding to the test accuracy and the constraints.”), and sending the model training task to the terminal (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”).

Regarding claim 16, Yang disclosed the method for model learning according to claim 15, wherein sending the model training task to the terminal comprises:
determining the target micro base station to which the terminal is switched on condition that a case that the terminal information comprises the terminal that executes the model training task last time, and sending, by the target micro base station, the model training task to the terminal (see Yang [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”; [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”); and/or
determining that the terminal does not execute the model training task any more on condition that a case that the terminal information does not comprise the terminal that executes the model training task last time, determining a newly-added terminal that executes the model training task, and sending the model training task to the newly-added terminal that executes the model training task.

Regarding claim 21, the claim contains the limitations, substantially as claimed, as described in claim 8 above. Examiner notes that claim 8 describes a method and claim 21 describes an apparatus that performs the method of claim 8.
Yang disclosed, as recited in claim 21: An apparatus for model learning, comprising:
a processor (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401…”); and
a memory configured to store an instruction executable by the processor (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401, a memory 402, and a communication bus 403…The processor 401 can call logic instructions in the memory 402 to execute a terminal device node selection method in the federated learning system…”), wherein the processor is configured to load and execute the instructions (see Yang Fig. 4, [n0124]: “…The processor 401 can call logic instructions in the memory 402 to execute a terminal device node selection method in the federated learning system…”) to implement the method for model learning according to claim 8.

Regarding claim 22, the claim contains the limitations, substantially as claimed, as described in claim 8 above. Examiner notes that claim 8 describes a method and claim 22 describes a non-transitory computer-readable storage medium that stores an instruction enabling a processor to perform the method of claim 8.
Yang disclosed, as recited in claim 22: A non-transitory computer-readable storage medium configured to store an instruction (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401, a memory 402, and a communication bus 403…The processor 401 can call logic instructions in the memory 402 to execute a terminal device node selection method in the federated learning system…”) that, when executed by a processor of a mobile terminal, enables the mobile terminal to execute the method for model learning according to claim 8.

Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.

Claims 1-7 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (CN 112668128 A) in view of Berggren et al. (U.S. Patent Publication 2022/0321647).

Regarding claim 1, Yang disclosed a method for model learning, performed by a macro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers.” | [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model.”), and comprising:
sending a model training request to a first number of micro base stations (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”) on condition that the model training request sent by an operation administration and maintenance (OAM) entity is received (see Berggren combination below),
wherein a communication coverage range of the first number of micro base stations being within a communication coverage range of the macro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers. The first MEC server within the macro base station has powerful computing and communication resources. Let Z represent the set of second MEC servers within a micro base station. Each second MEC server z∈Z has corresponding computing power and covers several terminal devices through the base station connected to it….”).

Yang did not explicitly disclose that the training request being sent to the micro base stations is sent “on condition that the model training request sent by an operation administration and maintenance (OAM) entity is received”.
While it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the training request would have been first sent from an OAM entity for further distribution, however in a related art, Berggren disclosed “FIG. 9 shows a signal flow diagram between a UE (920) and a network element (910), according to various exemplary embodiments of the present disclosure. In FIG. 9, the “network element” can be any node or function in, or associated with, a wireless network that is responsible for provisioning of an ML model to TEEs in UEs served by the wireless network. For example, the network element can be a RAN node (e.g., eNB, gNB, etc.), a core network node or function (e.g., NF in 5GC), an application function (AF), etc.” (see Berggren [0164]). The network node includes (see Berggren Fig. 18, [0297]) an OAM interface (see Berggren [0304], Fig. 18 #1860). Also, “In operation 8, the network element provisions and/or distributes the ML model to the TEE via the secure connection previously established. Before doing so, the network element can train the ML model based on the received capabilities of the UE and/or the TEE…” (see Berggren Fig. 11, [0187]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify the involvement of the OAM entity during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 2, Yang-Berggren disclosed the method for model learning according to claim 1, wherein the model training request is configured for triggering the micro base stations to report capability information (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…” | [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission; second, the training quality, dataset quality, and scale of terminal devices all have a significant impact on federated learning performance; in addition, edge terminal devices have limited energy, and energy consumption needs to be appropriately controlled when participating in training tasks….” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….” || Berggren [0165]: “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE…”; [0185]: “In operation 1, the network element sends a request to the UE ML client to prepare and verify a TEE for the ML model...After the TEE has been created, in operation 3 the ML client requests a remote attestation (also referred to as “cryptographic attestation or “quote”) pertaining to the TEE. In operation 4, the TEE sends the ML client the requested remote attestation…”; [0186]: “In operation 5, the ML client forwards the information received in operation 4 to the network element. In operation 6, the network element verifies the evidence of trust included in the remote attestation, e.g., against security requirements and/or parameters required to trust a TEE for provisioning and execution of the ML model…if the verification succeeds, in operation 7 the network element establishes a secure connection with the TEE using the endpoint information received in operation 5…”; [0187]: “In operation 8, the network element provisions and/or distributes the ML model to the TEE via the secure connection previously established. Before doing so, the network element can train the ML model based on the received capabilities of the UE and/or the TEE…The Communication network can then train an ML model by minimizing the loss…”); and the method further comprises:
determining a model structure and a model parameter value (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”) based on the capability information on condition that the capability information sent by the micro base stations is received (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission; second, the training quality, dataset quality, and scale of terminal devices all have a significant impact on federated learning performance; in addition, edge terminal devices have limited energy, and energy consumption needs to be appropriately controlled when participating in training tasks….” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….”), and sending the model structure and the model parameter value to the micro base stations (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”); and
the model structure being a model structure that the micro base stations are indicated to train based on the model training request, and the model parameter value being an initial parameter value of the model structure (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify the involvement of the OAM entity during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 3, Yang-Berggren disclosed the method for model learning according to claim 2, wherein the capability information comprises a data type feature (see Berggren [0165]: “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE. Although not shown in FIG. 9, in some embodiments the network element can receive other indicia of trust for the UE from other NFs, such as:”; [0166]: “an indication of whether data traffic for the UE is according to a type associated with the UE”) of the micro base stations (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission...” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….”); and the method further comprises:
receiving a first number of first model training results sent by the first number of micro base stations (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.”; [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model…”);
determining data type features (see Berggren [0165]: “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE. Although not shown in FIG. 9, in some embodiments the network element can receive other indicia of trust for the UE from other NFs, such as:”; [0166]: “an indication of whether data traffic for the UE is according to a type associated with the UE”) of different micro base stations in the first number of micro base stations (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission...” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….”), and determining a first model loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task, which can be expressed by the following formula:”; n[0048]: “w = argminL(w) (3)”);
unifying the data type features based on the data type features of the different micro base stations in the first number of micro base stations (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model. The specific weight aggregation is as follows:”; [n0055]: “Where: |H_z,d| represents the size of the dataset in which terminal device d participates in the federated learning task, and |H_i| represents the sum of the datasets of all terminal devices participating in the current federated learning task.”), and then performing first model alignment on the first number of first model training results (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model…”) with optimizing the first model loss function as an objective (see Yang [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task...”); and
determining a global model by performing global model learning based on a result of first model alignment (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify the involvement of the OAM entity during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 4, Yang-Berggren disclosed the method for model learning according to claim 3, wherein determining the global model by performing global model learning based on the result of first model alignment comprises:
sending a model learning result to the micro base stations on condition that a case that the model learning result of global model learning does not meet the model training request of the OAM entity, and receiving the first number of first model training results redetermined by the micro base stations based on the model learning result (see Yang [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”);
redetermining a first model loss function based on the model learning result of global model learning, and re-performing first model alignment on the first number of received first model training results (see Yang [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”) with optimizing the redetermined first model loss function as an objective (see Yang [n0036]: “Specifically, the test accuracy optimization target model is used to minimize the overall loss function of the terminal device nodes participating in each federated learning iteration, and satisfies the preset constraints; the overall loss function of the terminal device nodes is used to represent the test accuracy…” | [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task…”); and
redetermining a model learning result by performing global model learning next time based on the redetermined result of first model alignment until the model learning result meets the model training request (see Yang [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”), and determining a model corresponding to the model learning result that meets the model training request as the global model (see Yang [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”; [n0117]: “After the above training, the present invention finally obtains a well-trained terminal device node selection model…Finally, the terminal device node selection strategy that meets the actual needs is output.”).

Regarding claim 5, Yang-Berggren disclosed the method for model learning according to claim 4, wherein determining the first model loss function comprises:
determining a first loss function between the first number of first model training results of the micro base stations and the model learning result obtained by global model learning last time of the macro base station, and a first model alignment loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy, as shown in the following formula:…”); and
determining the first model loss function based on the first loss function and the first model alignment loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy, as shown in the following formula:…”).

Regarding claim 6, Yang-Berggren disclosed the method for model learning according to claim 3, wherein determining the global model by performing global model learning based on the result of first model alignment comprises:
sending information for stopping model training to the micro base stations on condition that a case that a model learning result of global model learning meets the model training request (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...”, i.e. stop training after a certain number of iterations is reached | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. stop training once conditions are met) of the OAM entity (see Berggren Fig. 18 #1860, [0304]: The network node includes an OAM interface), the information for stopping training indicating the micro base stations to stop a terminal from executing a model training task (see Yang [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. stop training once conditions are met); and
determining a model corresponding to the model learning result as the global model, and sending the global model (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…” | [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...”; [n0056]: “Step 204: Issuance of new parameters after aggregation.”) to the OAM entity (see Berggren Fig. 18 #1860, [0304]: The network node includes an OAM interface).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify the involvement of the OAM entity during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 7, Yang-Berggren disclosed the method for model learning according to claim 1, further comprising:
redetermining a terminal that executes a model training task (see Yang [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”) based on terminal switching information on condition that the terminal switching information sent by the micro base stations in a model training process is received (see Yang [n0075]: “In the specific implementation process, for a federated learning task i∈I, the node selection problem can be summarized as selecting a selection set Z_i∈Z in each iteration to minimize the loss function of this training while keeping the terminal energy consumption within a preset range.”; [n0076]: “Step 102: Obtain the current environment state data corresponding to each federated learning iteration, and input the current environment state data into the terminal device node selection model to obtain the terminal device node selection strategy output by the terminal device node selection model corresponding to the test accuracy and the constraints.”), and sending information of the terminal to the micro base stations (see Yang [n0075]: “In the specific implementation process, for a federated learning task i∈I, the node selection problem can be summarized as selecting a selection set Z_i∈Z in each iteration to minimize the loss function of this training while keeping the terminal energy consumption within a preset range.”; [n0076]: “Step 102: Obtain the current environment state data corresponding to each federated learning iteration, and input the current environment state data into the terminal device node selection model to obtain the terminal device node selection strategy output by the terminal device node selection model corresponding to the test accuracy and the constraints.”);
the terminal switching information comprising information of a terminal that exits model training and a target micro base station that the terminal re-accesses (see Yang [n0075]: “In the specific implementation process, for a federated learning task i∈I, the node selection problem can be summarized as selecting a selection set Z_i∈Z in each iteration to minimize the loss function of this training while keeping the terminal energy consumption within a preset range.”; [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”; [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”);
the terminal switching information being used for redetermining, by the macro base station, the terminal that executes the model training task (see Yang [n0077]: “Specifically, the terminal device node selection strategy is used to determine the terminal device nodes participating in each federated learning iteration to achieve federated learning model training…”).

Regarding claim 19, the claim contains the limitations, substantially as claimed, as described in claim 1 above. Examiner notes that claim 1 describes a method and claim 19 describes an apparatus that performs the method of claim 1.
Yang disclosed, as recited in claim 19: An apparatus for model learning, comprising:
a processor (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401…”); and
a memory configured to store an instruction executable by the processor (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401, a memory 402, and a communication bus 403…The processor 401 can call logic instructions in the memory 402 to execute a terminal device node selection method in the federated learning system…”),
wherein the processor is configured to:
send a model training request to a first number of micro base stations (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.” | [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model.”) on condition that the model training request sent by an operation administration and maintenance (OAM) entity is received (see Berggren combination below),
wherein a communication coverage range of the first number of micro base stations being within a communication coverage range of a macro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers. The first MEC server within the macro base station has powerful computing and communication resources. Let Z represent the set of second MEC servers within a micro base station. Each second MEC server z∈Z has corresponding computing power and covers several terminal devices through the base station connected to it….”).

Yang did not explicitly disclose that the training request being sent to the micro base stations is sent “on condition that the model training request sent by an operation administration and maintenance (OAM) entity is received”.
While it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the training request would have been first sent from an OAM entity for further distribution, however in a related art, Berggren disclosed “FIG. 9 shows a signal flow diagram between a UE (920) and a network element (910), according to various exemplary embodiments of the present disclosure. In FIG. 9, the “network element” can be any node or function in, or associated with, a wireless network that is responsible for provisioning of an ML model to TEEs in UEs served by the wireless network. For example, the network element can be a RAN node (e.g., eNB, gNB, etc.), a core network node or function (e.g., NF in 5GC), an application function (AF), etc.” (see Berggren [0164]). The network node includes (see Berggren Fig. 18, [0297]) an OAM interface (see Berggren [0304], Fig. 18 #1860). Also, “In operation 8, the network element provisions and/or distributes the ML model to the TEE via the secure connection previously established. Before doing so, the network element can train the ML model based on the received capabilities of the UE and/or the TEE…” (see Berggren Fig. 11, [0187]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify the involvement of the OAM entity during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 20, the claim contains the limitations, substantially as claimed, as described in claim 1 above. Examiner notes that claim 1 describes a method and claim 20 describes a non-transitory computer-readable storage medium that stores an instruction enabling a processor to perform the method of claim 1.
Yang-Berggren disclosed, as recited in claim 20: A non-transitory computer-readable storage medium configured to store an instruction (see Yang Fig. 4, [n0124]: “…The electronic device may include a processor 401, a memory 402, and a communication bus 403…The processor 401 can call logic instructions in the memory 402 to execute a terminal device node selection method in the federated learning system…”) that, when executed by a processor of a mobile terminal, enables the mobile terminal to execute the method for model learning according to claim 1.

Claims 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (CN 112668128 A) as applied to claim 8 above, and further in view of Berggren et al. (U.S. Patent Publication 2022/0321647).

Regarding claim 9, Yang disclosed the invention, substantially as claimed, as described in the method for model learning according to claim 8 above, further wherein the capability information is used for determining (see Yang [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission; second, the training quality, dataset quality, and scale of terminal devices all have a significant impact on federated learning performance; in addition, edge terminal devices have limited energy, and energy consumption needs to be appropriately controlled when participating in training tasks….” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….”), by the macro base station (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”), a model structure and a model parameter value (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”).

Yang did not explicitly disclose the entirety of “wherein the model training request is used for triggering the terminal to report a communication condition and a data type feature of the terminal; and after sending the model training request to the terminal, the method for model learning further comprises: receiving the communication condition and the data type feature sent by the terminal; and obtaining capability information by processing the communication condition and the data type feature of the terminal and a communication condition and a data type feature of the micro base stations, and sending the capability information to the macro base station”.
However in a related art Berggren disclosed “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE. Although not shown in FIG. 9, in some embodiments the network element can receive other indicia of trust for the UE from other NFs, such as:” (see Berggren [0165]) “an indication of whether data traffic for the UE is according to a type associated with the UE” (see Berggren [0166]). Also, “In operation 1, the network element sends a request to the UE ML client to prepare and verify a TEE for the ML model...After the TEE has been created, in operation 3 the ML client requests a remote attestation (also referred to as “cryptographic attestation or “quote”) pertaining to the TEE. In operation 4, the TEE sends the ML client the requested remote attestation…” (see Berggren Fig. 11, [0185]). “In operation 5, the ML client forwards the information received in operation 4 to the network element. In operation 6, the network element verifies the evidence of trust included in the remote attestation, e.g., against security requirements and/or parameters required to trust a TEE for provisioning and execution of the ML model…if the verification succeeds, in operation 7 the network element establishes a secure connection with the TEE using the endpoint information received in operation 5…” (see Berggren Fig. 11, [0186]). “In operation 8, the network element provisions and/or distributes the ML model to the TEE via the secure connection previously established. Before doing so, the network element can train the ML model based on the received capabilities of the UE and/or the TEE…The Communication network can then train an ML model by minimizing the loss…” (see Berggren Fig. 11, [0187]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify how device capabilities are determined and used during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 10, Yang-Berggren disclosed the method for model learning according to claim 9, further comprising:
receiving the model structure and the model parameter value, the model structure being a model structure that the micro base stations are indicated to train based on the model training request, and the model parameter value being an initial parameter value of the model structure (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.””; [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.”; [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model…”);
determining a second number of terminals that execute model training based on the communication condition (see Yang [n0058]: “In the aforementioned federated learning training process, the selection of network edge terminal device nodes is affected by many factors: First, the differentiated computing and communication capabilities of terminal devices directly affect the latency of local training and data transmission...” | [n0081]: “During the selection of terminal device nodes in federated learning, the MEC server in the federated learning system can act as an agent to interact with external environmental data. At each moment, the agent can obtain the current environmental state data and take actions based on the current environmental state data….” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”; examiner notes that “second number” is interpreted as being functionally equivalent to the number of terminals in another iteration of the training process) and the data type feature of the terminal (see Berggren [0165]: “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE. Although not shown in FIG. 9, in some embodiments the network element can receive other indicia of trust for the UE from other NFs, such as:”; [0166]: “an indication of whether data traffic for the UE is according to a type associated with the UE”), the model structure and the model parameter value (see Yang [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.” | [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”); and
sending scheduling information to the second number of terminals, wherein the scheduling information comprises the model structure, the model parameter value as well as indication information for indicating the terminals to perform model training (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…For federated learning tasks i∈I such as path selection and image recognition, the goal is to learn a task-related federated learning model M from the sample data set H_z,d＝{x_z,d,y_z,d} of the terminal device. Therefore, a federated learning task can be defined as follows: where Z_i and D_i represent the sets of second MEC servers and terminal devices associated with federated learning task i, respectively; C_i is the number of CPU cycles required for the federated learning model to process a set of data in the dataset; and is the initial model for the federated learning task.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify how device capabilities are determined and used during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 11, Yang-Berggren disclosed the method for model learning according to claim 10, further comprising:
receiving a second number of second model training results sent by a second number of terminals (see Yang [n0037]: “In the specific implementation process, the federated learning network model is first constructed, which consists of terminal devices, micro base stations, macro base stations and their corresponding MEC (Mobile Edge Computer) servers…”; [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.”; [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.” | examiner notes that “second number” is interpreted as referring to the results and terminals in a repetition of the training process);
determining data type features that different terminals in the second number of terminals have (see Berggren [0165]: “Initially, the network element requests ML-related characteristics from the UE, and the UE responds with its ML-related characteristics. These characteristics can be any of those listed above, as well as any others that are relevant for the network element to decide on provisioning of an ML model to the UE. Although not shown in FIG. 9, in some embodiments the network element can receive other indicia of trust for the UE from other NFs, such as:”; [0166]: “an indication of whether data traffic for the UE is according to a type associated with the UE”), and determining a second model loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task, which can be expressed by the following formula:”; [n0048]: “w = argminL(w) (3)”);
unifying the data type features based on the data type features that the different terminals in the second number of terminals have (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model. The specific weight aggregation is as follows:”; [n0055]: “Where: |H_z,d| represents the size of the dataset in which terminal device d participates in the federated learning task, and |H_i| represents the sum of the datasets of all terminal devices participating in the current federated learning task.”), and then performing second model alignment on the second number of second model training results (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations has been reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”) with optimizing the second model loss function as an objective (see Yang [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task...”); and
obtaining a first model training result by performing federated aggregation based on a result of second model alignment (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...” | [n0038]: “The federated learning process corresponding to the federated learning network model is shown in Figure 2, which mainly includes: Step 201: Local training of the model; Step 202: Uploading of model weights and parameters; Step 203: Aggregation of the model after parameter uploading; Step 204: Distribution of new parameters after aggregation.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang and Berggren to further describe the model training process and clarify how device capabilities are determined and used during model training and provisioning. Incorporating Berggren’s teachings into Yang’s federated learning model would improve network performance (see Berggren [0057]), reduce signaling and processing in the network (see Berggren [0120]), and allow the network to train, provision, and manage ML models based on trusted execution environment capabilities without exposing the ML models to third parties (see Berggren [0134]).

Regarding claim 12, Yang-Berggren disclosed the method for model learning according to claim 11, wherein obtaining the first model training result by performing federated aggregation based on the result of second model alignment comprises:
receiving a model learning result sent by the macro base station on condition that a continue-to-train request sent by the macro base station is received (see Yang [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. continue to train and send results);
updating the model structure and the model parameter value of the terminal based on the model learning result, and sending continue-to-train scheduling information to the terminal (see Yang [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. update and continue-to-train);
redetermining the second model loss function based on the first model training result on condition that a second number of second model training results is re-received, and performing second model alignment on the second number of second model training results (see Yang [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”) with optimizing the redetermined second model loss function as an objective (see Yang [n0036]: “Specifically, the test accuracy optimization target model is used to minimize the overall loss function of the terminal device nodes participating in each federated learning iteration, and satisfies the preset constraints; the overall loss function of the terminal device nodes is used to represent the test accuracy…” | [n0047]: “In this invention, the goal of federated learning is to optimize the global model parameters by minimizing the loss function L(w) of the federated learning task…”); and
redetermining a first model training result by performing federated aggregation next time based on a redetermined result of second model alignment (see Yang [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”).

Regarding claim 13, Yang-Berggren disclosed the method for model learning according to claim 12, wherein determining the second model loss function comprises:
determining a second loss function between the second number of second model training results of the terminals and the first model training result of the micro base stations obtained by federated aggregation last time, and a second model alignment loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy, as shown in the following formula:…” | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.” | examiner notes that “second” is interpreted as referring to the results and loss functions in a repetition of the training process); and
determining the second model loss function based on the second loss function and the second model alignment loss function (see Yang [n0044]: “The loss function l_z,d(x_z,d, y_z,d, w_z,d) for local training of the model on terminal device d can be defined as the difference between its predicted value and the actual value on the sample dataset H_z,d…” | [n0059]: “Regarding accuracy: For a federated learning task i∈I, its training quality can be defined as the test accuracy after the local terminal device completes N training iterations. Specifically, this invention uses the sum of loss functions of the test dataset to represent the test accuracy, as shown in the following formula:…”).

Regarding claim 14, Yang-Berggren disclosed the method for model learning according to claim 12, further comprising:
receiving information for stopping model training sent by the macro base station, the information for stopping training indicating the micro base stations to stop the terminal from executing a model training task (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...”, i.e. stop training after a certain number of iterations is reached | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. stop training once conditions are met); and
indicating, based on the information for stopping model training, the terminal to stop executing the model training task (see Yang [n0053]: “Once the uploaded local model parameters reach a certain number or the number of iterations is reached, the first MEC server at the macro base station will perform global model aggregation on the obtained local model...”, i.e. stop training after a certain number of iterations is reached | [n0109]: “Step 1: Input the initial state parameters into the Actor1 network…”; [n0110]: “Step 2: The global PPO network model is calculated…”; [n0113]: “Step 3: Calculate the loss function…”; [n0114]: “Step 4: Update the parameters of the actor network…”; [n0115]: “Step 5: Repeat step 4, and after the preset steps, update the parameters of Actor2 using the network parameters in Actor1.”; [n0116]: “Step 6: Repeat steps 1-5 until the model converges.”, i.e. stop training once conditions are met).

Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Angela Widhalm de Rodriguez whose telephone number is [phone]. The examiner can normally be reached M-F: 6am-2:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nicholas Taylor can be reached at [phone]. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.

/ANGELA WIDHALM DE RODRIGUEZ/Examiner, Art Unit 2443

Prosecution Timeline

• Nov 10, 2023: Application Filed
• Apr 22, 2026: Non-Final Rejection mailed — §102, §103 (current)