DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 12/01/2023, said application claims a priority filing date of 06/02/2021. Claims 1-13, 16, and 18-23 are pending. Claims 1, 10, and 16 are independent.
Specification
The use of the term "WiFi" and "Bluetooth" in ¶ [0311], which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Objections
Claims 4, 6-7, 10-11, 20, and 22-23 are objected to because of the following informalities:
in Claim 4, lines 3-7, "… determining input layers and hidden layers of the model training structures corresponding to the first number of radio access network devices as shared model layers, obtaining data of the plurality of radio access network devices, and adding, to the data, datum identifiers corresponding to respective radio access network devices …" appears to be "… determining input layers and hidden layers of the first number of model training structures corresponding to the first number of wireless access network devices as shared model layers, obtaining data of the plurality of wireless access network devices, and adding, to the data, datum identifiers corresponding to respective wireless access network devices …";
in Claim 6, lines 4-7, "… determining current model parameters and current model learning rates of the shared model layers; and determining update parameters of the shared model layers according to the weighted loss values, the model parameters and the model learning rates …" appears to be "… determining update parameters of the shared model layers according to the weighted loss values, the current model parameters and the current model learning rates …";
in Claim 7, lines 6-7, "… sending the model structural parameters of the shared model layers obtained after the Tth update …" appears to be "… sending the structural parameters of the shared model layers obtained after the Tth update …";
in Claim 7, lines 9-10, "… wherein T is a predetermined number of times to update the shared model layers and the unique model layers …" appears to be "… wherein T is a predetermined number of times to update the shared model layers and the first number of unique model layers …";
in Claim 10, lines 3-6, "… receiving a structural parameter of a unique model layer sent by an OAM, wherein the unique model layer is determined by the OAM … and the first number of model training structures are determined by the OAM …" appears to be "… receiving a structural parameter of a unique model layer sent by an operation administration and maintenance (OAM) entity, wherein the unique model layer is determined by the OAM entity … and the first number of model training structures are determined by the OAM entity …" according to Claim 1
in Claim 11, line 3, "… receiving model label values and first output data sent by the OAM …" appears to be "… receiving model label values and first output data sent by the OAM entity …";
in Claim 11, line 3, "… receiving model label values and first output data sent by the OAM …" appears to be "… receiving model label values and first output data sent by the OAM entity …";
in Claim 11, line 8, "… sending the training loss value to the OAM" appears to be "… sending the training loss value to the OAM entity";
in Claim 20, lines 3-7, "… determining input layers and hidden layers of the model training structures corresponding to the first number of radio access network devices as shared model layers, obtaining data of the plurality of radio access network devices, and adding, to the data, datum identifiers corresponding to respective radio access network devices …" appears to be "… determining input layers and hidden layers of the first number of model training structures corresponding to the first number of wireless access network devices as shared model layers, obtaining data of the plurality of wireless access network devices, and adding, to the data, datum identifiers corresponding to respective wireless access network devices …";
in Claim 22, lines 4-7, "… determining current model parameters and current model learning rates of the shared model layers; and determining update parameters of the shared model layers according to the weighted loss values, the model parameters and the model learning rates …" appears to be "… determining update parameters of the shared model layers according to the weighted loss values, the current model parameters and the current model learning rates …";
in Claim 23, lines 6-7, "… sending the model structural parameters of the shared model layers obtained after the Tth update …" appears to be "… sending the structural parameters of the shared model layers obtained after the Tth update …"; and
in Claim 23, lines 9-10, "… wherein T is a predetermined number of times to update the shared model layers and the unique model layers …" appears to be "… wherein T is a predetermined number of times to update the shared model layers and the first number of unique model layers …".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9, 11-12, 16, and 18-23 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 16 recite the limitation "… obtaining at least one wireless access network device group by grouping a plurality of wireless access network devices sending model subscription requests, the wireless access network device group comprising a first number of wireless access network devices …" in lines 3-6 and 9-12 respectively, which rendering these claims indefinite because (1) it.
Claims 2-9 and 18-23 are rejected for fully incorporating the deficiency of their respective base claims.
Claims 7 and 23 recite the limitation "… each wireless access network device in the plurality of wireless access network device groups …" in lines 7-8. There is insufficient antecedent basis for the limitation "the plurality of wireless access network device groups" in the claim. For examination purposes, "… each wireless access network device in the plurality of wireless access network devices …" is considered.
Claims 7 and 23 recite the limitation "... the structural parameters of the shared model layers are configured for the wireless access network devices to synthesize a model to which the wireless access network device subscribes" in 10-12, which rendering these claims indefinite because ".
Claim 8 recites the limitation "... obtaining the first number of wireless access network device groups by grouping the wireless access network devices" in line, which rendering the claim indefinite because ".
Claim 9 recites the limitation "the structural parameter of the unique model layer" in line 5. There is insufficient antecedent basis for the limitations "the unique model layer" and "the structural parameter of the unique model layer" in the claim because "structural parameters of the first number of unique model layers" is also recited in its based claim and it is unclear which "unique model layer" among "the first number of unique model layers" is referred here. Clarification is required.
Claim 11 recites the limitation "… a training loss value according to the model label value and the second output data …" in lines 7-8. There is insufficient antecedent basis for the limitation "the model label value" in the claim. For examination purpose, "… a training loss value according to the model label values and the second output data …" is considered according to Claim 12 (see also 112 Rejections to Claim 12 below).
Claim 12 is rejected for fully incorporating the deficiency of their respective base claims.
Claim 12 recites the limitation "… wherein determining the training loss value according to the model training data and the second output data comprises … determining, among the model label values, the model label value corresponding to the wireless access network device according to identifiers carried by the model training data … determining the training loss value by performing an operation on the second output data and the training label value … " in lines6. There is insufficient antecedent basis for the limitations "the model training data" and "the training label value" in the claim (see also 112 Rejections to Claim 11 above). For examination purpose, "… wherein determining the training loss value according to the model label values and the second output data comprises … determining, among the model label values, the model label value corresponding to the wireless access network device according to identifiers carried by the model label values … determining the training loss value by performing an operation on the second output data and the model label values … " is considered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13, 16, and 18-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Independent Claims 1 and 16
Step 1: Claim 1 is a process claim and Claim 16 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) recite(s) additional elements/limitations of ".
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitations/elements of "a plurality of wireless access network devices sending model subscription requests" and "sending, to the first number of wireless access network devices, structural parameters of the first number of unique model layers" are well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 2 and 18
Step 1: Claim 2 is a process claim and Claim 18 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 3 and 19
Step 1: Claim 3 is a process claim and Claim 19 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 4 and 20
Step 1: Claim 4 is a process claim and Claim 20 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitations/elements of "obtaining data of the plurality of radio access network devices", "obtaining first output data output by the shared model layers by inputting the model training data as first input data to the shared model layers", and "sending the model label values and the first output data to the plurality of wireless access network devices" are also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 5 and 21
Step 1: Claim 5 is a process claim and Claim 21 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "receiving training loss values from the plurality of wireless access network devices" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 6 and 22
Step 1: Claim 6 is a process claim and Claim 22 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 7 and 23
Step 1: Claim 7 is a process claim and Claim 23 is a device claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "after updating the shared model layers according to the update parameters: determining, in response to a Tth update of the structural parameters of the shared model layers, that training of the shared model layers is complete, wherein T is a predetermined number of times to update the shared model layers and the unique model layers", "the structural parameters of the shared model layers are configured for the wireless access network devices" (i.e., "configuring the structural parameters of the shared model layers for the wireless access network devices"), and "synthesize a model to which the wireless access network device subscribes" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "sending the model structural parameters of the shared model layers obtained after the Tth update, to each wireless access network device in the plurality of wireless access network device groups" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "sending the model structural parameters of the shared model layers obtained after the Tth update, to each wireless access network device in the plurality of wireless access network device groups" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 8
Step 1: Claim 8 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 9
Step 1: Claim 9 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "in response to (determining) that there is a newly-joined wireless access network device and the newly-joined wireless access network device satisfies a model training condition", and "re-determining the first number of model training structures in response to that there is an exiting wireless access network device" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "sending the structural parameter of the unique model layer corresponding to the newly-joined wireless access network device to the newly-joined wireless access network device" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Independent Claim 10
Step 1: Claim 10 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) recite(s) additional elements/limitations of .
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "receiving a structural parameter of a unique model layer sent by an OAM" is well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 11
Step 1: Claim 11 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitations/elements of "receiving model label values and first output data sent by the OAM", "obtaining second output data output by the unique model layer by using the first output data as input to the unique model layer and inputting the first output data to the unique model layer", and "sending the training loss value to the OAM" are also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 12
Step 1: Claim 12 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) ".
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "identifiers carried by the model training data" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 13
Step 1: Claim 13 is a process claim which falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "determining a structural parameter of a subscription model according to the structural parameter of the shared model layer and the structural parameter of the unique model layer after a Tth update, wherein T is a predetermined number of times to update the shared model layer and the unique model layer" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) .
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "receiving a structural parameter of a shared model layer sent by the OAM" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 8, 10-13, 16, and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over NORRMAN et al. (US 2022/0321423 A1, filed on 08/06/2020), hereinafter NORRMAN'423 in view of Yang et al. (US 2023/0038310 A1, priority date: 04/24/2020), hereinafter Yang.
Independent Claims 1 and 16
NORRMAN'423 discloses a model training method applied to an operation administration and maintenance (OAM) entity (NORRMAN'423, ABSTRACT and ¶¶ [0010] and [0012]: a first network entity in a communications network receives a request from a second network entity, the request comprising one or more selection criteria for selecting network entities to participate in a collaborative learning process to train a model using a machine learning algorithm; ¶¶ [0034]-[0042] with FIG.3: a collaborative (e.g. federated) learning process is used to train a model using machine-learning; the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network), comprising:
obtaining at least one wireless access network device group by grouping a plurality of wireless access network devices sending model subscription requests, the wireless access network device group comprising a first number of wireless access network devices (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities);
determining a (NORRMAN'423, ¶¶ [0035]-[0040] with FIG. 3: the NWDAF 308 initiates training of a model using machine-learning at each of the network functions, NF A 302, NF B 304 and NF C 306; e .g., the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; the message may comprise a copy of the model (e.g. a global copy that is common to each of the network functions 302-306), or each of the network functions 302-306 may be preconfigured with a copy of the model; in the latter case, the message may comprise an indicator of which model is to be trained; the message may specify a type of machine-learning algorithm to be used by the network entities; alternatively, the network entities 302-306 may be preconfigured with the type of machine learning algorithm to be used for a model; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; the training data may be data that is unique to the network entity; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the local model update may comprise updated values of the parameters of the model or the local model update may comprise an indication of a change in the values of the parameters of the model, e.g., differences between previous values for the parameters and updated values for the parameters; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network; ¶ [0059] with FIG. 3: after receiving the indication of the two or more candidate network entities from the NRF 310, the NWDAF 308 initiates, at the two or more candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; ¶ [0063]: queries relating to the configuration of a candidate network entity include queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; ¶ [0087]: the NWDAF 308 may send a trigger message to each of the selected candidate network entities instructing the candidate network entity to train the model using the machine-learning algorithm; the trigger message may, e.g., include information relating to the model (e.g. model parameters and/or identifying information for the model) and/or an indication of the machine-learning algorithm to be used).
NORRMAN'423 further discloses a model training device applied to an operation administration and maintenance (OAM) entity (NORRMAN'423, ABSTRACT and ¶¶ [0010] and [0012]: a first network entity in a communications network receives a request from a second network entity, the request comprising one or more selection criteria for selecting network entities to participate in a collaborative learning process to train a model using a machine learning algorithm; ¶¶ [0034]-[0042] with FIG.3: a collaborative (e.g. federated) learning process is used to train a model using machine-learning; the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network), comprising: a processor (NORRMAN'423, ¶ [0108] with 802 in FIG. 8: processing circuitry 802 (such as one or more processors, digital signal processors, general purpose processing units, etc.)); and a memory (NORRMAN'423, ¶ [a machine readable medium 804 (e.g., memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc.)) for storing instructions that are executable by the processor, wherein the instructions, when being executed by the processor, causes the processor to implement the operations shown above (NORRMAN'423, ¶ [0109] with FIG. 8: the machine-readable medium 804 stores instructions which, when executed by the processing circuitry 802, cause the apparatus 800 to perform select one or more of the plurality of candidate network entities to participate in a collaborative learning process to train a model using a machine learning algorithm).
NORRMAN'423 fails to explicitly disclose determining a first number of model training structures corresponding to the first number of wireless access network devices, and determining a first number of unique model layers according to the first number of model training structures; and sending, to the first number of wireless access network devices, structural parameters of the first number of unique model layers.
Yang teaches a system and a method relating to machine learning (Yang, ¶ [0002]), wherein determining a first number of model training structures corresponding to the first number of wireless access network devices, and determining a first number of unique model layers according to the first number of model training structures; and sending, to the first number of wireless access network devices, structural parameters of the first number of unique model layers (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
NORRMAN'423 and Yang are analogous art because they are from the same field of endeavor, a system and a method relating to machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Yang to NORRMAN'423. Motivation for doing so would improve local accuracy of a model of a neural network on each client device, while achieving generalization across client devices (Yang, ¶¶ [0005]-[0012]).
Claims 3 and 19
NORRMAN'423 in view of Yang discloses all the elements as stated in Claims 1 and 16 respectively and further discloses wherein determining the first number of unique model layers according to the first number of model training structures comprises: determining output layers of the first number of model training structures corresponding to the first number of wireless access network devices as the first number of unique model layers (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
Claims 4 and 20
NORRMAN'423 in view of Yang discloses all the elements as stated in Claims 1 and 16 respectively and further discloses determining input layers and hidden layers of the model training structures corresponding to the first number of radio access network devices as shared model layers, obtaining data of the plurality of radio access network devices, and adding, to the data, datum identifiers corresponding to respective radio access network devices; obtaining model training data and model label values by classifying and processing all the data with the datum identifiers; obtaining first output data output by the shared model layers by inputting the model training data as first input data to the shared model layers; and sending the model label values and the first output data to the plurality of wireless access network devices (NORRMAN'423, ¶ [0028] with FIG. 3: one or more of the network entities 302-306 may be implemented within entities outside the core network, such as radio access network nodes (e.g., base stations such as gNBs, eNBs etc. or parts thereof, such as central units or distributed units); ¶¶ [0047], [0054], and [0058] with FIGS. 3-4: in registering with the NRF 310, a network entity provides information relating to the services provided by the network entity (such as the type of network entity or the function performed thereby), and/or the capability of the network entity; such information may be stored by the NRF 310, and associated with an identifier (e.g., a unique number within the network) allowing the network entity to be identified and addressed); the NRF 310 may store a profile for each network entity that is registered with it; each profile may comprise information relating to the services provided by the network entity (such as the type of network entity or the function performed thereby), and/or the capability of the network entity; such information may be stored by the NRF 310, and associated with an identifier (e.g., a unique number within the network) or some other means allowing the network entity to be identified and addressed; the profile may comprise an indication of one or more services that the network entity is capable of providing; e.g., a profile may indicate the type of the network entity, e.g., that the network entity is a PCF, an ASF, or a dedicated machine-learning network function or entity; the identifier stored in the profile may correspond to the identification information and/or addressing information for the network entities) (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; the set of common layers 120 may be used to extract common features of the local dataset 211 and the set of client-specific layers 140 may be used to classify the extracted common features and generate an output corresponding to the local dataset 211; for classifying the extracted common features and generating an output corresponding to the local dataset 211, the client computing device 210 may be further configured to use a normalized exponential function (for instance, a softargmax or softmax function), in order to output label of the local dataset 211 with probabilities; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
Claims 5 and 21
NORRMAN'423 in view of Yang discloses all the elements as stated in Claims 4 and 20 respectively and further discloses updating, in response to receiving training loss values from the plurality of wireless access network devices, structural parameters of the shared model layers according to the training loss values (Yang, ¶¶ [0071]-[0077] and [0088]-[0090] with FIG. 2: the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; alternatively, the client computing device 210 may only send parameters of the updated set of common layers 120 that have been changed to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; these common features may also be exhibited on another dataset 211' of another client computing device 210', which can be seen on the lower right-hand side of FIG. 2; by sharing the updated set of common layers 120 with the server computing device 220, a global accuracy of the model 100 for performing the specific task of machine learning, such as identifying chat messages and video streaming clips in the above-mentioned example, can be improved across client computing devices 210, 210'; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; the repeating of the training may end when a mathematical condition or a criterion is fulfilled; the mathematical condition or the criterion may be a convergence of a gradient descent of the neural network).
Claim 8
NORRMAN'423 in view of Yang discloses all the elements as stated in Claim 1 and further discloses wherein obtaining the at least one wireless access network device group by grouping the plurality of wireless access network devices sending the model subscription requests comprises: determining a type of subscription model included in each of the model subscription requests; obtaining a first number of model subscription request groups by grouping the model subscription requests according to the type of the subscription model; and obtaining the first number of wireless access network device groups by grouping the wireless access network devices (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0035]-[0040] with FIG. 3: the NWDAF 308 initiates training of a model using machine-learning at each of the network functions, NF A 302, NF B 304 and NF C 306; e .g., the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; the message may comprise a copy of the model (e.g. a global copy that is common to each of the network functions 302-306), or each of the network functions 302-306 may be preconfigured with a copy of the model; in the latter case, the message may comprise an indicator of which model is to be trained; the message may specify a type of machine-learning algorithm to be used by the network entities; alternatively, the network entities 302-306 may be preconfigured with the type of machine learning algorithm to be used for a model; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; the training data may be data that is unique to the network entity; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the local model update may comprise updated values of the parameters of the model or the local model update may comprise an indication of a change in the values of the parameters of the model, e.g., differences between previous values for the parameters and updated values for the parameters; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the at least one primary criterion may specify one or more particular types of network entity; e.g., the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; after receiving the indication of the two or more candidate network entities from the NRF 310, the NWDAF 308 initiates, at the two or more candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities; the NWDAF 308 may send a trigger message to each of the selected candidate network entities instructing the candidate network entity to train the model using the machine-learning algorithm; the trigger message may, e.g., include information relating to the model (e.g. model parameters and/or identifying information for the model) and/or an indication of the machine-learning algorithm to be used).
Independent Claim 10
NORRMAN'423 discloses a model training method applied to a wireless access network device (NORRMAN'423, ABSTRACT and ¶¶ [0010] and [0012]: a first network entity in a communications network receives a request from a second network entity, the request comprising one or more selection criteria for selecting network entities to participate in a collaborative learning process to train a model using a machine learning algorithm; ¶¶ [0034]-[0042] with FIG.3: a collaborative (e.g. federated) learning process is used to train a model using machine-learning; the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network), comprising:
receiving a structural parameter of a (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0035]-[0040] with FIG. 3: the NWDAF 308 initiates training of a model using machine-learning at each of the network functions, NF A 302, NF B 304 and NF C 306; e .g., the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; the message may comprise a copy of the model (e.g. a global copy that is common to each of the network functions 302-306), or each of the network functions 302-306 may be preconfigured with a copy of the model; in the latter case, the message may comprise an indicator of which model is to be trained; the message may specify a type of machine-learning algorithm to be used by the network entities; alternatively, the network entities 302-306 may be preconfigured with the type of machine learning algorithm to be used for a model; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; the training data may be data that is unique to the network entity; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the local model update may comprise updated values of the parameters of the model or the local model update may comprise an indication of a change in the values of the parameters of the model, e.g., differences between previous values for the parameters and updated values for the parameters; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; after receiving the indication of the two or more candidate network entities from the NRF 310, the NWDAF 308 initiates, at the two or more candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities; the NWDAF 308 may send a trigger message to each of the selected candidate network entities instructing the candidate network entity to train the model using the machine-learning algorithm; the trigger message may, e.g., include information relating to the model (e.g. model parameters and/or identifying information for the model) and/or an indication of the machine-learning algorithm to be used).
NORRMAN'423 fails to explicitly disclose receiving a structural parameter of a unique model layer, wherein the unique model layer is determined by dividing a first number of model training structures, and the first number of model training structures are determined.
Yang teaches a system and a method relating to machine learning (Yang, ¶ [0002]), wherein receiving a structural parameter of a unique model layer, wherein the unique model layer is determined by dividing a first number of model training structures, and the first number of model training structures are determined (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
NORRMAN'423 and Yang are analogous art because they are from the same field of endeavor, a system and a method relating to machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Yang to NORRMAN'423. Motivation for doing so would improve local accuracy of a model of a neural network on each client device, while achieving generalization across client devices (.
Claim 11
NORRMAN'423 in view of Yang discloses all the elements as stated in Claim 10 and further discloses receiving model label values and first output data sent by the OAM; obtaining second output data output by the unique model layer by using the first output data as input to the unique model layer and inputting the first output data to the unique model layer; and determining a training loss value according to the model label value and the second output data, and sending the training loss value to the OAM (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; the repeating of the training may end when a mathematical condition or a criterion is fulfilled; the mathematical condition or the criterion may be a convergence of a gradient descent of the neural network; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected) (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0035]-[0040] with FIG. 3: the NWDAF 308 initiates training of a model using machine-learning at each of the network functions, NF A 302, NF B 304 and NF C 306; e .g., the NWDAF 308 may transmit a message to each of the network functions 302-306 instructing the network function to train a model using machine-learning; the message may comprise a copy of the model (e.g. a global copy that is common to each of the network functions 302-306), or each of the network functions 302-306 may be preconfigured with a copy of the model; in the latter case, the message may comprise an indicator of which model is to be trained; the message may specify a type of machine-learning algorithm to be used by the network entities; alternatively, the network entities 302-306 may be preconfigured with the type of machine learning algorithm to be used for a model; on receipt of the message from the NWDAF 308, each network entity 302-306 trains the model by inputting training data into the machine-learning algorithm to obtain a local model update to values of one or more parameters of the model; the training data may be data that is unique to the network entity; each of the network entities 302-306 transmits the local model update to the NWDAF 308; the local model update may comprise updated values of the parameters of the model or the local model update may comprise an indication of a change in the values of the parameters of the model, e.g., differences between previous values for the parameters and updated values for the parameters; the transmission between the network functions 302-306 and the NWDAF 308 may be via an Operation, Administration and Management function (OAM) 312; the NWDAF 308 combines the model updates received from the network entities 302-306 to obtain a combined model update; the NWDAF 308 transmits the combined model update to one or more network entities in the network; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; after receiving the indication of the two or more candidate network entities from the NRF 310, the NWDAF 308 initiates, at the two or more candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities; the NWDAF 308 may send a trigger message to each of the selected candidate network entities instructing the candidate network entity to train the model using the machine-learning algorithm; the trigger message may, e.g., include information relating to the model (e.g. model parameters and/or identifying information for the model) and/or an indication of the machine-learning algorithm to be used).
Claim 12
NORRMAN'423 in view of Yang discloses all the elements as stated in Claim 11 and further discloses wherein determining the training loss value according to the model training data and the second output data comprises: determining, among the model label values, the model label value corresponding to the wireless access network device according to identifiers carried by the model training data; and determining the training loss value by performing an operation on the second output data and the training label value, and updating the structural parameter of the unique model layer according to the training loss value (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; the repeating of the training may end when a mathematical condition or a criterion is fulfilled; the mathematical condition or the criterion may be a convergence of a gradient descent of the neural network; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
Claim 13
NORRMAN'423 in view of Yang discloses all the elements as stated in Claim 10 and further discloses receiving a structural parameter of a shared model layer sent by the OAM; and determining a structural parameter of a subscription model according to the structural parameter of the shared model layer and the structural parameter of the unique model layer after a Tth update, wherein T is a predetermined number of times to update the shared model layer and the unique model layer (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; the repeating of the training may end when a mathematical condition or a criterion is fulfilled; the mathematical condition or the criterion may be a convergence of a gradient descent of the neural network; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
Claims 2 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over NORRMAN'423 in view of Yang as applied to Claims 1 and 16 above, and further in view of Vitaladevuni et al. (US 10,354,184 B1, date of patent: 07/16/2019), hereinafter Vitaladevuni.
Claims 2 and 18
NORRMAN'423 in view of Yang discloses all the elements as stated in Claims 1 and 16 respectively and further discloses wherein determining the first number of model training structures corresponding to the first number of wireless access network devices comprises: determining a first number of model subscription requests sent by the first number of wireless access network devices, and determining model training (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities) (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
NORRMAN'423 in view of Yang fails to explicitly disclose determining model training task characteristics of the first number of model subscription requests, the model training task characteristic being configured to indicate a number of layers and a number of nodes of a model; and determining the first number of model training structures according to the number of layers and the number of nodes of the model indicated by the model training task characteristics.
Vitaladevuni teaches a system and a method relating to a neural network-based joint model (ABSTRACT), wherein determining model training task characteristics of the first number of model subscription requests, the model training task characteristic being configured to indicate a number of layers and a number of nodes of a model; and determining the first number of model training structures according to the number of layers and the number of nodes of the model indicated by the model training task characteristics (Vitaladevuni, Col. 2, lines 12-14: utilizing a unified neural network to jointly model related tasks and/or applications; Col. 4, line 7 – Col. 5, line 10: a neural network may be generated and trained for one or more core tasks and/or applications such as product recommendation and targeted advertising; the neural network can consist of a base neural network portion and a task-specific neural network portion; the base neural network portion can include the input layer and one or more hidden layers; these layers may be low-level features of the system; the low-level features of the system may be common across multiple tasks and/or applications as they are task-independent, e.g., the lower levels can correspond to common ways of representing customers, common ways of representing products, etc.; the neural network can be trained using a shared base neural network portion common to all tasks and/or applications; the neural network can branch out for the task-specific portion, comprising at least the output layer; the task-specific portion can include hidden layers as well, but may consist of just the output layer; these higher level layers are specific to a given task and/or application, e.g., product recommendations versus targeted advertising; the trained base neural network portion can be shared across all tasks and/or applications; the base neural network portion can be embedded for various input vectors; the output of the base neural network portion can be record for each input vector; this output can be used as an input to task-specific neural networks; the initial neural network can be represented by a base neural network portion and a task-specific neural network portion; the task-specific neural network portion can be replaced by a different task-specific neural network portion corresponding to a different task; the task-specific neural network portion can vary in size; for some tasks, only the output layer of the initial neural network is replaced; for other tasks, the output layer plus one or more hidden layers may be replaced with a new task-specific portion; Col. 8, line 15 – Col. 10, line 67 with FIG. 2: block 204 involves obtaining user and task and/or application data; Block 206 involves initializing and training the joint model 100 based on said user and task and/or application data; the joint model 100 can have data regarding three specific tasks, A, B, and C; the joint model 100 can also have data regarding three specific users, 1, 2, 10 and 3; the NN-based joint model 100 can contain a base neural network portion and a task-specific neural network portion; the base neural network portion can include the input layer and one or more hidden layers; the task-specific neural network portion can include the output layer; the task-specific neural network portion can include one or more hidden layers in addition to the output layer; the task-specific neural network portion can vary in size depending on the task; block 208 involves determining whether the neural network should be trained for additional tasks; if the joint model 100 does not need to be trained with additional tasks, the process can move to block 214; there, the system can determine whether to update any task-specific portions of the neural network; if the joint model 100 determines that it should update any task-specific portions of the neural network with additional data, this update can occur at block 216; this update can involve training the task-specific portions of the joint model 100 using additional data gathered from user-system interactions, additional information about one or more users, additional users, additional products, etc.; the base portion of the joint model 100 may not be updated; the entire neural network 100, including both the base portion and the task-specific portion, can be updated with additional information; if the process instead involves a determination that the joint model should be trained for additional tasks, the task-specific portion of the neural network is replaced with a different task-specific neural network corresponding to a task on which the joint model 100 has not been trained at block 210; the different task-specific neural network can include an output layer; the different task-specific neural network can include one or more hidden layers in addition to the output layer; the portion of the neural network that is replaced with a different task-specific neural network portion can vary depending on the task; e.g. if the joint model 100 has been trained for product recommendation and is now being trained for price sensitivity, the task-specific portion that is replaced may include the output layer and no hidden layers; however, if the joint model 100 has been trained for product recommendation and is now being trained for targeted advertising, the task-specific portion that is being replaced may include the output layer plus two hidden layers; the base neural network portion can vary in the number of hidden layers, but is used across all tasks and/or applications; once the task-specific portion of the neural network has been replaced with a different task-specific neural network, the different task-specific neural network can be trained; this training, performed at block 212, can be in addition to the initial neural network training performed at block 206; the task-specific portion of the neural network can be trained using the obtained data regarding users, products, and tasks and/or applications; when training the task-specific portion of the neural network, the output from the joint model 100 can be compared to some known output; the weights in the task-specific portion of the neural network can then be adjusted if necessary; this adjustment process can be repeated until the task-specific portion of the neural network is trained for the corresponding task and/or applications; e.g., the newly trained joint model 100 can now be used to predict what products a customer would purchase; the original joint model 100 may instead have been trained to predict which user interface a user would prefer; by replacing the task-specific portion of the neural network with a different task-specific neural network portion, the joint model 100 can be trained as needed for a specific task, such as automatic speech recognition; the process then circles back to block 208, where the system determines whether the joint model 100 should be trained on additional tasks and/or applications; the process ends at block 218; Col. 11, line 3 – Col. 13, line 40 with FIGS. 3-4: an illustrative neural network-based joint model 100 containing several non-linear hidden layers 308, a base neural network portion 310, and one or more task-specific neural network portions 312. Input features 300 can be fed into the joint model 100; these features 300 can relate to different users or products; each layer of the joint model 100 can contain a set of nodes 304; each node 304 is a computational unit interconnected to other computational units within the joint model 100; each set of nodes 304 can be associated with a respective task and/or application (e.g., recommending products to a user, determining how much the price of a product must drop before the specific user will purchase the product, what user interface design is most likely to appeal to the specific user, etc.); the NN-based joint model 100 can include a base neural network portion 310 and a task-specific neural network 15 portion 312; the base neural network portion 310 can include the input layer and one or more hidden layers; the task-specific neural network portion 312 can include the output layer; the task-specific neural network portion 312 may contain one or more hidden layers; the base neural network 310 contains those lower layers of the neural network that are common across multiple tasks and applications, e.g., are task and/or application-independent; e.g., the low-level layers making up the base neural network portion 310 can relate to identifying and clustering different user characteristics; the high-level layers 312 can relate to specific tasks and/or applications; the joint model 30 100 may be trained for at least one task and/or application; the task-specific portion 312 of the neural network can be replaced by a different task specific-portion 312; this different task-specific portion can correspond to a different task; as a result, the joint model 100 can easily be adjusted for specific tasks and/or applications without affecting the base neural network portion 310 shared across all tasks and/or applications; the number of layers in the task-specific portion 312 can vary from task to task; the task-specific portion 312 have one or more additional inputs 318 separate from the base neural network portion 310; the additional inputs 318 may be specific to a given task and/or application; the system may take the output from the base neural network 310; the system can then input this output into a task-specific portion 312 of the neural network 100).
NORRMAN'423 in view of Yang, and Vitaladevuni are analogous art because they are from the same field of endeavor, a system and a method relating to a neural network-based joint model. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Vitaladevuni to NORRMAN'423 in view of Yang. Motivation for doing so would improve accuracy regarding the joint model's predictions (Vitaladevuni, Col. 2, line 62 - Col. 3, line 19) .
Claims 6-7 and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over NORRMAN'423 in view of Yang as applied to Claims 5 and 21 respectively above, and further in view of Abbas et al., ("Adaptively Weighted Multi-task Learning Using Inverse Validation Loss", 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),May 12-17, 2019, pp. 1408-1412), hereinafter Abbas.
Claims 6 and 22
NORRMAN'423 in view of Yang discloses all the elements as stated in Claims 5 and 21 respectively and further discloses wherein updating the shared model layers according to the training loss values comprises: obtaining (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; the repeating of the training may end when a mathematical condition or a criterion is fulfilled; the mathematical condition or the criterion may be a convergence of a gradient descent of the neural network; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
NORRMAN'423 in view of Yang fails to explicitly disclose obtaining weighted loss values by weighting the training loss values; determining current model parameters and current model learning rates of the shared model layers; and determining update parameters of the shared model layers according to the weighted loss values, the model parameters and the model learning rates.
Abbas teaches a system and a method relating to machine learning obtaining weighted loss values by weighting the training loss values; determining current model parameters and current model learning rates of the shared model layers; and determining update parameters of the shared model layers according to the weighted loss values, the model parameters and the model learning rates (Abbas, ABSTRACT in Page 1408: Multi-task learning aims to enhance the performance of a model by inductive transfer of information among tasks; however, joint optimization of multiple tasks is challenging due to unbalanced data ranges and variations in task difficulties which can cause the model to converge only for a single task which has large values; to address these problems, we propose a novel weighting scheme based on validation loss; Section I in Page 1408: like humans, intelligent machines or models can also learn multi-tasks by training on multiple tasks simultaneously and it is frequently used in several applications, including natural language processing [2], computer vision [3] and speech signal processing [4]; however, when model trained in such a manner to perform multitasks, easy tasks can dominate learning by ignoring harder tasks; this posed an important question: what is the right balance between learning of multiple tasks from easy versus hard tasks? To address the problem a diverse domain, multi-tasks learning can be categorized into two approaches: i) task weighting or ii) sharing information between the linked tasks; in task weighting scheme, the objective function is obtained by assigning weights, automatically or via external human supervision, to each task according to their difficulty level and task weights [5]; information sharing schemes among the tasks share common features but maintain separate task-specific modules; this can be categorized into hard parameter sharing, soft parameter sharing and task-hierarchy; a critical assumption in multi-tasks learning is that the underlying distribution across all the tasks remains the same; however, this assumption is broken when defining a multi-task problem over dynamic tasks; parameter sharing can lead to a critical layer responsible for learning representations of discriminative downstream tasks; further, the selection of the representation layer is difficult for a dynamic multi-task problem; propose a solution for task weighting scheme to predict multiple outputs present in a scenario; propose a multi-task architecture with soft weight sharing technique by observing shared information among the tasks; our scheme is encouraged to prioritize harder tasks over easier tasks; Section 2 with FIG. 1 in Pages 1409-1410: propose an adaptive weighting scheme in addition to a CNN based deep architecture containing shared layers as well as task independent layers for Multi-Task Learning (MTL); the proposed scheme assign more weights to the harder tasks and use previous validation loss to determine relative task difficulties; to evaluate the proposed model and task weighting scheme, we apply MTL to autonomous driving and show that the task weighting scheme influence the performance of an individual task by prioritizing the tasks which require more training iterations to be learned; in order to learn all parameters, apart from minimizing the combined loss, we propose objective function as
L
D
=
m
i
n
∑
j
ω
θ
j
L
θ
j
x
,
ϱ
, where
ω
θ
j
is the weighting factor that defines the contribution of the jth individual task and
L
θ
j
is the individual loss function for the jth task; we used Mean Squared Error (MSE) as Loss function/Objective function and it can be formulated as:
L
D
=
m
i
n
∑
j
ω
θ
j
1
β
∑
i
=
1
β
θ
i
j
-
θ
^
i
j
2
; assign adaptive weighting factors to individual task by obtaining the initial weights from the first configuration; Let
L
θ
j
,
v
is the final validation loss obtained from the first configuration; using these validation losses, weighting factors for objective function are computed as
ω
θ
j
=
1
L
θ
j
,
v
; this configuration assigns an equal loss to each individual task and minimization of combined loss tries to converge the model for each equally; this scheme introduces task specific weighting. Minimization of combined loss tries to converge the model for tasks which were not better learned during Scheme 1; employ commonly used Stochastic Gradient Descent (SGD) method to obtain the network parameters, W and B, of the proposed model; updated learning parameters can be computed as equations (7)-(11); updating rule for model parameters (W;B) using the SGD can be defined as equations (12)-(13); where η is the learning rate; Adam optimizer is used for adaptive learning rate decay with early stopping criterion selected empirically; propose an adaptive learning rate algorithm; define a learning rate decay mechanism as:
η
t
+
1
=
k
×
η
t
, where
η
t
and
η
t
+
1
are the current epoch t and the next epoch t+1 learning rates, and k is the learning rate decay factor; MTL in deep learning architectures is performed by introducing layers that are common to all the tasks; we show that such inductive transfer can improve the model when the multiple tasks trained under a unified objective function with a specified task weighting scheme; we input the data to the multiple separate but identical branches; each task-specific branch consists of three convolutional blocks followed by a flattening layer and final regression layer as shown in Fig. 1; by sharing initial layers of the model across the tasks, training examples force the parameters to be generalized in a manner such that model more compelling towards good weights yielding a better generalization of the learned parameters).
NORRMAN'423 in view of Yang, and Abbas are analogous art because they are from the same field of endeavor, a system and a method relating to machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Abbas to NORRMAN'423 in view of Yang. Motivation for doing so would prevent the model to converge only for a single task which has large values.
Claims 7 and 23
NORRMAN'423 in view of Yang and Abbas discloses all the elements as stated in Claims 6 and 22 respectively and further discloses after updating the shared model layers according to the update parameters: determining, in response to a Tth update of the structural parameters of the shared model layers, that training of the shared model layers is complete, and sending the model structural parameters of the shared model layers obtained after the Tth update, to each wireless access network device in the plurality of wireless access network device groups, wherein T is a predetermined number of times to update the shared model layers and the unique model layers, and the structural parameters of the shared model layers are configured for the wireless access network devices to synthesize a model to which the wireless access network device subscribes (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over NORRMAN'423 in view of Yang as applied to Claim 1 above, and further in view of ICKIN et al. (US 2024/0119305 A1, priority date: 02/15/2021), hereinafter ICKIN.
Claim 9
NORRMAN'423 in view of Yang discloses all the elements as stated in Claim 1 and further discloses in response to that there is a or (NORRMAN'423, ¶¶ [0028]-[0032] with FIG. 3: each of the network entities 302-306 is registered at a network registration entity 310 which may be any suitable network entity that provides registration and discovery for network entity services; a Network Data and Analytics Function (NWDAF) 308 is configured to collect network data from one or more network entities, and to provide network data analytics information to network entities which request or subscribe to receive it; ¶¶ [0043]-[0045]: a co-ordination network entity in a communications network transmits a request message to a network registration entity in the communications network, for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the co-ordination entity then receives identification information for a plurality of candidate network entities from the network registration entity and initiates, at one or more network entities of the plurality of candidate network entities, training of a model using a machine-learning algorithm as part of a collaborative learning process; by sending queries for candidate network entities to determine which of the candidate network entities satisfy one or more selection criteria; ¶¶ [0046]-[0088] with FIGS. 3-4: the signaling shown in FIG. 4 permits the co-ordination network entity (hereinafter, the NWDAF 308) to select one or more network entities to participate in a collaborative learning process such as federated learning; the procedure begins with the NWDAF 308 transmitting, to the NRF 310, a first request message 400 for identification information for a plurality of candidate network entities in the communications network capable of performing collaborative learning; the first request message 400 may be sent directly from the NWDAF 308 to the NRF 310 (as illustrated) or indirectly via one or more intermediate entities in the communication network; the first request message 400 thus comprises a request that the NRF 310 provide a list of network entities which are capable of performing collaborative learning (such as federated learning); the first request message 400 may comprise one or more first selection criteria for selecting candidate network entities for performing a collaborative learning process; the one or more first selection criteria may comprise at least one primary criterion relating to a capability of the candidate network entity to perform the collaborative learning process; the first request message 400 may indicate that each of the candidate network entities must be an Access and Mobility management Function (AMF), or a dedicated machine learning network function; the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing a particular service; e.g., the first request message 400 may specify that each network entity selected as a candidate network entity is capable of providing subscriber authentication data, or a particular type of collaborative learning; the one or more first selection criteria in the first request message 400 may comprise at least one secondary criterion relating to a capability of the candidate network entity to support a type of query; the NRF 310 identifies, from a plurality of network entities registered at the network registration entity, two or more candidate network entities capable of performing collaborative learning; the NRF 310 transmits, in a first response message 402, an indication of the two or more candidate network entities to the NWDAF 308; the NWDAF 308 transmits a second request message 404 comprising at least one query for additional information for each candidate network entity in the two or more candidate network entities; the at least one query may comprise one or more second selection criteria for selecting candidate network entities; queries relating to one or more of the following: software versions, licenses, neighbor relations, one or more configuration parameters, and a type and/or quantity of hardware and/or software at the candidate network entity; queries relating to a performance requirement may include, e.g., queries relating to a performance measurement (PM) procedure, a key performance indicator (KPI) and/or any other suitable performance metric for the candidate network entity; the at least one query for a candidate network entity may relate to an availability of training data at the candidate network entity; the at least one query for a candidate network entity may relate to the performance of the candidate network entity in training a model using machine-learning; a candidate network entity may be configured to, in response to receiving a query of this type, obtain one or more metrics indicative of a performance of a preliminary model developed using the machine-learning algorithm; the NWDAF 308 transmits the second request message 404 comprising the at least one query for additional information to the OAM 312; the NWDAF 308 may thus transmit a single second request message 404 to the OAM 312 for all of the candidate network entities; alternatively, the NWDAF 308 may transmit two or more second request messages to the OAM 312 (e.g. one second request message per candidate network entity); the OAM 312 receives the second request message 404 comprising the at least one query from the NWDAF 308; the OAM 312 stores the required information to answer the at least one query for each of the candidate network devices (e.g. in a cache); e.g., if the at least one query specifies that the candidate devices must have a network traffic load that is less than 50%, then the OAM 312 may determine which candidate network devices satisfy this requirement and send an indication of which candidate network devices satisfy this requirement to the NWDAF 308; alternatively, if the OAM 312 does not store the information required to respond to the at least one query for all of the candidate network devices, the OAM 312 may generate a third request message 406 based on the at least one query and transmit the third request message 406 to at least one of the two or more candidate network entities; each of the candidate network entities receiving the third request message 406 from the OAM 312 may send, to the OAM 312, a second response message 408 comprising the additional information requested in the at least one query; the OAM 312 thus receives a second response message 408 from one or more of the candidate network entities; the OAM 312 sends a third response message 410 to the NWDAF 308 based on the second response message 408; if the at least one query comprises one or more second selection criteria, then the OAM 312 may transmit a third response message 410 to the NWDAF 308, in which the third response message 410 comprises an indication of which of the two or more candidate network entities satisfy the one or more second selection criteria; the OAM 312 may send an indication for only a subset of the candidate network entities meeting the one or more second selection criteria; the OAM 312 may select a subset of the candidate network entities based on a characteristic of the candidate network entities; the OAM 312 may rank the candidate network entities according to the particular characteristic and select the best (e.g. highest) ranking candidate network entities; the OAM 312 selects a subset of the candidate network entities based on the one or more second response messages 408 received from the candidate network entities) (Yang, ¶¶ [0068]-[0069], [0071]-[0091] and [0093]-[0101] with FIGS. 1-2: the model 100 may comprise an input layer 121, an output layer 143, and a set of intermediate layers 122, 123, 141, 142; these layers may be connected, one by one, wherein the output of one layer may be the input of the next layer; treat the model 100 as having two separate parts: a set of common layers 120 and a set of client-specific layers 140; a server computing device 220 (see FIG. 2) may provide each of one or more client computing devices 210 (see FIG. 2) with the model 100; each of the one or more client computing devices 210 may, after training the model 100, share only the updated common layers 120 back to the server computing device 220 (see FIG. 2), and may store its updated client-specific layers 140 locally after the training; as such, the client-specific layers 140 may be kept independently across the different client computing devices 210, i.e., the client-specific layers 140 may not be shared by the client computing devices 210, and any updates relating to the client-specific layers 140 may not be sent to the server computing device 220; this is beneficial, since a richer feature extractor may be possible for each client computing device 210 by sharing the common layers 120, while each client computing device 210 keeps its client-specific layers 140 adapted to unique features of its local dataset; the client computing device 210 is configured to obtain a model 100 of a neural network, e.g., the model 100 shown in FIG. 1, from the server computing device 220, wherein the model 100 comprises the set of common layers 120 and the set of client-specific layers 140; each layer 120, 140 of the model 100 may further comprise parameters, e.g., learnable weights and/or biases, to be adjusted/trained for performing a specific task of machine learning; the client computing device 210 is configured to train the model 100 to obtain an updated set of common layers 120 and an updated set of client-specific layers 140; thereby, parameters of each layer of the model 100 may be adjusted based on the local dataset 211 of the client computing device 210, for instance, by using a training algorithm commonly known in the field of machine learning, such as backpropagation; after the training of the model 100, the client computing device 210 is configured to send the updated set of common layers 120 to the server computing device 220; the updated set of common layers 120 may be adjusted according to common features of the local dataset 211; the client computing device 210 is configured to store the updated set of client-specific layers 140; the updated set of client-specific layers 140 may be adjusted according to unique features, which are rarely exhibited on other datasets 211' of other client computing devices 210'; i.e., the updated set of client-specific layers 140 may not be sent to the server computing device 220 and may not be shared with other client computing devices 210'; the set of common layers 120 may be stacked prior to the set of client-specific layers 140; the set of client-specific layers 140 comprises less parameters than the set of common layers 120; as such, the set of client-specific layers 140 may require less data for the training than the set of common layers 120; the set of common layers 120 may comprise information for feature extraction, and the set of client-specific layers 140 may comprise information for classification; different client computing devices 210, 210' located in distinct environments can still cooperate to improve the model 100 of the neural network by sharing the set of common layers 120, and to achieve a richer feature extractor of the model 100; moreover, the set of client-specific layers 140 may be stored and updated locally by each client computing device 210,210', wherein these layers 140 may advantageously be adapted to unique features of each respective local dataset 211, 211' for classification; after sending the updated set of common layers 120 to the server computing device 220, the client computing device 210 may be further configured to receive an aggregated set of common layers 120 from the server computing device 220; then the client computing device 210 may update the model 100 based on the received aggregated set of common layers 120; in particular, the client computing device 210 may concatenate the received aggregated set of common layers 120 and the updated set of client-specific layers 140 to obtain an updated model 100; after obtaining the updated model 100, the client computing device 210 may be configured to train the updated model 100 again by using the local dataset 211 and/or another local dataset (e.g., from another client computing device 210') to obtain a further updated set common layers 120 and a further updated set of client-specific layers 140; then the client computing device 210 may send the further updated set of common layers 120 to the server computing device 220 and may store the further updated set of client-specific layers 140; the training may be repeated to achieve a final model 100, which is fit for performing the specific task of machine learning; ¶¶ [0105]-[0108]: the virtual separation of the model 100 of the neural network - here it is exemplarily a CNN network-into a set of common layers 120 and a set of client-specific layers 140; the way of separating the model 100 may be performed according to the CNN's property; here the set of common layers 120 is referred to as "Backbone", e.g., stacked convolutional layers, and the set of client-specific layers 140 is referred to as last layers (LL), e.g., last fully connected layers; in particular, the CNN may be a common classification network using stacked convolutional layers at the beginning, followed by fully connected layers; the LL may also be referred to as "LL Classifier", since it/they is/are the classifier that contains class specific information; the Backbone may be interpreted as feature extraction, particularly it may contain the common feature extraction procedure among the client computing devices 210; each client computing device 210 may share its updated Backbone (after training of the model 100 based on the local dataset 211) to the server computing device 220; sharing the Backbones helps to learn a richer feature extractor; the Backbones may be aggregated in the server computing device 220; each client computing device 210 may further keep (a) specific LL layer(s) ("LL Classifier A", "LL Classifier B" . . . "LL Classifier N") to further adapt to a local data distribution; the updated LL Classifier is not shared back to the server computing device 220 after training of the model; after receiving an update of the server computing device 220, each client computing device 210 may replace the local Backbone (stored at the respective client computing device 201) with a received aggregated Backbone; thereby, the LL classifier does not participate in the aggregation performed by the server computing device 220, and may thus be kept independent between the client computing devices 210; ¶¶ [0109]-[0115] with FIG. 4: perform a heterogeneous data-adaptive federated learning algorithm; Step 1, the client computing devices 210 may update the local model 100 by copying the Backbone; if it is the first round of communication, the LL (Classifier) may be copied as well; in Step 2, the client computing devices 210 may update the received model 100 on their local dataset 211, until convergence or by fixing epochs; in Step 3, one or more of the client computing devices 210, or each client computing device 210, may send back the Backbone to the server computing device 220; upon receiving the Backbones from the client computing devices 210, in Step 4, the server computing device 220 aggregates the Backbones; in Step 5, the server computing device 220 may then broadcast the aggregated Backbone to the client computing devices 210; ¶¶ [0117]-[0121] and [0123]-[0134] with FIGS. 5-6: S501: obtaining, by a client computing device, a model from a server computing device, wherein the model comprises a set of common layers and a set of client-specific layers; S502: training, by the client computing device, the model based on a local dataset to obtain an updated set of common layers and an updated set of client-specific layers, wherein the local dataset is stored at the client computing device; S503: sending, by the client computing device, the updated set of common layers to the server computing device, and S504: storing, by the client computing device, the updated set of client-specific layers; S601: aggregating, by the server computing device, the received updated sets of common layers to obtain an aggregated set of common layers; S602: sending, by the server computing device, the aggregated set of common layers to each of the client computing devices; S603: updating, by the client computing device, the model based on the aggregated set of common layers; the steps of S502, S503, S504, S601, S602, and S603 may be repeated multiple times, until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing the specific task of machine learning; instead of constructing a single global Full Model (FM) 100 for N client computing devices 210, N models 100, namely one at each of the N client computing devices 210, may be constructed; each model 100 has the same set of common layers 120 and an individual set of client-specific layers 140; in particular, the set of common layers 120 (e.g., Backbone portion) may be globally shared by the server computing device 220, whereas the set of client-specific layers 140 (e.g., N×LL portions) may be specialized for each client computing device 210 and may remain locally at the client computing devices 210, 210'; during the training process, the server computing device 220 can ensure/infer that the client computing devices 210 have a set of common layers 120 (e.g., Backbone portion) for their model 100 and a set of client-specific layers 140 (e.g., LL parts) for their model 100; notably, the split between common layers 120 and client-specific layers 140 does not need to be the LL only; training a model 100 of a neural network, in particular common layers 120 like a CNN backbone, usually requires a large amount of data, and not every client computing device may have enough data; sharing the set of common layers 120 allows every client computing device 210 to benefit from the large amount of data (datasets 211, 211') collected from all of the client computing devices 210; the client-specific layers 140, e.g., LL Classifier, have typically much less parameters, so that the local dataset 211 at each client computing device 210 is enough for training; the local accuracy is further optimized to ensure a best performance for imbalanced distributed data at the various client computing devices 210; the client-specific layers 140 (e.g., LL Classifier) allow the model 100 to adapt quickly to local client computing device's distribution, despite the imbalanced data distribution existing between client computing devices 210; the set of common layers 120 (e.g., Backbone) can be seen as a common feature extraction process; although multi modal signals may exist in a local client computing device 210, independent client-specific layers 140 (e.g., LL Classifier) can select corresponding features for different signals; the client-specific layers 140 (e.g., LL Classifier) is not used for the aggregation, hence, even if labels are disjoint, the convergence will not be affected).
NORRMAN'423 in view of Yang fails to explicitly disclose in response to that there is a newly-joined wireless access network device and the newly-joined wireless access network device satisfies a model training condition, sending the structural parameter of the unique model layer corresponding to the newly-joined wireless access network device to the newly-joined wireless access network device; or re-determining the first number of model training structures in response to that there is an exiting wireless access network device.
ICKIN teaches a system and a method relating to distributed machine learning (ICKIN, ¶ [0001]), wherein in response to that there is a newly-joined wireless access network device and the newly-joined wireless access network device satisfies a model training condition, sending the structural parameter of the unique model layer corresponding to the newly-joined wireless access network device to the newly-joined wireless access network device; or re-determining the first number of model training structures in response to that there is an exiting wireless access network device (ICKIN, ¶¶ [0009]-[0017] with FIG. 2: a single NN may be split between worker nodes 120A, 120B and a master node 110, with the worker nodes hosting one set of layers 210, 212 of the NN and the master node hosting another set of layers 214, 216 of the NN; as shown in FIG. 2, different workers may collect data related to different features of the network; in addition to the input layer 210, each worker node may host zero or more other layers of the NN model, including intermediate layers 224 and an output layer, or cut-layer, that faces the master node 110; the master node 110 hosts the remaining layers 214, 216 of the NN, including a cut-layer 214 that faces the workers 120A, 120B; ¶¶ [0019]-[0021]: determining whether layer outputs for the cut-layer were not received from the one of the workers from which layer outputs were not received for more than a threshold number of sample intervals, and in response to determining that layer outputs for the cut-layer were not received from the one of the workers from which layer outputs were not received for more than the threshold number of sample intervals, reshaping the cut-layer of the split neural network to exclude neurons associated with the one of the workers from which layer outputs were not received; determining a new training batch size, cut layer, and/or neuron count for the split neural network based on the re-shaped cut-layer; ¶¶ [0140]-[0141] with FIGS. 7-8: there may be cases in which communication is lost for a longer time interval (e.g., more than a predetermined threshold); in that case, the cut-layer may be re-shaped; note that cut-layer re-shaping can be performed simultaneously with exponential smoothing of inputs; the cut-layer is reshaped after N=3 consecutive failures to receive input from one of the workers (in this case, worker 120C, the SGW); when the cut-layer is reshaped, the missing worker 120C is removed from the pool of workers contributing to the federation; FIG. 8 illustrates the pool of workers before re-shaping (with the SGW) and after reshaping (without the SGW)).
NORRMAN'423 in view of Yang, and ICKIN are analogous art because they are from the same field of endeavor, a system and a method relating to distributed machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of ICKIN to NORRMAN'423 in view of Yang. Motivation for doing so would prevent the NN to continually rely on the missing inputs.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
NORRMAN et al. (US 2022/0292398 A1, filed on 08/06/2020) discloses in ¶¶ [0028]-[0046] with FIG. 3 that (1) the system 300 comprises an aggregating entity 302, a plurality of network entities or network functions (NFs)-labelled NF A 304, NF B 306, NFC 308 and NF D 310-an operations, administration and maintenance node (OAM) 312 and a NF repository function (NRF) 314; (2) the network entities 304-310 are configured to provide one or more services; (3) each of the network entities 304-310 is able to communicate with the NWDAF 302, and such communication may be direct or indirect via one or more intermediate network nodes; (4) a collaborative (e.g. federated) learning process is used to train a model using machine-learning; (5) rather than collating training data for training the model at a single network entity, instances of the model are trained locally at multiple network functions to obtain local updates to parameters of the model at each network entity; (6) the local model updates are collated at the aggregating entity ( such as the NWDAF) 302 and combined to obtain a combined model update; (7) the NWDAF 302 may transmit a message to each of the network functions 304-310 instructing the network function to train a model using machine-learning; (8) the message may comprise an initial copy or version of the model ( e.g. a global copy that initially is common to each of the network functions 304-310), or each of the network functions 304-310 may be preconfigured with a copy of the model; (9) in the latter case, the message may comprise an indicator of which model is to be trained; (10) the message may specify a type of machine-learning algorithm to be used by the network entities; (11) each trained version is for the same model, and the model has the same architecture or structure, uses the same input data and has the same purpose; (12) however, the precise values for parameters (e.g., weights) of the model will differ between trained versions from different network entities; (13) each of the network entities 304-310 transmits its respective local model update to the NWDAF 302; (14) the local model update may comprise updated values of the parameters of the model or the local model update may comprise an indication of a change in the values of the parameters of the model, e.g., differences between initial values for the parameters and updated values for the parameters after training; (15) transmissions between the network entities 304-310 and the NWDAF 302 may be direct (e.g. the NWDAF 308 transmits directly to a network entity) or the transmissions may be via an intermediate network entity; e.g., the transmission between the network functions 304-310 and the NWDAF 302 may be via an Operation, Administration and Management function (OAM) 312; (16) the NWDAF 302 combines the model updates received from the network entities 304-310 to obtain a combined model update; (17) the NWDAF 302 transmits the combined model update to one or more network entities in the network; (18) the combined model update may be transmitted to one or more further network entities in addition to the network entities 304-310 used to train the model; (19) this process may be repeated one or more times; e.g., the process may be repeated until the local model updates received from each of the network entities 304-310 are consistent with each other to within a predetermined degree of tolerance; (20) the model updates transmitted between the aggregator entity (e.g., the NWDAF 302) and the plurality of network entities 304-310 are subjected to a serialization function which removes all information relating to a structure or architecture of the model; (21) in this way, while an intercepting entity may be able to gain knowledge as to the values of parameters of the model, it is unable to gain any practical insight into the nature of the model; and (22) further, the aggregator entity 302 itself may not have knowledge of the model structure, which further increases security by reducing the opportunities for information relating to the model leaking into the public domain (e.g., from the aggregator entity), and also reduces the complexity of processing necessary to combine the multiple local model updates.
KOURDIS (WO 2022/069059 A1, filed on 10/02/2020) discloses in Page 6, line 29 – Page 11, line 34 that (1) use multi-task learning to train a model for a target task, wherein the target task is one of the available tasks that is treated as the task to optimize performance for, and all other tasks are treated as auxiliary, meaning that their contributions to model updates are only of interest in so far as they improve the performance of the target task; (2) the use of arbitrary auxiliary tasks during training can therefore boost target task performance; (3) given a set of classification tasks T = {t1, t2, … ,tn}, each associated with training and validation data, a target task t* [Symbol font/0xCE] T, and a set of task-specific weights W = {w1, w2 , … , wn}, parameters θ of a deep neural network are trained on the data of all tasks in T while minimizing the objective loss of the model on the target task's development set as shown in eqn. (1), where pθ(xi*) =
y
^
i
*
is the model's prediction for an instance of the target task data; (4) the cross-entropy loss for model training is used on all tasks in the set T of tasks, and the task-specific weights are used to augment the standard loss function as shown in eqn. (2), where yt is the one-hot vector encoding the correct label of an instance of task t [Symbol font/0xCE] T, yt is the classifier prediction for that instance, • denotes the inner product of true label and prediction, and wt is the current weight associated with the task; (5) while the w weights in Equation (2) can take any, preferably positive, value, it should be clear that in cases where
w
t
=
1
,
∀
t
∈
T
, this loss is the same as for standard un-weighted multitask training, and as for single task training where
w
t
=
1
∧
w
t
'
=
0
,
∀
t
'
≠
t
,
wherever
L
(
y
,
y
^
)
is used, this means that loss that is unweighted; (6) provide a way of estimating the influence of tasks on the target's performance, and of adjusting weights according during model training; (7) the method replaces the traditional estimation of task weights with a method based on the model updates, as expressed by the difference of the model's parameters before and after one or more, preferably multiple, optimization steps (or an optimization procedure), henceforth referred to as the model "deltas"; (8) instead of estimating iteratively scaling weights based on heuristic cycles of weight application, target task evaluation, and weight re-scaling, the method estimates the optimal mixing ratio of task-specific model deltas with respect to performance of the target task with a meta-optimizer, and uses the mixing ratio to scale the deltas during training; (9) based on the determined optimal mixing ratio of deltas, the task-specific weights are then updated; (10) while the weights of tasks can be limited to be positive, the mixing ratio of any given task may be negative and instead allow the weights to become arbitrarily large or small, including negative; (11) additional parameters, herein referred to as a variables, are introduced to the model training process, which are used to scale the individual tasks' model updates by finding an optimal mixing ratio with respect to a performance measure on the target task; (12) these variables are associated with the actually realized task-specific model updates; (13) these a variables can either be "global", applying exactly one mixing ratio per task to all layers of the model, or "local", where one variable per task per model layer is used; (14) one implementation of the training method can use a pre-trained RoBERTa model as an input encoder 101, having encoder layers 1-n, as illustrated at 102-104, and task-specific classification heads 1-t (i.e. each tasks has its own classification head), as illustrated at 105-107, comprising simple feed-forward layers on top; (15) in Figure 2, the encoder block 201 comprises a self-attention mechanism 202, a normalization layer 203, a feed-forward layer 204, 205 and a second normalization layer 206; (16) the training of the model comprises two main phases, carried out in alternation, as schematically illustrated in Figures 3(a) and 3(b); (17) Figure 3(a) schematically illustrates phase 1 of the training: the model training phase, wherein (a) any number of tasks are sampled from the plurality of available tasks (i.e. the target task and at least one auxiliary task); (b) each task of the one or more sampled tasks is used to update the underlying model on its own using the task's current weight; (c) after this, the differences in model parameters (deltas) are collected; and (d) the model is then reset; (18) in phase 2, as illustrated in Figure 3(b), showing the a variable update phase, the previously collected deltas from phase 1 are used to find the optimal mixing ratio with respect to model performance on the target task; (19) therefore, in phase 2, a machine learning operation is performed, by means of one or more predetermined evaluation criteria and based on one or more predetermined constraints, to assess the performance of the refined set of model parameters of the one or more of the plurality of tasks (that underwent optimization steps in phase 1) on the target task and thereby determine a set of mixing weights; (20) the set of model parameters of the machine learning model are then updated in accordance with the refined set of model parameters as weighted by the set of mixing weights (a variables) and the candidate scaling factors are updated in dependence on the set of mixing weights; (21) training then returns to phase 1 (Figure 3(a)), and the cycle continues for a pre-defined number of iterations over the training data, or until model performance on the target task converges; (22) the optimal set of model parameters are then selected and used as the final model parameters for inference after training is complete; (23) in summary, it is desirable to make use of auxiliary updates if and when they help push the target task's performance, but ignore them if they do not; (24) during training, the approach first performs weighted task-specific model updates on a proportion of the available training data, starting from the current model parameters for each individual task; (25) it collects the resulting model deltas, i.e., the differences between the model's parameters before and after the single task update, and resets the model; (26) after this delta collection phase, a variables are used as additional parameters which are optimized via gradient descent to find a good interpolation of the individual tasks' model updates, with respect to the loss on the target task's development data; (27) the best-found interpolation of deltas is used to update the model parameters, and the task-specific weights are updated using the new interpolation parameters; (28) an exemplary algorithm for training the model shown in Figure 4 takes model M with parameters θ, a set of tasks T = {t1, t2, … , tn}, a target task t*, training data Dttrain for each task t, development data Dt*dev for the target task, maximum number of
training epochs ε, ratio ρ of tasks' training data to sample in inner loop, and number of α tuning steps s; and (29) the result is an updated set of model parameters optimized for performance on the target task t*.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
Prosecution Insights