Detailed Action
Claims 1-22 are pending in this application.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Singapore on 10/30/23. It is noted, however, that applicant has not filed a certified copy of the 10202303060U application as required by 37 CFR 1.55.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
As per claim 21 this application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
a first module configured to
a second module configured to
a communication interface configured to
a training unit configured to
an aggregation unit configured to
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 21 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim limitation “a first module configured to; a second module configured to; a communication interface configured to; a training unit configured to; an aggregation unit configured to;” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the algorithm and corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21,22 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per claim 21, recites claim limitation “a first module configured to; a second module configured to; a communication interface configured to; a training unit configured to; an aggregation unit configured to;” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
As per claim 22, has been evaluated under the three-prong test set forth in MPEP § 2181, subsection I, but the result is inconclusive. Thus, it is unclear whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim recites
implementing a first model via a first module to process
implementing a second model via a second module to process
aggregating parameters of the second model via an aggregation unit to form
the term “means” or generic placeholder is modified by a word(implementing a first model via, implementing a second model via, aggregating parameters of the second model via), which is ambiguous regarding whether it conveys structure or function.
The boundaries of this claim limitation are ambiguous; therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
In response to this rejection, applicant must clarify whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Mere assertion regarding applicant’s intent to invoke or not invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph is insufficient. Applicant may:
(a) Amend the claim to clearly invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by reciting “means” or a generic placeholder for means, or by reciting “step.” The “means,” generic placeholder, or “step” must be modified by functional language, and must not be modified by sufficient structure, material, or acts for performing the claimed function;
(b) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, should apply because the claim limitation recites a function to be performed and does not recite sufficient structure, material, or acts to perform that function;
(c) Amend the claim to clearly avoid invoking 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by deleting the function or by reciting sufficient structure, material or acts to perform the recited function; or
(d) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, does not apply because the limitation does not recite a function or does recite a function along with sufficient structure, material or acts to perform that function.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recites
1. A method comprising:
implementing a first model to process a first set of data corresponding to a first data space, wherein the first model comprises a first feature extractor configured to extract a first set of feature representations;
implementing a second model to process a second set of data corresponding to a second data space, wherein the second model comprises a second feature extractor configured to extract a second set of feature representations;
transferring information from the second model to the first model via a connection, wherein the connection links the first model and the second model;
training the second model using a labeled set derived from the second set of data;
training the first model using a labeled set derived from the first set of data and a plurality of outputs from the second feature extractor; and
aggregating parameters of the second model to form a global model.
2. The method of claim 1, wherein the first model further comprises a first predictor configured to make predictions based on the extracted first set of feature representations, the first model being parameterized by parameters corresponding to the first feature extractor and the first predictor; and wherein the second model further comprises a second predictor configured to make predictions based on the extracted second set of feature representations, the second model being parameterized by parameters corresponding to the second feature extractor and the second predictor.
3. The method of claim 1, wherein the first model and second model are linked to form a cascading structure, the cascading structure being configured such that outputs of the second model serve as inputs to the first model.
4. The method of claim 1, wherein the method is implemented in a distributed learning framework comprising a plurality of data management entities, wherein each data management entity possesses its own local data and operates its own cascading structure comprising the first model and the second model, wherein each cascading structure is further configured to perform local supervised learning using the local data of the respective data management entity for implementing the cascading structure.
5. The method of claim 1, wherein the connection linking the first model and the second model is defined by a plurality of connection parameters and facilitates the linking of a plurality of layers corresponding to the first feature extractor to a plurality of layers corresponding to the second feature extractor, wherein the information transferred to the first model comprises features or representations learned by the second model.
6. The method of claim 1, wherein each layer of a plurality of layers of the first feature extractor is configured to receive input from both an output of a preceding layer of the plurality of layers of the first feature extractor and a corresponding layer of a plurality of layers of the second feature extractor, thereby facilitating in the transfer of information from the second model to the first model.
7. The method of claim 2, wherein the training of the second model using the labeled set of the second data further comprises minimizing a first optimization function, the first optimization function measuring a discrepancy between a prediction generated from the second predictor and a ground truth label.
8. The method of claim 7, wherein the training of the first model further comprises fixing parameters of the second model and minimizing a second optimization function, the second optimization function measuring a discrepancy between a prediction generated from the first predictor and a ground truth label, and first model parameters are updated to minimize the second optimization function.
9. The method of claim 1, wherein the aggregating of the parameters of the second model to form the global model further comprises: receiving the parameters of the second model from a plurality of parties, computing a weighted average of the received parameters wherein a size of a local training batch for the second model is determined, and forming the global model using the computing weighted average of the parameters.
10. The method of claim 1, wherein a learning using publicly available data is employed for implementing a learning process on the global model, said learning module employing a consistency regularization technique to regulate a consistency of loss, thereby improving the learning process.
11. The method of claim 10, wherein the learning comprises a mask generator, a pretext generator, and a self-supervised learner, wherein the self-supervised learner is configured to train the global model using the consistency regularization technique.
12. The method of claim 11, wherein the mask generator generates a plurality of binary mask vectors for the publicly available data, and the pretext generator uses the binary mask vectors to generate masked samples of the publicly available data.
13. The method of claim 11, wherein the self-supervised learner trains the global model by minimizing a consistency loss function, the consistency loss function defined as a discrepancy between a set of feature representations derived from the publicly available data and a set of feature representations derived from the masked data samples of the publicly available data.
14. The method of claim 12, wherein each binary mask vector from the plurality of binary mask vectors is generated by sampling from a Bernoulli distribution defined by a probability value with each binary mask vector having a length corresponding to a feature dimension of the second data space.
15. The method of claim 13, wherein the consistency loss function is structured to encourage the global model to maintain a consistent output distribution in response to perturbations in the publicly available data caused by the pretext generator.
16. The method of claim 11, wherein for a fixed data point from the publicly available data, a loss function is computed based on a stochastic approximation which comprises evaluating a batch of the fixed data point and a plurality of generated variants for each fixed data point in the batch.
17. The method of claim 1, wherein a Vertical Federated Learning initialize its respective private and public feature extractors and parameters from the first and second feature extractors of the first and second models respectively are provided as input for the initialization of the private and public feature extractors.
18. The method of claim 17, wherein the initialization facilitates the training of the Vertical Federated Learning on overlapping samples shared among multiple data management entities, each entity possessing its own local data and operating within a distributed learning framework.
19. The method of claim 17, wherein the Vertical Federated Learning comprises a general predictor configured to receive feature representations outputted from the private and public feature extractors, and to generate a prediction based on said feature representations.
20. The method of claim 1, wherein the first set of data corresponding to the first data space comprises personal or sensitive data and the second set of data corresponding to the second data space comprises publicly accessible or publicly available data.
21. A machine learning system, comprising:
a first model to process a first set of data corresponding to a first data space, wherein the first model comprises a first feature extractor configured to extract a first set of feature representations;
a second model to process a second set of data corresponding to a second data space, wherein the second model comprises a second feature extractor configured to extract a second set of feature representations;
facilitate the transfer of information from the second model to the first model via a connection, wherein the connection links the first and second model;
train the second model using a labeled set derived from the second set of data and train the first model using a labeled set derived from the first set of data and a plurality of outputs from the second feature extractor; and
aggregate parameters of the second model to form a global model.
22. implementing a first model to process a first set of data corresponding to a first data space, wherein the first model comprises a first feature extractor configured to extract a first set of feature representations;
implementing a second model to process a second set of data corresponding to a second data space, wherein the second model comprises a second feature extractor configured to extract a second set of feature representations;
transferring information from the second model to the first model via a connection, wherein the connection links the first model and the second model;
training the second model with a labeled set derived from the second set of data;
training the first model using the training unit with a labeled set derived from the first set of data and a plurality of outputs from the second feature extractor; and
aggregating parameters of the second model to form a global model.
The claims are directed towards training of models using output data from different models. The specification pg.11-16,18 describes multiple mathematical concepts such as relationships, formulas/equations, and calculations that are used for processing data and training of the models.
Therefore the claims and the specification is drawn to mathematical concepts and/or mental processes(the steps of implementing a first and second model, transferring information between models, training model, and aggregation can be performed mentally and/or with the aid of pen and paper). If the claim under broadest reasonable interpretation covers limitation that is drawn to mathematical concepts and mental processes but for recitation of a generic computer and/or generic computer components described at a high level of generality or linking the use of the judicial exception to a particular technological environment or field of use, then it falls within the grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, prong 1).
This judicial exception is not integrated into a practical application. In particular, the claims recites additional elements such as
1. A machine learning
9. at a server
10,11,17,18,19,. Module or module is deployed
21. A machine learning system, comprising: a first module configured to; a second module configured to; a communication interface configured to; a training unit configured; and an aggregation unit configured.
22. A non-transitory computer-readable medium having instructions stored thereon, which when executed by a computer, cause the computer to perform operations comprising: via a first module; via a second module; through a communication interface; using the training unit; and via an aggregation unit.
The claim does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the limitations of 1. A machine learning 9. at a server 10,11,17,18,19, module or module is deployed, 21. A machine learning system, comprising: a first module configured to; a second module configured to; a communication interface configured to; a training unit configured; and an aggregation unit configured to. 22. A non-transitory computer-readable medium having instructions stored thereon, which when executed by a computer, cause the computer to perform operations comprising: via a first module; via a second module; through a communication interface; using the training unit; and via an aggregation unit.
are generic computer components described at a high level of generality and limitations amounts to mere instructions to implement the abstract idea on a computer and/or adding the words “apply it”(or an equivalent) with the judicial exception, or merely uses a computer as a tool to perform an abstract idea. MPEP 2106.05(f).
Therefore the additional limitation/elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Even considering all the additional element in combination, they are just providing a computerized system to perform the invention, but doesn’t improve the computing technology as the additional elements do not integrate the invention into a practical application, rather the claims and the additional limitations are drawn to training models. The claims is directed to an abstract idea and merely reciting generic computer components described at a high level of generality and limitations amounts to mere instructions to implement the abstract idea on a computer and/or adding the words “apply it”(or an equivalent) with the judicial exception, or merely uses a computer as a tool to perform an abstract idea MPEP 2106.05(f) . Therefore the claims are not patent eligible. (Step 2A, prong2).
The claim does not include additional elements/limitations that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements/limitations is drawn to limitations that use a computer as a tool, and includes well-understood, routine, and conventional activities(ie. receiving or transmitting data over a network MPEP 2106.05(d)(II)) that amount to no more than implementing the abstract idea with a computerized system. The claim is not patent eligible(Step 2B).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 17-20 rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0073678 issued to Chu et al.(Chu) in view of US 2025/0037032 issued to Tang et al.(Tang) in view of US 2024/0320507 issued to Chalkidis et al.(Chalkidis).
As per claims 1,21, 22, Chu teaches A machine learning method/ A machine learning system/ A non-transitory computer-readable medium having instructions stored thereon, which when executed by a computer, cause the computer to perform operations(Fig.10) comprising: a first module configured to implement a first model to process a first set of data corresponding to a first data space, wherein the first model comprises a first feature extractor configured to extract a first set of feature representations(Fig.4,6, [0041] As used herein, the term “data sample” refers to stored data that refers to a particular entity, such as a person, object, or other identifiable entity having characteristics which can be observed and recorded as data. Each data sample can be identified using a corresponding sample ID. A data sample set refers to a plurality of different data samples. A data sample can have multiple data features (or simply “features”) which describe different attributes of the corresponding entity. For example, different data features may relate to different financial, social, or behavioural data for a person. If the data sample is represented as a vector, the data features might be represented as components of the vector. In various embodiments, for training purposes, it is not necessary to sample from a data sample set. Rather, it may be assumed that a data sample set is sampled from a latent distribution of all data. [0047] In some embodiments, each private machine learning model can be viewed as a feature extractor. The central machine learning model then concatenates the features (e.g. feature maps) provided by these private machine learning models and feeds the concatenated feature maps into its own model. The term “feature” in this paragraph may be distinguished from the term as it is used elsewhere herein. [0118] FIG. 6 illustrates the SLVF system for training a machine learning model for a particular task using a supervised learning algorithm according to another embodiment of the present invention. Similarly to FIG. 2, the illustrated SLVF system 600 is shown during a training phase in which both private machine learning models and a centralized machine learning model are trained. In the embodiment of FIG. 2, the task owner 116 owns and stores the labels related to task T, and the central machine learning model 204 communicates with the computing system 116 of the task owner to compute the loss. In contrast, in the embodiment of the SVFL system 600 shown in FIG. 6, the computing system 116 of the task owner shares certain information with the central server 205 via a communication 604. This information can include the labels 602 and the definition of the loss function related to the task T. Upon receipt of the information 604, the central server 205 can compute the loss independently from the computing system 116 of the task owner. The above embodiment mitigates the need for the central server to communicate with the computing system 116 of the task owner. The information 604 can be communicated to the central server 205 prior to the learning operations being performed, for example using one or more secure messages. Because the computing system 116 of the task owner shares the labels and loss function definition with the central server 205, the central server 205 can compute the loss independently. This reduces communication overhead between the central server 205 and the computing system 116 of the task owner.);
a second module configured to implement a second model to process a second set of data corresponding to a second data space, wherein the second model comprises a second feature extractor configured to extract a second set of feature representations(Fig.4,6, [0041] As used herein, the term “data sample” refers to stored data that refers to a particular entity, such as a person, object, or other identifiable entity having characteristics which can be observed and recorded as data. Each data sample can be identified using a corresponding sample ID. A data sample set refers to a plurality of different data samples. A data sample can have multiple data features (or simply “features”) which describe different attributes of the corresponding entity. For example, different data features may relate to different financial, social, or behavioural data for a person. If the data sample is represented as a vector, the data features might be represented as components of the vector. In various embodiments, for training purposes, it is not necessary to sample from a data sample set. Rather, it may be assumed that a data sample set is sampled from a latent distribution of all data. [0047] In some embodiments, each private machine learning model can be viewed as a feature extractor. The central machine learning model then concatenates the features (e.g. feature maps) provided by these private machine learning models and feeds the concatenated feature maps into its own model. The term “feature” in this paragraph may be distinguished from the term as it is used elsewhere herein. [0118] FIG. 6 illustrates the SLVF system for training a machine learning model for a particular task using a supervised learning algorithm according to another embodiment of the present invention. Similarly to FIG. 2, the illustrated SLVF system 600 is shown during a training phase in which both private machine learning models and a centralized machine learning model are trained. In the embodiment of FIG. 2, the task owner 116 owns and stores the labels related to task T, and the central machine learning model 204 communicates with the computing system 116 of the task owner to compute the loss. In contrast, in the embodiment of the SVFL system 600 shown in FIG. 6, the computing system 116 of the task owner shares certain information with the central server 205 via a communication 604. This information can include the labels 602 and the definition of the loss function related to the task T. Upon receipt of the information 604, the central server 205 can compute the loss independently from the computing system 116 of the task owner. The above embodiment mitigates the need for the central server to communicate with the computing system 116 of the task owner. The information 604 can be communicated to the central server 205 prior to the learning operations being performed, for example using one or more secure messages. Because the computing system 116 of the task owner shares the labels and loss function definition with the central server 205, the central server 205 can compute the loss independently. This reduces communication overhead between the central server 205 and the computing system 116 of the task owner.);
a training unit configured to train the second model using a labeled set derived from the second set of data and train the first model using a labeled set derived from the first set of data(Fig.4,6, [0040] Embodiments of the invention provide Secure Vertical Federated Learning (SVFL) methods and systems to securely train private machine learning models for a supervised machine learning task. The training is performed using the private data owned and stored by multiple data owners. Data owners operate computing systems which interact with a central machine learning model, which is operated on a central server. The training also uses a central machine learning model for the supervised machine learning task, without infringing the data privacy and model privacy of any of these data owners. Model privacy refers to privacy of the structure and parameters of used by a private data owner to represent data for use. The use of data may include, for example, use of the data to generate a private machine learning model, to generate inferences or predictions using the private machine learning model based on observations (i.e., new data samples) input to the model, or a combination thereof. [0118] FIG. 6 illustrates the SLVF system for training a machine learning model for a particular task using a supervised learning algorithm according to another embodiment of the present invention. Similarly to FIG. 2, the illustrated SLVF system 600 is shown during a training phase in which both private machine learning models and a centralized machine learning model are trained. In the embodiment of FIG. 2, the task owner 116 owns and stores the labels related to task T, and the central machine learning model 204 communicates with the computing system 116 of the task owner to compute the loss. In contrast, in the embodiment of the SVFL system 600 shown in FIG. 6, the computing system 116 of the task owner shares certain information with the central server 205 via a communication 604. This information can include the labels 602 and the definition of the loss function related to the task T. Upon receipt of the information 604, the central server 205 can compute the loss independently from the computing system 116 of the task owner. The above embodiment mitigates the need for the central server to communicate with the computing system 116 of the task owner. The information 604 can be communicated to the central server 205 prior to the learning operations being performed, for example using one or more secure messages. Because the computing system 116 of the task owner shares the labels and loss function definition with the central server 205, the central server 205 can compute the loss independently. This reduces communication overhead between the central server 205 and the computing system 116 of the task owner.);
Chu however does not explicitly teach a communication interface configured to facilitate the transfer of information from the second model to the first model via a connection, wherein the connection links the first and second model; train the first model using a plurality of outputs from the second feature extractor; an aggregation unit configured to aggregate parameters of the second model to form a global model.
Tang explicitly teaches a communication interface configured to facilitate the transfer of information from the second model to the first model via a connection, wherein the connection links the first and second model([0038] According to a second broad aspect of the present disclosure, there is provided herein another method for configuring AI/ML in a wireless communication network. The method according to the second broad aspect of the present disclosure may include configuring, by a network device in a wireless communication network, a first global AI/ML model such that the first global AI/ML model includes a first global AI/ML sub-model and a second global AI/ML sub-model, the first global AI/ML sub-model and the second global AI/ML sub-model being cascaded such that an output of the first global AI/ML sub-model is an input of the second global AI/ML sub-model. For example, one of the first global AI/ML sub-model and the second global AI/ML sub-model may be a global common AI/ML sub-model that is common to a group of devices in the wireless communication network, and the other one of the first global AI/ML sub-model and the second global AI/ML sub-model may be a first global cluster-specific AI/ML sub-model that is common to a first cluster of one or more devices within the group of devices.; teaches cascaded AI/ML therefore it is obvious to one ordinary skill in the art that the two model are connected); train the first model using a plurality of outputs from the second feature extractor([0190] The present disclosure describes examples of cluster-based AI/ML model training procedures that avoid or at least mitigate one or more of the foregoing problems with conventional AI/ML model training procedures. For example, as discussed in further detail below, in some embodiments described herein different techniques are used to configure UEs and a network device to collaboratively participate in a multi-stage AI/ML model training process utilizing cascaded AI/ML models. For example, a first aspect of the present disclosure provides a personalized cascaded AI/ML model comprising a common AI/ML sub-model and a cluster AI/ML sub-model that are cascaded such that an output of the common AI/ML sub-model is an input of the cluster AI/ML sub-model or vice versa. The common AI/ML sub-model is common to all devices participating in the AI/ML model training process and each cluster of devices within that group has its own cluster-specific cluster AI/ML sub-model. A second aspect of the present disclosure provides a multi-stage or multi-mode AI/ML model training process for training personalized cascaded AI/ML models….).
Therefore it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify Chu’s teachings of vertical federated learning to apply the teachings of Tang of cascaded AI/ML in order to provide the predictable result of having cascaded AI/ML in vertical federated learning.
One ordinary skill in the art would have been motivated to combine the teachings in order to optimize performance, speed, and cost and to preserve privacy, confidentiality, and security of data(Chu, para.6).
Chu in view of Tang does not explicitly teach an aggregation unit configured to aggregate parameters of the second model to form a global model.
Chalkidis explicitly teaches an aggregation unit configured to aggregate parameters of the second model to form a global model(Fig.5,9, [0139] The data selection unit 950 receives the public dataset 920, and uses the set of trained private machine learning models 937 to process the public dataset 920, thereby attaching a group of labels and weights to the public dataset 920. The aggregation unit 960 aggregates the group of labels and weights as a processed public dataset 970. In embodiments, the processed public dataset 970 may be selected by adjusting filtering thresholds for the attached weights. As one example, the processed public data 970 may be determined by assigning the public dataset 920 to separate partitions according to weight thresholds (for example, thresholds determined based on the group of labels and weights), training a plurality of public machine learning models, and selecting the optimal threshold and model parameters according to an evaluation metric. [0140] The machine learning unit 980 uses the processed public dataset 970 to train a public machine learning model 985. This process may be controlled by the control unit 975, which establishes a feedback loop from the public machine learning model 985 to the partitioning optimization unit 925 and the data selection unit 950 to optimize the transfer capacity of the weighted knowledge transfer. The public machine learning model 985 may be used to provide a variety of machine-learning based services to users 995 over the interface unit 990..).
Therefore it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify Chu in view of Tang of cascaded AI/ML in vertical federated learning to apply the teachings of Chalkidis of aggregations and selection of data for machine learning models in order to provide the predictable result of having cascaded AI/ML in vertical federated learning for training of a machine learning model.
One ordinary skill in the art would have been motivated to combine the teachings in order to optimize performance, speed, and cost and to preserve privacy, confidentiality, and security of data(Chu, para.6, Chalkidis, para.1).
As per claim 2, Chu in view of Tang in view of Chalkidis teaches teaches the method of claim 1, wherein the first model further comprises a first predictor configured to make predictions based on the extracted first set of feature representations, the first model being parameterized by parameters corresponding to the first feature extractor and the first predictor; and wherein the second model further comprises a second predictor configured to make predictions based on the extracted second set of feature representations, the second model being parameterized by parameters corresponding to the second feature extractor and the second predictor(Chu, [0040] …. The use of data may include, for example, use of the data to generate a private machine learning model, to generate inferences or predictions using the private machine learning model based on observations (i.e., new data samples) input to the model, or a combination thereof. [0049] For example, labels may refer to a value of some prediction being made by the machine learning models. In some cases, some data may be labelled a priori, for example in the case of supervised machine learning. This may be the case for training data in the case of supervised machine learning. The prediction output by a machine learning model can be compared to such labels, and the error between the two can be quantified as a loss which is fed back to the machine learning model. The label may be interpreted as the ground truth prediction result of a data sample. Labels may be stored locally, in the cloud, or both. [0060] As will be readily understood by a worker skilled in the art, a loss function is a function (comparable to a cost function, error function or objective function) that receives a machine learning model's output (e.g. a prediction) and computes a loss which is the output of the loss function computed based on this received output. In supervised learning, each training data includes an input (e.g. training data sample) and an output (the label, which may be a ground truth or expected output). The loss reflects the difference between the machine learning's output (e.g. prediction) and the label. The loss is used in backpropagation to update model parameters of the machine learning model. Various loss functions can be used, such as those that determine mean squared error, hinge loss, cross entropy loss, etc. The appropriate loss function used in training a machine learning model is dependent on the particular task the machine learning model is being trained for).
As per claim 3, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein the first model and second model are linked to form a cascading structure, the cascading structure being configured such that outputs of the second model serve as inputs to the first model(Tang, [0190] … in a multi-stage AI/ML model training process utilizing cascaded AI/ML models. For example, a first aspect of the present disclosure provides a personalized cascaded AI/ML model comprising a common AI/ML sub-model and a cluster AI/ML sub-model that are cascaded such that an output of the common AI/ML sub-model is an input of the cluster AI/ML sub-model or vice versa. The common AI/ML sub-model is common to all devices participating in the AI/ML model training process and each cluster of devices within that group has its own cluster-specific cluster AI/ML sub-model. A second aspect of the present disclosure provides a multi-stage or multi-mode AI/ML model training process for training personalized cascaded AI/ML models…..). Motivation to combine set forth in claim 1.
As per claim 4, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein the method is implemented in a distributed learning framework comprising a plurality of data management entities, wherein each data management entity possesses its own local data and operates its own cascading structure comprising the first model and the second model, wherein each cascading structure is further configured to perform local supervised learning using the local data of the respective data management entity for implementing the cascading structure(Chu, Fig.2, [0099] Each of the computing systems 118-1, . . . , 118-k of the other k−1 data owners stores a private data set 224-1, . . . , 224-k, respectively. The private data sets 224-1, . . . , 224-k owned and stored by the k−1 data owners are vertically partitioned (see FIG. 1). Each data owner privately owns and stores their own trained private machine learning model 222-1, . . . , 222-k. Each private machine learning model can fully access the private data of its data owner. All data owners use their own private data and their own trained private machine learning model to collaborate with the trained central machine learning model, in order to achieve a prediction for the task T. [0118] FIG. 6 illustrates the SLVF system for training a machine learning model for a particular task using a supervised learning algorithm according to another embodiment of the present invention. Similarly to FIG. 2, the illustrated SLVF system 600 is shown during a training phase in which both private machine learning models… Tang, [0192] As noted above, one aspect of the present disclosure provides a personalized cascaded AI/ML model comprising a common AI/ML sub-model and a cluster AI/ML sub-model that are cascaded such that an output of the common AI/ML sub-model is an input of the cluster AI/ML sub-model or vice versa. The features of the common sub-model are shared and trained by all devices participating in the AI/ML model training process. In contrast, the features of the cluster AI/ML sub-model are shared and trained by the devices in a cluster corresponding to that cluster AI/ML sub-model. In other words, each cluster of devices within the total group of devices participating in the AI/ML model training process has its own cluster-specific cluster AI/ML sub-model. The common AI/ML sub-model and the cluster AI/ML sub-model may be implemented as separate AI/ML models that are cascaded or as one multi-layer AI/ML model (e.g., a deep neural network). For example, FIG. 8A illustrates an example of an AI/ML model 800 in accordance with one embodiment of the present disclosure in which a common AI/ML sub-model 1802 and a cluster AI/ML sub-model 2 804 are implemented as separate AI/ML models that are cascaded such that an output of the common AI/ML sub-model 1802 is an input of the cluster AI/ML sub-model 2 804. On the other hand, FIG. 8B illustrates an example of the AI/ML model 800 in accordance with another embodiment of the present disclosure in which the common AI/ML sub-model 1 802 and the cluster AI/ML sub-model 2 804 are implemented as one multi-layer AI/ML model 810 having N total layers in which the first M layers of the multi-layer AI/ML model 810 implement the common AI/ML sub-model 1 802 and the remaining N-M layers implement the cluster-specific cluster AI/ML sub-model 2 804.). Motivation to combine set forth in claim 1.
As per claim 5, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein the connection linking the first model and the second model is defined by a plurality of connection parameters and facilitates the linking of a plurality of layers corresponding to the first feature extractor to a plurality of layers corresponding to the second feature extractor, wherein the information transferred to the first model comprises features or representations learned by the second model(Tang, Fig.6A,B [0171] In some embodiments, an AI model encompasses neural networks, which are used in machine learning. A neural network is composed of a plurality of computational units (which may also be referred to as neurons), which are arranged in one or more layers. The process of receiving an input at an input layer and generating an output at an output layer may be referred to as forward propagation. In forward propagation, each layer receives an input (which may have any suitable data format, such as vector, matrix, or multidimensional array) and performs computations to generate an output (which may have different dimensions than the input). The computations performed by a layer typically involves applying (e.g., multiplying) the input by a set of weights (also referred to as coefficients). With the exception of the first layer of the neural network (i.e., the input layer), the input to each layer is the output of a previous layer. A neural network may include one or more layers between the first layer (i.e., input layer) and the last layer (i.e., output layer), which may be referred to as inner layers or hidden layers. For example, FIG. 6A depicts an example of a neural network 600 that includes an input layer, an output layer and two hidden layers. In this example, it can be seen that the output of each of the three neurons in the input layer of the neural network 600 is included in the input vector to each of the three neurons in the first hidden layer. Similarly, the output of each of the three neurons of the first hidden layer is included in an input vector to each of the three neurons in the second hidden layer and the output of each of the three neurons of the second hidden layer is included in an input vector to each of the two neurons in the output layer. As noted above, the fundamental computation unit in a neural network is the neuron, as shown at 650 in FIG. 6A. FIG. 6B illustrates an example of a neuron 650 that may be used as a building block for the neural network 600. As shown in FIG. 6B, in this example the neuron 650 takes a vector x as an input and performs a dot-product with an associated vector of weights w. The final output z of the neuron is the result of an activation function ƒ( ) on the dot product. Various neural networks may be designed with various architectures (e.g., various numbers of layers, with various functions being performed by each layer. ) Motivation to combine set forth in claim 1.
As per claim 6, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein each layer of a plurality of layers of the first feature extractor is configured to receive input from both an output of a preceding layer of the plurality of layers of the first feature extractor and a corresponding layer of a plurality of layers of the second feature extractor, thereby facilitating in the transfer of information from the second model to the first model(Tang, Fig.6A,B [0192] As noted above, one aspect of the present disclosure provides a personalized cascaded AI/ML model comprising a common AI/ML sub-model and a cluster AI/ML sub-model that are cascaded such that an output of the common AI/ML sub-model is an input of the cluster AI/ML sub-model or vice versa. The features of the common sub-model are shared and trained by all devices participating in the AI/ML model training process. In contrast, the features of the cluster AI/ML sub-model are shared and trained by the devices in a cluster corresponding to that cluster AI/ML sub-model. In other words, each cluster of devices within the total group of devices participating in the AI/ML model training process has its own cluster-specific cluster AI/ML sub-model. The common AI/ML sub-model and the cluster AI/ML sub-model may be implemented as separate AI/ML models that are cascaded or as one multi-layer AI/ML model (e.g., a deep neural network). For example, FIG. 8A illustrates an example of an AI/ML model 800 in accordance with one embodiment of the present disclosure in which a common AI/ML sub-model 1802 and a cluster AI/ML sub-model 2 804 are implemented as separate AI/ML models that are cascaded such that an output of the common AI/ML sub-model 1802 is an input of the cluster AI/ML sub-model 2 804. On the other hand, FIG. 8B illustrates an example of the AI/ML model 800 in accordance with another embodiment of the present disclosure in which the common AI/ML sub-model 1 802 and the cluster AI/ML sub-model 2 804 are implemented as one multi-layer AI/ML model 810 having N total layers in which the first M layers of the multi-layer AI/ML model 810 implement the common AI/ML sub-model 1 802 and the remaining N-M layers implement the cluster-specific cluster AI/ML sub-model 2 804.) Motivation to combine set forth in claim 1.
As per claim 7, Chu in view of Tang in view of Chalkidis teaches the method of claim 2, wherein the training of the second model using the labeled set of the second data further comprises minimizing a first optimization function, the first optimization function measuring a discrepancy between a prediction generated from the second predictor and a ground truth label(Chu, [0040] [0060] As will be readily understood by a worker skilled in the art, a loss function is a function (comparable to a cost function, error function or objective function) that receives a machine learning model's output (e.g. a prediction) and computes a loss which is the output of the loss function computed based on this received output. In supervised learning, each training data includes an input (e.g. training data sample) and an output (the label, which may be a ground truth or expected output). The loss reflects the difference between the machine learning's output (e.g. prediction) and the label. The loss is used in backpropagation to update model parameters of the machine learning model. Various loss functions can be used, such as those that determine mean squared error, hinge loss, cross entropy loss, etc. The appropriate loss function used in training a machine learning model is dependent on the particular task the machine learning model is being trained for)..
As per claim 8, Chu in view of Tang in view of Chalkidis teaches the method of claim 7, wherein the training of the first model further comprises fixing parameters of the second model and minimizing a second optimization function, the second optimization function measuring a discrepancy between a prediction generated from the first predictor and a ground truth label, and first model parameters are updated to minimize the second optimization function(Chu, [0040] [0060] As will be readily understood by a worker skilled in the art, a loss function is a function (comparable to a cost function, error function or objective function) that receives a machine learning model's output (e.g. a prediction) and computes a loss which is the output of the loss function computed based on this received output. In supervised learning, each training data includes an input (e.g. training data sample) and an output (the label, which may be a ground truth or expected output). The loss reflects the difference between the machine learning's output (e.g. prediction) and the label. The loss is used in backpropagation to update model parameters of the machine learning model. Various loss functions can be used, such as those that determine mean squared error, hinge loss, cross entropy loss, etc. The appropriate loss function used in training a machine learning model is dependent on the particular task the machine learning model is being trained for)..
As per claim 9, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein the aggregating of the parameters of the second model to form the global model further comprises: receiving, at a server, the parameters of the second model from a plurality of parties, computing a weighted average of the received parameters wherein a size of a local training batch for the second model is determined, and forming the global model using the computing weighted average of the parameters(Chalkidis, Fig.2, [0056] The private device 101 may be maintained in a private network of an individual, business, or other organization. For example, the private device 101 may belong to a hospital. In embodiments, the private device 101 may be connected to a weighted knowledge transfer device 104 via interface 110. The private device 101 may be insulated from the public device 106 by the weighted knowledge transfer device 104 (that is, the private device 101 may be inaccessible from the public device 106). Accordingly, users 113 accessing the public machine learning model 107 through the interface unit 109 of the public device 106 cannot retrieve the private dataset 102 through malicious acts (e.g., hacking) because the public machine learning model 107 has been trained solely using the public dataset 108. [0061] The weighted knowledge transfer device 104 is a storage device configured to store one or more functional units used to perform the weighted knowledge transfer process according to the present disclosure. As illustrated in FIG. 2, the weighted knowledge transfer device 104 may include a weighted knowledge transfer unit 105. The weighted knowledge transfer unit 105 is a functional unit configured to perform a weighted knowledge transfer from the private machine learning model 103 to the public machine learning model 107 in order to create a privacy preserving public machine learning model that achieves high performance while maintaining data privacy.).
As per claim 10, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein a learning module using publicly available data is employed for implementing a learning process on the global model, said learning module employing a consistency regularization technique to regulate a consistency of loss, thereby improving the learning process(Chalkidis, Fig.5, [0098] In embodiments, the machine learning unit 512 may utilize a mapping function that adjusts the priority of sets of features in the public training dataset 207 based on their corresponding weights in the similarity weight vector 208. For instance, the public machine learning model 107 being trained by the machine learning unit 512 may use the sets of features in the public training dataset 207 as input and the labels in the public label vector 511 as the prediction target. The performance of the public machine learning model 107 is optimized by minimizing a loss function. Further, as part of the training process, the weights of the sets of features of the public training dataset 207 in the similarity weight vector 208 may be adjusted through a feedback loop between the machine learning unit 512 and the data selection unit 206.).
As per claim 17, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein a Vertical Federated Learning module is deployed to initialize its respective private and public feature extractors and parameters from the first and second feature extractors of the first and second models respectively are provided as input for the initialization of the private and public feature extractors(Chu, Abstract, [0040] Embodiments of the invention provide Secure Vertical Federated Learning (SVFL) methods and systems to securely train private machine learning models for a supervised machine learning task. The training is performed using the private data owned and stored by multiple data owners. Data owners operate computing systems which interact with a central machine learning model, which is operated on a central server. The training also uses a central machine learning model for the supervised machine learning task, without infringing the data privacy and model privacy of any of these data owners. Model privacy refers to privacy of the structure and parameters of used by a private data owner to represent data for use. The use of data may include, for example, use of the data to generate a private machine learning model, to generate inferences or predictions using the private machine learning model based on observations (i.e., new data samples) input to the model, or a combination thereof.., Chalkidis, [0035] For instance, in order to create a user-facing, privacy preserving, public machine learning model that achieves comparable performance to a private machine learning model, knowledge needs to be transferred from the private model to the public model. Conventional methods, however, lack the ability to transfer knowledge from a private model to a public model where great variability exists between a private data set used to train the private model and a public data set used to train the public model. For instance, conventional methods do not disclose techniques for generating a knowledge transfer dataset that can be used in the creation of a privacy preserving machine learning model. As a result, conventional methods do not provide techniques for conveying the characteristics of features in private datasets that can be used in the creation of a privacy preserving machine learning model.) Motivation to combine set forth in claim 1.
As per claim 18, Chu in view of Tang in view of Chalkidis teaches the method of claim 17, wherein the initialization facilitates the training of the Vertical Federated Learning module on overlapping samples shared among multiple data management entities, each entity possessing its own local data and operating within a distributed learning framework(Chu, [0008] Embodiments of the present invention provide for methods, computing device and system for secure vertical federated learning (SVFL) and subsequent operation of the SVFL system. Security is provided by allowing data owners to securely keep their data private. Vertically federated learning is a class of machine learning that involves training a machine learning model using a supervised learning algorithm and samples of data from multiple data owners, and in which different data owners can have the same or overlapping data sample sets but record different sets of features for at least one of the data samples in these data sample sets. Various learning operations, described below according to embodiments of the present invention, can be performed iteratively and in their proper turn by centralized and private machine learning models, or corresponding computing apparatuses. Learning operations are performed by computing systems of different data owners, along with a (e.g. cloud-based) server. Each data owner may be associated with one or more virtual or physical machines that execute a learning algorithm to learn parameters of a model during training of the model. The server may also include one or more such machines. Subsequent operation of a trained (via machine learning) computing systems and servers are also provided for. This operation is also performed in a vertically federated manner which maintains privacy of data and model parameters. [0055] As illustrated in FIG. 1, data owner 1 owns data 106 corresponding to a number of their users (persons) and data owner 2 owns data 108 corresponding to a number of their users. The vertical span of box 110 represents the persons (data samples) for which both data owners 1 and 2 own data. However, data owner 1 owns different (possibly overlapping) data features for these common persons than data owner 2 does. The box 110 represents the data features collectively owned by both data owners for the common persons. When the box 110 is roughly as shown, data is vertically partitioned, and vertical federated learning as described herein may be used. According to vertical partitioning, different data sets {D.sub.1, D.sub.2, . . . , D.sub.N} contain different subsets of features of the same set of data samples. The subsets of features contained in {D.sub.1, D.sub.2, . . . , D.sub.N} may or may not overlap with each other.)
As per claim 19, Chu in view of Tang in view of Chalkidis teaches the method of claim 17, wherein the Vertical Federated Learning module comprises a general predictor configured to receive feature representations outputted from the private and public feature extractors, and to generate a prediction based on said feature representations(Chu, [0060] As will be readily understood by a worker skilled in the art, a loss function is a function (comparable to a cost function, error function or objective function) that receives a machine learning model's output (e.g. a prediction) and computes a loss which is the output of the loss function computed based on this received output. In supervised learning, each training data includes an input (e.g. training data sample) and an output (the label, which may be a ground truth or expected output). The loss reflects the difference between the machine learning's output (e.g. prediction) and the label. The loss is used in backpropagation to update model parameters of the machine learning model. Various loss functions can be used, such as those that determine mean squared error, hinge loss, cross entropy loss, etc. The appropriate loss function used in training a machine learning model is dependent on the particular task the machine learning model is being trained for.).
As per claim 20, Chu in view of Tang in view of Chalkidis teaches the method of claim 1, wherein the first set of data corresponding to the first data space comprises personal or sensitive data and the second set of data corresponding to the second data space comprises publicly accessible or publicly available data(Chalkidis, [0035] For instance, in order to create a user-facing, privacy preserving, public machine learning model that achieves comparable performance to a private machine learning model, knowledge needs to be transferred from the private model to the public model. Conventional methods, however, lack the ability to transfer knowledge from a private model to a public model where great variability exists between a private data set used to train the private model and a public data set used to train the public model. For instance, conventional methods do not disclose techniques for generating a knowledge transfer dataset that can be used in the creation of a privacy preserving machine learning model. As a result, conventional methods do not provide techniques for conveying the characteristics of features in private datasets that can be used in the creation of a privacy preserving machine learning model.) Motivation to combine set forth in claim 1.
Allowable Subject Matter
Claims 11-16 objected to as being dependent upon a rejected base claim, but would be allowable over prior art if rewritten in independent form including all of the limitations of the base claim and any intervening claims and overcoming the 101 rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892.
US 2023/0237321 issued to Cirillo et al., teaches a method preserving privacy in federated machine learning system is provided. In the method, a first computing entity in the federated learning system determines a first labeling matrix based on applying a first set of labeling functions to first data points. The first labeling matrix includes a plurality of first labels. The first computing entity obtains a similarity matrix indicating similarity scores between the first data points and second data points associated with a second computing entity.
US 2025/0259112 issued to Athavale et al., teaches combining machine learning models which respectfully use public and private data using a third machine learning model. Upon training a public data machine learning model and a private data machine learning model, the system trains a public and private data machine learning model using a combination of: (1) historical public data machine learning predictions output by the public data machine learning model, and (2) historical private data machine learning predictions output by the private data machine learning model.
US 2023/0084507 issued to Zhou, teaches fair and secure vertical federated learning. Fair and secure vertical federated learning (FSVFL) systems are disclosed that achieve one or more of the following properties: model fairness, high security, high accuracy, high efficiency, and/or high generality. Private data is retained on local computing systems, which share only their model outputs, and a server or a trusted computing system shares only model gradients and randomly partitioned sets of data sample identifiers with untrusted computing systems. A fairness constraint protects protected classes of data samples against model training resulting in bias on the basis of the protected classes.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BACKHEAN TIV whose telephone number is (571)272-5654. The examiner can normally be reached on Mon.-Thurs. 5:30-3:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TONIA DOLLINGER can be reached on (571) 272-4170. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BACKHEAN TIV/
Primary Examiner
Art Unit 2459