Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/22/2025 has been entered.
Response to Arguments
Applicant’s argument filed 12/22/2025 have been fully considered but the arguments provided for 103 rejections are not persuasive.
Applicant’s Argument: On page 9-10 of Applicant’s response to rejections under 35 U.S.C. 103, applicant states that the cited references do not teach the new claim limitations.
Examiner’s Response: Applicant’s argument is not persuasive. Applicant’s arguments with respect to claims 1, 7, and 13 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 19-24 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 19-24 recites “a first client subsystem that is classified as a label holder”. There is no support in the Specification that discloses the label holder is a client subsystem. The Specification (par. 50-51) does not explicitly disclose a client subsystem is the label holder. Figure 5 and Figure 6 describes the algorithms in par. 50 and 51 in the Specification. From Figure 5 and 6, the representations are sent to the server. Under the broadest reasonable interpretation, the server subsystem or client subsystem can be classified as the label holder. The Examiner interprets the label holder to be a server based on the disclosure of the Specification and the recited claims as a whole.
Claims 20, 22, and 24 recites the first client subsystem performs the computing of a loss and sending the gradient information to each of the client subsystems. There is no support in the Specification that discloses the steps are performed by a client subsystem. The Specification (par. 51) disclose a “label holder computes the loss and sends gradient information to parties”. As explained above, the label holder is not limited to only a client subsystem. Examiner interprets the computing and sending steps can be performed by a server subsystem.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 19, 21, and 23 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 19, 21, and 23 are dependent claims to independent claims 1, 7, and 13, respectively. Claims 1, 7, and 13 recites that the plurality of client subsystems stores unlabeled data and the client subsystem is trained using unsupervised learning. It is unclear what constitutes as “all available labeled data” from claims 19, 21, and 23. There is no details on where the labeled data is originating from. The claims need to be amended to further define the scope of the invention with more clarity. The Examiner interpret the client subsystem may store both unlabeled and labeled data.
Claims 19, 21, and 23 also recites “training, by the first client subsystem, its untrained server-side ML model using the respective representations”. From the independent claims, the untrained server-side ML model resides in the server subsystem. It is unclear why the client subsystem is performing the training on the server-side ML model. The Examiner interprets the training to be performed by the server subsystem.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-8, 13-14, 19, 21, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Milletari (US11804050B1), in view of Suzuki (US20220405606A1).
Regarding claim 1, Milletari teaches:
“A computer-implemented method (CIM) for use with a federated computer system that includes a server subsystem and a plurality of client subsystems, (abstract, Figure 5, Figure 5 shows a plurality of client devices as training nodes and the results are sent to the aggregation servers) with each client subsystem including stored unlabeled data, with the server subsystem being in communication with each client subsystem through a communication network, and with each client subsystem not being permitted access to stored unlabeled data of other client subsystems of the federated computer system, the CIM comprising” (col. 6, lines 22-32; col. 10, lines 45-65; col. 27, lines 30-44; Figure 5, The training of the models can be unsupervised and using unlabeled data. The client devices train the models locally and communicate the results to the aggregation servers over a network. The model trainer at each client device uses a different training dataset and the client devices do not share training data. In one embodiment, vertical federated learning can be implemented between the client devices to ensure data privacy.)
“receiving, by the server subsystem, an ” ([col. 3, lines 28-37; col. 4, lines 16-25; col. 19, lines 47-55, Figure 5], Machine learning model (MLM) 108 may be collaboratively trained by training nodes and MLM 106. Federated learning may be used to train a central model by a plurality of local models. In at least one embodiment, a client-server network environment may include client devices as edge device for training MLM 106 and aggregation servers that train MLM 108. MLM 108 is trained by a plurality of MLM 106 and it is implied that during the first round of training, the client and server side starts off with an untrained model.)
“receiving, from each given client subsystem, over the communication network and by the server subsystem, a respective collection of representations that have been obtained at the given client subsystem through models located at the given client subsystem and trained on the unlabeled data of the given client subsystem using Contrastive Unsupervised Representation Learning (CURL)” ([col. 3, lines 28-37; col. 4, lines 26-48; col. 10, lines 45-65; col. 19, lines 47-55; Figure 1A and 5], The plurality of client devices uses unsupervised learning to train a ML model locally and provide the learned parameters of weights and biases to the training aggregator server over the network. Training aggregator receives information from training nodes and uses the information to train MLM 108. A client server network environment may be used for the collaborative training. The client devices train MLM 106 and provide information to the aggregation servers to train MLM 108. Consensus determiner and discrepancy determiner may analyze data from the training nodes to determine the similarities and differences (contrastive learning) of parameters between training nodes. The claim does not define the scope of the limitation “Contrastive Unsupervised Representation Learning”. Under the broadest reasonable interpretation, Milletari discloses the claim limitation.)
“aggregating, by the server subsystem, the representations of the collections of representations respectively received from each client subsystem of the plurality of client subsystems to obtain an input to the ” ([col. 4, lines 49-57; col. 11, lines 5-27; col. 19, lines 47-55, Figure 5], The parameter determiner aggregates the parameters of local machine learning models to generate parameters for the central model. Machine learning model (MLM) 108 may be collaboratively trained by training nodes and MLM 106. Federated learning may be used to train a central model by a plurality of local models. In at least one embodiment, a client-server network environment may include client devices as edge device for training MLM 106 and aggregation servers that train MLM 108. MLM 108 is trained by a plurality of MLM 106 and it is implied that during the first round of training, the client and server side starts off with an untrained model. Consensus determiner analyzes the representation data from the plurality of training nodes and may determine a consensus value by combining values of parameters for each training nodes.)
“inputting the input to the Self-Supervised Vertical Federated Learning (SS-VFL) to thereby obtain a trained server-side ML model that can recognize a plurality of patterns when the trained server-side ML model receives new representations from the plurality of client subsystems” ([col. 3, lines 55-65; col. 4, lines 16-25 & 65-67; col. 5, lines 1-14; col. 27, lines 29-37], The aggregated parameters are input into the central model for training over multiple iterations. A contribution determiner may determine the amount of influence (plurality of patterns) for each training nodes in training the central model based on weights. The claim does not define the scope of the limitation “Self-Supervised Vertical Federated Learning”. It is unclear what constitutes as Self-Supervised Vertical Federated Learning without further defining the functionality of the claim limitation. Under the broadest reasonable interpretation, Milletari discloses the claim limitation. Machine learning model may be auto-encoders, which is a type of self-supervised learning process. It is also disclosed that the training of the models can consist of vertical federated learning.)
Milletari does not explicitly disclose an implementation of “untrained server-side ML model”.
However, Suzuki discloses in the same field of endeavor:
“receiving, by the server subsystem, an untrained server-side machine learning (ML) model” ([0042-0043], The server receives a base prediction model that is an untrained neural network.)
“aggregating, by the server subsystem, the representations of the collections of representations respectively received from each client subsystem of the plurality of client subsystems to obtain an input to the untrained server-side machine learning (ML) model ...” ([0042-0043; 0048-0049], The clients send model parameters to the server and the server performs an integration process of the parameter to generate an updated model.)
“inputting the input to the untrained server-side ML model to train the untrained server-side ML model ...” ([0042-0043; 0048-0050], The model parameter from each training base is combined at the server to train the untrained server-side machine learning model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “untrained server-side ML model” from Suzuki into the teaching of Milletari. Doing so can improve training efficiency in federated learning system by using a plurality of client training devices to update a global ML model (Suzuki, abstract).
Regarding Claim 7:
The claim recites an article of manufacture (“A computer program product”) that performs the method as described in claim 1. Therefore claim 7 is rejected under the same reasons mention for claim 1. The additional elements of claim 7 is addressed below with the Milletari reference:
“one or more computer readable storage media” ([col. 113, lines 1-43], A computer-readable storage medium stores the instructions to perform the operations.)
“computer code stored collectively in the one or more computer readable storage media, with the computer code including data and instructions to cause a processor(s) set to perform at least the following operations” ([col. 113, lines 1-43], A computer-readable storage medium stores the instructions to perform the operations.)
Regarding Claim 13:
The claim recites a system (“A computer system”) that performs the method as described in claim 1. Therefore claim 13 is rejected under the same reasons mention for claim 1. The additional elements of claim 13 is addressed below with the Milletari reference:
“a processor(s) set” ([col. 54, lines 47-57], Figure 16A shows a computer system consisting of a processor.)
“a set of storage device(s)” ([col. 6, lines 52-65 , Figure 2A], Figure 2A shows multiple data storage components of the system.)
“computer code stored collectively in the set of storage device(s), with the computer code including data and instructions to cause a processor(s) set to perform at least the following operations” ([col. 6, lines 52-65], The hardware shown in Figure 2A and 2B may store data and instructions to perform the steps of the federated learning system.)
Regarding claims 2, 8, and 14, Milletari teaches:
“receiving, by each client subsystem of the plurality of client subsystems, new data for analysis” (col. 5, lines 13-19; col. 6, lines 20-32, The training of the models is performed over multiple iterations. Training dataset may be data generated on-site at the local training node.)
“inputting the new data to a respective trained ML model of each client subsystem of the plurality of client subsystems, to obtain a collection of new representations” ([col. 5, lines 13-19], Local models at each training node process new training datasets and a new set of parameters are determined.)
“sending, by each client subsystem, through the communication network and to the server subsystem, the collection of new representations” ([col. 5, lines 28-33], The parameters of the local models are sent to the training aggregator through the network.)
“aggregating, by the server subsystem, the representations of the collections of new representations to obtain a new input to the trained server-side ML model” ([col. 5, lines 34-39], The parameter of the local models are combined by the training aggregator.)
“using the trained server-side ML model to make a first prediction based on the new input” ([col. 54, lines 14-31], The federated learning system may be implemented in cloud-based servers and autonomous vehicle. The servers receive image data from the vehicle to perform inferencing of an object classification task.)
Regarding claims 19, 21, and 23, Milletari teaches:
“computing, by each client subsystem, respective representations for all available labeled data” (col. 3, lines 28-37; col. 10, lines 63-67; col. 11, lines 1-4, The plurality of client devices trains a ML model locally and provide the learned parameters of weights and biases to the training aggregator server over the network. Semi-supervised learning may be used and it consists of a training dataset with both labeled and unlabeled data.)
“aggregating the respective representations at a first client subsystem that is classified as a label holder” ([col. 4, lines 49-57; col. 11, lines 5-27; col. 19, lines 47-55, Figure 5], The parameter determiner aggregates the parameters of local machine learning models to generate parameters for the central model. Machine learning model (MLM) 108 may be collaboratively trained by training nodes and MLM 106. Federated learning may be used to train a central model by a plurality of local models.)
“training, by the first client subsystem, its ” ([col. 3, lines 55-65; col. 4, lines 16-25 & 65-67; col. 5, lines 1-14; col. 27, lines 29-37], The aggregated parameters are input into the central model for training over multiple iterations.)
Milletari does not explicitly disclose an implementation of “untrained server-side ML model”.
However, Suzuki discloses in the same field of endeavor:
“training, by the first client subsystem, its untrained server-side ML model using the respective representations” ([0042-0043], The server receives a base prediction model that is an untrained neural network. The neural network may be trained.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “untrained server-side ML model” from Suzuki into the teaching of Milletari. Doing so can improve training efficiency in federated learning system by using a plurality of client training devices to update a global ML model (Suzuki, abstract).
Claims 3, 9, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Milletari (US11804050B1), in view of Suzuki (US20220405606A1), and Cha, "Implementing Vertical Federated Learning Using Autoencoders: Practical Application, Generalizability, and Utility Study".
Regarding claims 3, 9, and 15, Milletari in view of Suzuki teaches:
“the collections of representations obtained at the given client subsystem are in the form of vectors” ([Milletari, col. 11, lines 28-35; col. 13, lines 1-8], The set of parameters from the local training nodes are provided as a vector)
Milletari in view of Suzuki does not explicitly disclose an implementation of “the unlabeled data of the client subsystems cannot be derived from the vectors”. However, Cha discloses in the same field of endeavor:
“the unlabeled data of the client subsystems cannot be derived from the vectors” ([pg. 5, section “Principal Results”, par. 1; Table 1], A vertical federated learning framework consists of transforming original data into latent representations for ML model training. The perturbed data ensures data privacy during training. Table 1 shows the dataset with 5 feature dimensions, which can be represented as a vector that has 5 attributes to describe a data.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the unlabeled data of the client subsystems cannot be derived from the vectors” from Cha into the teaching of Milletari in view of Suzuki. Doing so can provide data protection during unsupervised learning of the ML model (Cha, abstract).
Claims 20, 22, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Milletari (US11804050B1), in view of Suzuki (US20220405606A1), and Han, "Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach".
Regarding claims 20, 22, and 24, Milletari in view of Suzuki teaches:
“inputting labeled data to the models on each client subsystem” ([Milletari, col. 10, lines 63-67; col. 11, lines 1-4, Semi-supervised learning may be used and it consists of a training dataset with both labeled and unlabeled data. The training nodes can receive labeled data for semi-supervised learning.)
“aggregating outputs of the models at a first client subsystem classified as a label holder” ([Milletari, col. 4, lines 49-57; col. 11, lines 5-27; col. 19, lines 47-55, Figure 5], The parameter determiner aggregates the parameters of local machine learning models to generate parameters for the central model. Machine learning model (MLM) 108 may be collaboratively trained by training nodes and MLM 106. Federated learning may be used to train a central model by a plurality of local models. Under the broadest reasonable interpretation, a server can be classified as the label holder.)
Milletari in view of Suzuki does not explicitly disclose an implementation of “computing, by the first client subsystem, a loss using the outputs of the models” and “sending, by the first client subsystem and to each of the client subsystems, gradient information, wherein the models on each client subsystem are updated according to the gradient information”. However, Han discloses in the same field of endeavor:
“computing, by the first client subsystem, a loss using the outputs of the models” ([pg. 3, section A. par. 1-2; pg. 7-8, section E, par. 1-2], Each client computes losses on a data sample and the losses are sent to the server. The server also computes averages of the losses.)
“sending, by the first client subsystem and to each of the client subsystems, gradient information, wherein the models on each client subsystem are updated according to the gradient information” ([pg. 3, section A. par. 1-5; pg. 7-8, section E, par. 4; Figure 3], The framework consists of aggregating the sparsified gradients after every local update step and model weight vector is updated in every training round. Figure 3 shows the communication between the server and clients. The server computes and sends the gradient information to all client devices.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “computing, by the first client subsystem, a loss using the outputs of the models” and “sending, by the first client subsystem and to each of the client subsystems, gradient information, wherein the models on each client subsystem are updated according to the gradient information” from Han into the teaching of Milletari in view of Suzuki. Doing so can reduce the communication overhead and improve the overall efficiency of federated learning by implementing gradient sparsification (Han, abstract).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/JEREMY L STANLEY/Examiner, Art Unit 2127
Prosecution Insights