Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 03/03/2026 have been entered. Claims 1-20 remain pending in the
application.
Applicant’s amendment, with respect to the claim rejection(s) of claim 1-20 under 35 U.S.C 112b filed 12/11/2025 have been considered and are persuasive. Therefore, the previous rejections as set forth in the previous office action has been removed.
Applicant’s amendment and argument, with respect to the claim rejection(s) of claim 1-20 under 35 U.S.C 103 filed 12/11/2025 have been considered and are not persuasive. Therefore, the previous rejections as set forth in the previous office action will be maintained.
The applicant argues that the current rejection fails to teach the newly amended limitation directed to the historical weights of a historical global ML model. In particular, Applicant asserts the Office previously acknowledged that Wang does not disclose transmitting historical weights for a corresponding historical version of the global ML model. Applicant further argues that Yao does not cure this deficiency because Yao merely teaches using historical global weights as teaches for knowledge distillation, but does not teach historical weights that corresponds to prior primary weights for a prior primary version of the global ML model from one or more prior rounds of decentralized learning, where those historical weights are further updated using stale updates asynchronously received from straggler computing devices after the prior rounds. Accordingly, the contends that the cited combination fails to disclose or suggest the amended historical weight/stale-update feature.
The examiner respectfully disagrees. Applicant argues that Wang fails to disclose the amended historical-weight limitation and that Yao merely teaches historical global models used as teacher for knowledge distillation. However, the rejection does not rely on Yao alone for the entirety of the amended limitation. Rather, Wang teaches or at least suggests the recited relationship between prior/current model parameters, stale updates, and updating an earlier/historical version of the global model, while Yao further teaches the use of past/historical global models as teacher models.
Specifically, Wang discloses an asynchronous federated learning system in which a client pulls a version of a global parameter from the server, computes a local gradient using the client’s local data and the pulled version of the global parameter, and transmits the gradient to the server. The local gradient corresponds to the claimed update from a computing device. Wang further discloses that the server tracks versions of the global parameter over time and checks which version of the global parameter was used to compute a received gradient. Thus, when a received gradient is computed using an earlier version of the global parameter but is received after another client has already pushed a gradient based on a later version, the received gradient corresponds to a stale/asynchronously received update from a straggler computing device (Wang at ¶22)
Wang further teaches comparing that received gradient with another gradient computed using the same or similar version of the global parameter and, depending on the determined distance, aggregating the received gradient with either the most recent version of the global parameter or an earlier version of the global parameter. Accordingly, Wang teaches or at least suggests that a stale update from a straggler device may be used to update an earlier version of the global parameter.
Wang’s current version of the global parameter corresponds to the claimed primary weights/primary version of the global ML model, and Wang’s earlier version of the global parameter corresponds to the claimed historical weights/historical version of the global ML model. because Wang tracks version of the global parameter over time, the earlier version of the global parameter would have been a prior current/primary version, and therefore its parameters correspond to prior primary weight for a prior primary version of the global ML model from a prior round.
Yao further supports the combination by teaching that past/historical global models may be used as teacher models for client training. Therefore, Wang in view of Yao teaches or at least suggests the amended limitation requiring historical weights corresponding to prior primary weights for a prior primary version of the global ML model, where the historical weights are further updated using stale updates asynchronously received from straggler computing devices.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-9, 11-17 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et.al (US 20210342749 A1) in view of Yao et.al (NPL: LOCAL-GLOBAL KNOWLEDGE DISTILLATION IN HETEROGENEOUS FEDERATED LEARNING WITH NON-IID DATA)
Regarding claim 1,
Wang teaches or at least suggest the 1st limitation “transmitting, to a population of computing devices, (i) primary weights for a primary version of the global ML model, and (ii) historical weights for a corresponding historical version of the global ML model, wherein the historical weights for the corresponding historical version of the global ML model correspond to prior primary weights for a prior primary version of the global ML model from one or more prior rounds of decentralized learning for updating the global ML model, that occurred prior to the given round of decentralized learning, that are further updated using stale updates asynchronously received from one or more straggler computing devices subsequent to one or more of the prior rounds of decentralized learning for updating of the global ML model” (paragraph 20 “Embodiments of the present invention further provide a method for training generic machine learning models (such as deep neural networks or edge computing nodes) from decentralized data. Implementing aspects of embodiments of the invention with asynchronous federated learning is not restricted to the type of data”, paragraph 21 “The asynchronous federated learning system 100 causes each of clients 11, 12 and 13 to pull a current version of the global parameter 31 from the server 30”, and paragraph 22 “Whenever a gradient (e.g., any of local gradients 32, 33, and 34) is received by the server 30, the asynchronous federated learning system 100 causes the server 30 to update the current version of the global parameter to a new version of the global parameter ... The server 30 tracks the versions of the global parameter over time. For example, based on client 11 pushing a gradient 32 to the server 30, the server 30 checks which version of global parameter the client 11 used to compute the received gradient 32. If there is another client (e.g., client 12) that has already pushed a gradient 33 to the server 30 that was computed based on a later version of the global parameter 31, server 30 computes the difference (according to any appropriate distance metric) of the gradient 32 from client 11 and an earlier gradient from client 12, wherein the earlier gradient from client 12 was computed using a same or similar version of the global parameter as the received gradient 32 from client 11. Depending on the determined distance, the server 30 may aggregate the received gradient 32 with either the most recent version of the global parameter or an earlier version of the global parameter.” Wang discloses a method and system of adaptive asynchronous federated learning from decentralized data. Wang discloses the method comprise of a global server and clients with global model parameters. The global parameters may be transmitted to each client as each clients pull a current version of the global parameter from the server, wherein the global server with ML model corresponds to the global machine learning (ML) model, as claimed. Furthermore, Wang discloses that each clients pulls a version of the global parameter from the server and computes a local gradient using the client’s local data and the pulled version of the global parameter, wherein the local gradient corresponds to the claimed update from a computing device for updating the global ML model. Wang further discloses that the server tracks versions of the global parameter over time and checks which version of the global parameter was used to compute a received gradient. According to Wang, there are recent version of the global parameter or an earlier version of the global parameter, which are pulled by each client to compute gradient for updating the global model over time, which corresponds to the transmitting of historical weight for the historical version and primary weight for the primary version of the global model, as claimed. Wang’s earlier version of the global parameter corresponds to the recent version of the global parameter because the recent version of the parameter is obtained from updating the earlier version of the parameter with the gradient from one of the client device, which teaches or at least suggest the claimed historical weights corresponds to prior primary weights of the global ML model for updating the global ML model that occurred prior to the given round of decentralized learning. Finally, Wang discloses clients may compute and transmit gradient (update) at different times, wherein a client that received an earlier version of the global parameter may already sent gradient to update the model, while another client may transmit another gradient (update) obtained from the same earlier version of the global parameter at a later time (example of client 11 and client 12 with earlier gradient), in which the system may determine to aggregate such new gradient with the most recent version or the earlier version of the global parameter, thereby teaches or at least suggest the claimed process of historical weights are updated using stale updates asynchronously received from one or more straggler computing devices subsequent to one or more of the prior learning rounds, because such later transmission of gradient from another client and the aggregating with earlier version of the global parameter corresponds to the straggler device and stale update, as claimed.)
Wang teaches or at least suggests a part of the 2nd limitation “causing each of the computing devices of the population to generate a corresponding update for the primary version of the of the global ML model via utilization of the primary weights for the primary version of the global ML model at each of the computing devices of the population and via utilization of the historical weights for the corresponding historical version of the global ML model...” (paragraph 21 “The asynchronous federated learning system 100 causes each of clients 11, 12 and 13 to pull a current version of the global parameter 31 from the server 30. The asynchronous federated learning system 100 further causes each of clients 11, 12 and 13 to compute a gradient using the client's respective local dataset 20, 21, and 23 and the received version of the global parameter 31”, and paragraph 22 “... Depending on the determined distance, the server 30 may aggregate the received gradient 32 with either the most recent version of the global parameter or an earlier version of the global parameter” Wang discloses causing each client to pull a current version of the global parameter, which corresponds to the claimed primary weights for the primary version of the global ML model. Wang also discloses the updating of the global parameter with the received gradient, thereby obtaining current version global parameter as well as earlier version of the global parameter, which corresponds to the claimed historical weights. Wang further discloses causing each client to compute a gradient using each client's respective local dataset and the recent/earlier of the global parameter to update the weight of the global ML model, wherein the generated gradients by each client may be aggregated to update the most recent version of the global parameter or an earlier version of the global parameter, which teaches or at least suggests the process of generating a corresponding update via the utilization of the primary weights for the primary version of the global ML model, and utilization of the historical weights for the corresponding historical version of the global ML model, as claimed.)
Wang teaches the 3rd limitation “asynchronously receiving, from one or more of the computing devices of the population, a first subset of the corresponding updates for the primary version of the global ML model” (paragraph 21 “The asynchronous federated learning system 100 further causes each of clients 11, 12 and 13 to compute a gradient using the client's respective local dataset 20, 21, and 23 and the received version of the global parameter 31, and causes each of clients 11, 12 and 13 to push the respective computed local gradient 32, 33, and 34 to the server 30” Wang discloses the asynchronous federated learning system further cause each client to compute a gradient to be pushed to the global server to update the weight of the ML model at the global server, which is analogous to the receiving of the subset of the corresponding updates for the primary version of the global ML model from one or more of the computing devices within the claim.)
Wang teaches the 4th limitation “causing, based on the first subset of the corresponding updates, the primary version of the global ML model to be updated to generate updated primary weights for an updated primary version of the global ML model” (paragraph 22 “Whenever a gradient (e.g., any of local gradients 32, 33, and 34) is received by the server 30, the asynchronous federated learning system 100 causes the server 30 to update the current version of the global parameter to a new version of the global parameter by aggregating the received local gradient (e.g., any of local gradients 32, 33, and 34) with an appropriate version of the global parameter” Wang discloses the asynchronous federated learning system causes the server to update the current version of the global parameter to a new version of the global parameter by aggregating the received local gradient, which is analogous to the claimed process of causing the primary version of the global ML model to be updated for an updated primary version of the global ML model based on the received update.)
Wang teaches the 5th limitation “subsequent to the given round of decentralized learning for updating of the global ML model: asynchronously receiving, from one or more of the other computing devices of the population, a given corresponding update for the primary version of the global ML model that was not received during the given round of decentralized learning for updating of the global ML model” (paragraph 22 “The server 30 tracks the versions of the global parameter over time. ... The server 30 tracks the versions of the global parameter over time. For example, based on client 11 pushing a gradient 32 to the server 30, the server 30 checks which version of global parameter the client 11 used to compute the received gradient 32. If there is another client (e.g., client 12) that has already pushed a gradient 33 to the server 30 that was computed based on a later version of the global parameter 31, server 30 computes the difference (according to any appropriate distance metric) of the gradient 32 from client 11 and an earlier gradient from client 12, wherein the earlier gradient from client 12 was computed using a same or similar version of the global parameter as the received gradient 32 from client 11.” Wang discloses the global server tracks the version of the global parameter over time. Wang further discloses tracking the gradient computed from the version of the received global parameter at each client such that the server may receive an update (e.g., gradient 32 from client 11) after another update (e.g., gradient 33 from client 12) has already been processed, meaning gradient 32 is received after gradient 33 has already been processed. Thus, gradient 32 constitutes a later-arriving update that is received after an earlier update has been processed. This is analogous to the claimed process of asynchronously receiving from one or more of the other computing devices a given corresponding update for the primary version of the global ML model that was not received during the given round of decentralized learning because Wang explicitly teaches the server receiving and handling updates after it has already processed other updates, and that the server continues updating the global model using the later-received update within the same asynchronous learning process.)
Wang teaches the 6th limitation “causing, based on the given corresponding update, corresponding historical weights for the corresponding historical version of the global ML model to be updated to generate a corresponding updated historical version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating of the global ML model” (paragraph 22 “For example, based on client 11 pushing a gradient 32 to the server 30, the server 30 checks which version of global parameter the client 11 used to compute the received gradient 32. If there is another client (e.g., client 12) that has already pushed a gradient 33 to the server 30 that was computed based on a later version of the global parameter 31, ... Depending on the determined distance, the server 30 may aggregate the received gradient 32 with either the most recent version of the global parameter or an earlier version of the global parameter. ... If the determined distance is relatively large, the data distributions at client 11 and client 12 are different, such that the server 30 aggregates the received gradient 32 with an earlier version of the global parameter” Wang discloses the server may aggregate the received gradient 32 with an earlier version of the global parameter, thus Wang teaches updating an earlier version of the global parameter based on a later-received gradient, which corresponds to updating the historical version of the global ML model using the given corresponding update within the claim. By aggregating the later-received gradient with an earlier version of the global parameter, the server effectively produces an updated version of the earlier (historical) model state for continued use. This behavior aligns with the claimed process of using the received update to generate updated historical weights of the global ML model and enabling that updated historical version to be utilized in one or more subsequent rounds of learning.)
Wang teaches the 7th limitation “in response to determining that one or more deployment criteria are satisfied, causing a most recently updated primary version of the global ML model to be deployed as a final version of the global ML model” (paragraph 19 “the server can aggregate a received gradient with multiple versions of global parameters, in which case multiple models will be produced after this aggregation. Out of these multiple models, one model (i.e., one global parameter) will be selected according to the loss or accuracy on a validation dataset. In accordance with aspects of the invention, the best version of the model is determined based on the above-described loss or accuracy on a validation dataset. Such model selection can happen either immediately after aggregating (with multiple versions of global parameters) after one gradient update, or multiple models can coexist for any appropriate amount of time before the best model gets selected.” Wang discloses selecting from among multiple updated versions of the global model parameter generated during asynchronous aggregation, a version that meets a performance threshold on a validation dataset. This model-selection process corresponds to determining whether deployment criteria are satisfied within the claim. Once the criterion is met, the selected global model parameters is treated as the best version of the model for deployment, which aligns with the claimed step of causing the most recently updated primary version of the global ML model to be deployed as the final version of the global ML model.)
Wang does not teach the using historical version as the teacher model aspect the 2nd limitation “causing each of the computing devices of the population to generate a corresponding update for the primary version of the of the global ML model ... via utilization of the historical weights for the corresponding historical version of the global ML model as a corresponding teacher model at each of the computing devices of the population” However, Yao teaches or at least suggests that aspect of the limitation (Algorithm 1 FEDGKD: Global Knowledge Distillation in Federated Learning, Page 1 “To tackle this issue, one intuitive idea is to guide the local model training by the global teachers, i.e., past global models, where each client learns the global knowledge from past global models via adaptive knowledge distillation techniques”, and Page 2 section 1 “we propose a novel ensemble-based global knowledge distillation method, named FEDGKD, which fuses the knowledge from past global models to tackle the client drift in training ... We introduce an ensemble-based knowledge distillation technique to transfer the information of historical global models to local model training” Yao discloses employ the past global models as the teacher to guide the local model training at each client via adaptive knowledge distillation techniques such that the result from the local training may be transmitted back to the global model for further update as indicated in the Algorithm 1, which corresponds to the claimed process of generating a corresponding update for the primary version of the of the global ML model via utilization of the corresponding historical version of the global ML model as a corresponding teacher model at each of the computing devices.)
Before the effective filing date, it would have been obvious to a person ordinary skilled in the art to combine the teaching of the method and system of adaptive asynchronous federated learning by Wang, with using the past global model as the teacher model at the local model via adaptive knowledge distillation techniques by Yao. The motivation to do so is referred to in Yao’s disclosure (page 1 “As clients perform local updates on heterogeneous data through heterogeneous systems, their local models drift apart. To tackle this issue, one intuitive idea is to guide the local model training by the global teachers, i.e., past global models, where each client learns the global knowledge from past global models via adaptive knowledge distillation techniques. Coming from these insights, we propose a novel approach for heterogeneous federated learning, namely FEDGKD, which fuses the knowledge from historical global models for local training to alleviate the “client-drift” issue ... The proposed method is guaranteed to converge under common assumptions, and achieves superior empirical accuracy in fewer communication runs than five state-of-the-art methods”, page 3 section 3.1 “local data fail to represent the overall global distribution. For example, given a model trained on airplane and bike images, we cannot expect the features learned by the model to recognize birds and dogs. Therefore, for non-IID data, we should control the drift and bridge the gap between the representations learned by the local model and the global model ... Additionally, the ensemble (Polyak & Juditsky, 1992) of multiple historical global models can further enhance the power of global knowledge to handle the local model drift problem by providing a more comprehensive view of the global data distribution”, and page 12 section 6 “FEDGKD benefits from the ensemble and knowledge distillation mechanisms to produce a more accurate model.” Yao discloses the benefit of the FEDGKD method, which help incorporating the past (historical) global models as the teacher to configure the local model at each client to alleviate the “client-drift” issue since local data may fail to represent the overall global distribution, thus the method helps provide a more accurate and robust model learning at the client side and the global side. Therefore, one of ordinary skilled in the art may combine the teaching by Yao with the teaching by Wang to configure the local model at each worker based on the past (historical) version of the global model to improve the overall training procedure.)
Regarding claim 8 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches a part of the limitation “The method of claim 1, wherein the primary weights for the primary for the primary version of the ML model were generated based on an immediately preceding round of the decentralized learning for updating of the global ML model” (paragraph 27 “In block 301 a server, such as server 30 of FIG. 1, determines an initial version of a global parameter for an asynchronous federated learning process. In block 302, the initial version of the global parameter that was determined in block 301 is provided from the server to each client (e.g., clients 11, 12, and 13) in the asynchronous federated learning system as a current version of the global parameter.” Wang discloses the server determines an initial version of the global parameter for an asynchronous federated learning process and then provides this current version of the global parameter to the clients for their local training. Wang further explains that the server updates the global parameter by aggregating the gradients that were previously received from the clients. Since the initial version of the global parameter is provided for the asynchronous federated learning process of transmitting and updating process from each client, the initial version of the global parameter corresponds to the primary weights generated based on an immediately preceding round of the decentralized learning for updating of the global ML model within the claim. In wang, the global parameter corresponds to the weights of the global machine learning model, since each clients compute gradients based on the received global parameter, which in gradient-based learning, the update process using gradient are the learnable model parameters, i.e., the model weights as understood by one of ordinary skilled in the art. Therefore, the global parameter by Wang can be recognized as representation for model weights of the global model.)
Yao teaches a part of the limitation “wherein the corresponding historical version of the global ML model was generated based on at least one further preceding round of the decentralized learning for updating of the global ML model that is prior to the immediately preceding round of the decentralized learning for updating of the global ML model” (Page 4 section 3.1 “Figure 1: An overview of FEDGKD: an ensemble model wt is learned by aggregating information from historical global models. The ensemble model is then sent to sampled client, whose knowledge is distilled to local models for a good feature distribution” Yao discloses the FEDGKD method in which an ensemble model wt is learned by aggregating information from historical global models, wherein one of ordinary skilled in the rat would recognize that each historical global model by Yao corresponds to a version of the global model produced in a past update of the federated learning process and analogous to the historical version of the global ML model generated based on at least one further preceding round prior to the immediately preceding round for updating of the global ML model within the claim, because these historical global models are explicitly described as results of prior training iterations, a person ordinary skilled in the art would understand that each historical global model necessarily originates from a past update of the federated learning process which corresponds to the further preceding round of decentralized learning in the claim.)
Regarding claim 9 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches the limitation “The method of claim 1, further comprising: causing, based on the given corresponding update, prior corresponding historical weights for a prior corresponding historical version of the global ML model, that was generated based on at least one further preceding round of the decentralized learning for updating of the global ML model that is prior to the immediately preceding round of the decentralized learning for updating of the global ML model, to be updated to update the prior corresponding updated historical version of the global ML model for utilization in one or more of the subsequent rounds of decentralized learning for further updating of the global ML model” (paragraph 22 “For example, based on client 11 pushing a gradient 32 to the server 30, the server 30 checks which version of global parameter the client 11 used to compute the received gradient 32. If there is another client (e.g., client 12) that has already pushed a gradient 33 to the server 30 that was computed based on a later version of the global parameter 31, ... Depending on the determined distance, the server 30 may aggregate the received gradient 32 with either the most recent version of the global parameter or an earlier version of the global parameter. ... If the determined distance is relatively large, the data distributions at client 11 and client 12 are different, such that the server 30 aggregates the received gradient 32 with an earlier version of the global parameter” Wang discloses the server may aggregate the received gradient 32 with an earlier version of the global parameter, thus Wang teaches updating an earlier version of the global parameter based on a later-received gradient, which corresponds to updating the historical version of the global ML model using the given corresponding update within the claim. By aggregating the later-received gradient with an earlier version of the global parameter, the server effectively produces an updated version of the earlier (historical) model state for continued use. This behavior aligns with the claimed process of using the received update to generate updated prior corresponding historical weights of the prior corresponding historical version of the global ML model and enabling that updated historical version to be utilized in one or more subsequent rounds of learning.)
Regarding claim 11 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches the limitation “transmitting, to a plurality of computing devices, most recently updated primary weights for the most recently updated primary version of the global ML model, wherein transmitting the most recently updated primary weights for the most recently updated primary version of the global ML model to given computing device, of the plurality of computing device, causes the given computing device to” (paragraph 24 “System 200 includes a first client 11 and a second client 12 that are in communication with a global parameter version tracking module 48, which is included in server 30 of FIG. 1. As illustrated in system 200, the global parameter version tracking module 48 tracks versions (i.e., V0, V1, V2, V3, and V4) of the global parameter over time 47. Initially, both clients 11 and 12 receive an initial version (V0) 39 of the global parameter from the global parameter version tracking module 48. Client 12 determines a gradient 40 based on V0, and pushes the gradient 40 to the server 30. The server 30 determines a new version (V1) of the global parameter based on the gradient 40, and client 12 receives V1 41 from the global parameter version tracking module 48. Client 12 determines a gradient 42 based on V1 41, and pushes the gradient 42 to the server 30. The server 30 determines a new version (V2) of the global parameter based on the gradient 42, and client 12 receives V2 43 from the global parameter version tracking module 48. Client 12 determines a gradient 44 based on V2 43, and pushes the gradient 44 to the server 30” Wang demonstrates the continuous process of updating and transmitting the updated global parameter to one of the clients. The client 12 receives the initial global parameter, determines a gradient and pushes the gradient back to the global server, wherein the server determines a new version of the global parameter based on aggregating the gradient to update its global model and then the global server continuously pushed the updated new version of the global parameter into the client 12. This continuous process of pushing the updated new version of the global parameter after receiving gradient from one of the clients is analogous to the claimed process of transmitting most recently updated primary weights for the most recently updated primary version of the global ML model)
Wang teaches the limitation “replace any prior weights for a prior version of the global ML model with the most recently updated primary weights for the most recently updated primary version of the global ML model” (paragraph 24 “System 200 includes a first client 11 and a second client 12 that are in communication with a global parameter version tracking module 48, which is included in server 30 of FIG. 1. As illustrated in system 200, the global parameter version tracking module 48 tracks versions (i.e., V0, V1, V2, V3, and V4) of the global parameter over time 47. Initially, both clients 11 and 12 receive an initial version (V0) 39 of the global parameter from the global parameter version tracking module 48. Client 12 determines a gradient 40 based on V0, and pushes the gradient 40 to the server 30. The server 30 determines a new version (V1) of the global parameter based on the gradient 40, and client 12 receives V1 41 from the global parameter version tracking module 48. Client 12 determines a gradient 42 based on V1 41, and pushes the gradient 42 to the server 30. The server 30 determines a new version (V2) of the global parameter based on the gradient 42, and client 12 receives V2 43 from the global parameter version tracking module 48. Client 12 determines a gradient 44 based on V2 43, and pushes the gradient 44 to the server 30” Wang demonstrates the continuous process of updating and transmitting the updated global parameter to one of the clients. The client 12 receives the initial global parameter, determines a gradient and pushes the gradient back to the global server, wherein the server determines a new version of the global parameter based on aggregating the received gradient to update its global model and then the global server continuously pushed the updated new version of the global parameter into the client 12 for further gradient determination and push back. The client 12 uses the newly pulled updated version of the global parameter but not the old parameter for local training and gradient determination, which is analogous to the process of replacing any prior weights for a prior version of the global ML model with the most recently updated primary weights for the most recently update within the claim.)
Wang teaches the limitation “utilize the most recently updated primary version of the global ML model in processing corresponding data obtained at the given computing device” (paragraph 25 “After further processing of the two models by the asynchronous federated learning system, one model can be selected as a best model by server 30 based on validation dataset 28 of FIG. 1. In some embodiments, an updated version of the global parameter (i.e., model) can be determined by the server 30” Wang discloses selecting the best model from the asynchronous federated learning system, wherein the best model has gone through the continuously process of updating the global model parameter and gradient determination as described above, such that this best model comprises of the most recent update of the global model parameter, which is analogous to the claimed most recently updated primary version of the global ML model in processing corresponding data obtained at the given computing device.)
Regarding claim 12 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches the limitation “The method of claim 1, wherein the computing devices of the population comprise client devices of a respective population of users” (paragraph 17 “In aspects of the invention, a computing system includes a server and one or more clients where each client has a local dataset and the server develops a global parameter.” Wang discloses the asynchronous federated learning comprises communication with one or more clients, which is analogous to the claimed client devices of a respective population of users.)
Regarding claim 13 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches the limitation “The method of claim 1, wherein the computing devices of the population comprise remote servers” (paragraph 61 “Computer system 600 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices” Wang discloses the computer system may be practiced in cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network, suggesting the global server as well as the clients may comprises of remote processing devices that are linked through a communications network to perform asynchronous federated learning.)
Regarding claim 14, the applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same rationale, because the claim recites similar limitations and processing steps.
Regarding claim 15 depends on claim 14, thus the rejection of claim 14 is incorporated. The
applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same
rationale, because the claim recites similar limitations and processing steps.
Regarding claim 16, the applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same rationale, because the claim recites similar limitations and processing steps.
Regarding claim 17, depends on claim 16, thus the rejection of claim 16 is incorporated. The applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same rationale, because the claim recites similar limitations and processing steps.
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et.al (US 20210342749 A1) in view of Yao et.al (NPL: LOCAL-GLOBAL KNOWLEDGE DISTILLATION IN HETEROGENEOUS FEDERATED LEARNING WITH NON-IID DATA), further in view of Parnell et.al (US 20210264320 A1)
Regarding claim 2 depends on claim 1, thus the rejection of claim 1 is incorporated.
Yao teaches a part of the limitation “The method of claim 1, wherein the corresponding historical version of the global ML model is one of a plurality of corresponding historical versions of the global ML model” (Page 4 section 3.1 “Figure 1: An overview of FEDGKD: an ensemble model wt is learned by aggregating information from historical global models.”, and Page 9 section 5.1 “The proposed method in Fig. 1, sends the ensemble of global models to participants and updates local weights via optimizing Eq. 4” Yao discloses multiple historical global models are generated and maintained on the server, and the ensemble model is learned by aggregating from multiple historical global models. One of ordinary skilled in the art would understand that Yao’s ensemble model corresponds to the plurality of corresponding historical versions of the global ML model within the claim. Yao also discloses the ensemble model is sent to participants (client) for local training and updating of local weights, which corresponds to the transmitting aspect within the claim.)
Wang/Yao does not teach the selection aspect of the limitation “wherein transmitting the corresponding historical version of the global ML model to the population of computing devices comprises: selecting, from among the plurality of corresponding historical versions of the global ML model, the corresponding historical version of the global ML model to transmit to each of the computing devices of the population”. However, Parnell teaches this part of the limitation (paragraph 20 “Initially, a uniform distribution of the base learners is assumed for the first iterations. Thus, base learners are first uniformly sampled at random at each of the first k iterations”, and paragraph 23 “That is, at each of said boosting iterations, a base learner is selected at random from candidate base learners, according to a sampling probability distribution of the base learners” Parnell teaches the selection of a base learner from a uniform distribution of the base learners at random, which is analogous to the selecting the corresponding historical version of the global ML model from among the plurality of corresponding historical versions of the global ML model within the claim.)
Before the effective filing date, it would have been obvious to one of ordinary skilled in the art to combine the teaching of the method and system of adaptive asynchronous federated learning by Wang, and using the past global model as the teacher model at the local model via adaptive knowledge distillation techniques by Yao, with the teaching of model selection based on a random uniform distribution by Parnell. The motivation to do so is referred to in Parnell’s disclosure (paragraph 1 “The present invention relates generally to the field of boosting methods for constructing ensemble models from a set of base learners, and more particularly to approaches relying on randomly selected base learners.”, and paragraph 14 “Embodiments of the present approach can achieve higher generalization accuracy (i.e., the resulting model can achieve a higher accuracy on new examples that were unseen during training) than existing algorithms in practice, for the following reasons: (i), embodiments of the present approach yield a less biased estimate of the gradient relative to algorithms relying on a single base learner; (ii) embodiments of the present approach lead to less over-fitting relative to algorithms that select base learners so as to minimize the mean squared error; and (iii) altering the random selection of the base learners provides additional stochasticity that allows the algorithm to converge to a wider local minimum” Parnell discloses methods for constructing ensemble models from a set of base learners and the benefits of the proposed method, which help achieve higher generalization accuracy as there is less bias relying on a single base learner, and less over-fitting relative to algorithms that select base learners. The proposed method by Parnell comprises the step of selecting a base learner according to a uniform and random distribution of base learners, thus a person ordinary skilled in the art would understand that the uniform and random distribution selection of models contribute to these improvements. Furthermore, one of ordinary skilled in the art would recognize that Parnell’s base learners correspond to Yao’s historical global models as both represent multiple candidate models from which a model can be selected for downstream training, and Wang discloses the downstream training which involve the transmission of parameters of the global model, the earlier version of the global model and a plurality of clients. Therefore, a person ordinary skilled in the art may further incorporate the teaching of model selection under a uniform and random distribution by Parnell into the training framework of Wang/Yao such that one or more earlier versions of the global model can be appropriately selected as the teacher model to configure the local model at each client thereby obtaining higher generalization accuracy as taught by Parnell.)
Regarding claim 3 depends on claim 2, thus the rejection of claim 2 is incorporated.
Parnell teaches the limitation “The method of claim 2, wherein selecting the corresponding historical version of the global ML model to transmit to each of the computing devices of the population and from among the plurality of corresponding historical versions of the global ML model is based on a uniform and random distribution of the plurality of corresponding historical versions of the global ML model” (paragraph 20 “Initially, a uniform distribution of the base learners is assumed for the first iterations. Thus, base learners are first uniformly sampled at random at each of the first k iterations”, and paragraph 23 “That is, at each of said boosting iterations, a base learner is selected at random from candidate base learners, according to a sampling probability distribution of the base learners” Parnell teaches the selection of a base learner from a uniform distribution of the base learners at random, which is analogous to the selecting the corresponding historical version of the global ML model from among the plurality of corresponding historical versions of the global ML model based on a uniform and random distribution within the claim.)
Regarding claim 4 depends on claim 2 thus the rejection of claim 2 is incorporated.
Wang teaches the limitation “The method of claim 2, wherein a first computing device, of the computing devices of the population, generates a first corresponding update, of the corresponding updates, via utilization of the primary version of the global ML model and via utilization of a first corresponding historical version of the global ML model, of the plurality of corresponding historical versions of the global ML model, and wherein a second computing device, of the computing devices of the population, generates a second corresponding update, of the corresponding updates, via utilization of the primary version of the global ML model and via utilization of a second corresponding historical version of the global ML model, of the plurality of corresponding historical versions of the global ML model” (paragraph 21 “FIG. 1 depicts an asynchronous federated learning system 100 in accordance with one or more embodiments of the present invention. System 100 can be implemented in conjunction with any appropriate computing device, such as computer system 600 of FIG. 6. The asynchronous federated learning system 100 includes a server 30 and plurality of clients 11, 12 and 13. Each of clients 11, 12 and 13 has a respective local dataset 20, 21 and 22. Embodiments of the server 30 determine and track multiple versions of a global parameter based on gradients 32, 33, and 34 from clients 11, 12 and 13.”, and paragraph 22 “the asynchronous federated learning system 100 causes the server 30 to update the current version of the global parameter to a new version of the global parameter by aggregating the received local gradient (e.g., any of local gradients 32, 33, and 34) with an appropriate version of the global parameter” Wang discloses a plurality of clients with computing devices corresponding to the claimed first and second computing devices of the population. Each client generates a gradient to be pushed to the global server for update via utilizing the pulled global parameter of the current version of the global ML model. Each client local model may be configured based on the one or more teacher historical global models as disclosed by Yao above, in which a historical global model may be selected as a teacher model based on the teaching of model selection by Parnell above.)
Claims 5, 10, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et.al (US 20210342749 A1) in view of Yao et.al (NPL: LOCAL-GLOBAL KNOWLEDGE DISTILLATION IN HETEROGENEOUS FEDERATED LEARNING WITH NON-IID DATA), further in view of Parnell et.al (US 20210264320 A1), further in view of Toporek et.al (US 20230316141 A1)
Regarding claim 5 depends on claim 2 thus the rejection of claim 2 is incorporated.
Wang teaches a part of the limitation “The method of claim 2, further comprising: subsequent to causing the corresponding historical weights for the corresponding historical version of the global ML model to be updated to generate the corresponding updated historical version of the global ML model for utilization in one or more of the subsequent rounds of decentralized learning for further updating of the global ML model:” (paragraph 22 “For example, based on client 11 pushing a gradient 32 to the server 30, the server 30 checks which version of global parameter the client 11 used to compute the received gradient 32. If there is another client (e.g., client 12) that has already pushed a gradient 33 to the server 30 that was computed based on a later version of the global parameter 31, ... Depending on the determined distance, the server 30 may aggregate the received gradient 32 with either the most recent version of the global parameter or an earlier version of the global parameter. ... If the determined distance is relatively large, the data distributions at client 11 and client 12 are different, such that the server 30 aggregates the received gradient 32 with an earlier version of the global parameter” Wang discloses the server may aggregate the received gradient 32 with an earlier version of the global parameter, thus Wang teaches updating an earlier version of the global parameter based on a later-received gradient, which corresponds to updating the historical version of the global ML model using the given corresponding update within the claim. By aggregating the later-received gradient with an earlier version of the global parameter, the server effectively produces an updated version of the earlier (historical) model state for continued use. This behavior aligns with the claimed process of using the received update to generate updated historical weights of the global ML model and enabling that updated historical version to be utilized in one or more subsequent rounds of learning.)
Wang/Yao/Parnell does not teach the limitation “purging an oldest corresponding historical version of the global ML model from the plurality of corresponding historical versions of the global ML model”. However, Toporek teaches this limitation (paragraph 87 “In certain embodiments, the global server 110 may have a predetermined performance threshold, for example, a performance metric such as reliability of prediction of a previous version of the global model 120 prior to updating in block 320. In case where the global model 120 does not perform as well as the previous version of the global model 120, the global server 110 can reinstate the previous version of the global model 120 and terminate a current learning cycle.” Toporek discloses the termination of a current learning cycle comprising the current global model if the global model does not perform well according to a predetermined performance threshold. Toporek explicitly discloses termination of a version of the machine learning model that does not perform well, wherein a person ordinary skilled in the art would understand that the termination may be performed if the earlier version of the global model exhibits poor performance. In model version management, discarding or terminating an inferior or outdated version inherently serves as a purge operation, Thus, Toporek teaches a mechanism by which a version of the global model is removed from the maintained set of historical versions based on their performance, which corresponds to purging an oldest corresponding historical version, as claimed.)
Before the effective filing date, it would have been obvious to one of ordinary skilled in the art to combine the teaching of the method and system of adaptive asynchronous federated learning by Wang, the teaching of using the past global model as the teacher model at the local model via adaptive knowledge distillation techniques by Yao, and the teaching of model selection based on a random uniform distribution by Parnell, with the teaching of using performance threshold to terminate poor performance version of the model by Toporek. The motivation to do so is referred to in Toporek’s disclosure (paragraph 6 “ascertaining that the global machine-learning model as being updated needs to be validated prior to a deployment to the local participants in the federated learning system; and validating that the global machine-learning model as being updated performs better than a version of the global machine-learning model prior to being updated, by use of a validation dataset stored in a global server from which the global machine-learning model is trained.”, and paragraph 87 “the global server 110 may have a predetermined performance threshold, for example, a performance metric such as reliability of prediction of a previous version of the global model 120 prior to updating in block 320. In case where the global model 120 does not perform as well as the previous version of the global model 120, the global server 110 can reinstate the previous version of the global model 120 and terminate a current learning cycle” Toporek discloses a federate learning method and system, which also comprises a global machine learning model and several local participants similar to the teaching by Wang. Toporek further discloses validation of the performance of the global model by using a validation dataset and compare to a performance threshold to determine if the updated global model perform better than its prior version, thus obtain an improved global machine learning model and terminate the lesser performance version. Therefore, the teaching by Wang can also incorporate the technique of performance validation and termination by Toporek to obtain an improved version of the global ML model before continually perform the federated learning.)
Regarding claim 10 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang/Yao does not teach the limitation “The method of claim 1, wherein the one or more deployment criteria comprise one or more of: a threshold quantity of rounds of decentralized learning for updating of the global ML model being performed, or a threshold performance measure of the most recently updated primary version of the global ML model being achieved”. However, Toporek teaches this limitation (paragraph 87 “In certain embodiments, the global server 110 may have a predetermined performance threshold, for example, a performance metric such as reliability of prediction of a previous version of the global model 120 prior to updating in block 320. In case where the global model 120 does not perform as well as the previous version of the global model 120, the global server 110 can reinstate the previous version of the global model 120 and terminate a current learning cycle” Toporek discloses the global server, which corresponds to the global model by Wang, may have a predetermined performance threshold, which corresponds to the threshold performance measure of the most recently updated primary version of the global ML model being achieved, as claimed.)
Before the effective filing date, it would have been obvious to one of ordinary skilled in the art to combine the teaching of the method and system of adaptive asynchronous federated learning by Wang, the teaching of using the past global model as the teacher model at the local model via adaptive knowledge distillation techniques by Yao with the teaching of performance threshold by Toporek. The motivation to do so is referred to in Toporek’s disclosure (paragraph 6 “ascertaining that the global machine-learning model as being updated needs to be validated prior to a deployment to the local participants in the federated learning system; and validating that the global machine-learning model as being updated performs better than a version of the global machine-learning model prior to being updated, by use of a validation dataset stored in a global server from which the global machine-learning model is trained.”, and paragraph 87 “the global server 110 may have a predetermined performance threshold, for example, a performance metric such as reliability of prediction of a previous version of the global model 120 prior to updating in block 320. In case where the global model 120 does not perform as well as the previous version of the global model 120, the global server 110 can reinstate the previous version of the global model 120 and terminate a current learning cycle” Toporek discloses using the performance threshold to indicate the best version of the model, thus the best version of the model can finally be obtained. One of ordinary skilled in the art would have been able to incorporate the technique of threshold comparison into the global model at the global server by Wang to validate the performance of each time the global model is updated to obtain the best version of the global model.)
Regarding claim 18 depends on claim 17, thus the rejection of claim 17 is incorporated. The applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same rationale, because the claim recites similar limitations and processing steps.
Regarding claim 19 depends on claim 16, thus the rejection of claim 16 is incorporated. The applicant is directed to the rejections to claim 1 set forth above, as they are rejected based on the same rationale, because the claim recites similar limitations and processing steps.
Regarding claim 20 depends on claim 19, thus the rejection of claim 19 is incorporated
Toporek teaches the limitation “The method of claim 19, wherein the one or more update criteria comprises one or more of: a threshold quantity of the corresponding updates being received from the one or more of the computing devices of the population and during the given round of decentralized learning for updating of the global ML model, or a threshold duration of time lapsing prior to conclusion of the given round of decentralized learning for updating of the global ML model” (paragraph 70 “The global server 110 can be configured to perform a learning cycle as presented herein in a certain preconfigured interval, upon receiving a certain number of update matrices from all local participants, upon receiving a certain number of update matrices from a selected group of local participants based on a preconfigured condition, upon receiving a certain number of update matrices that represents changes outside of threshold ranges, or any combinations thereof.” Toporek discloses one of the condition to be configured at the global server to perform a learning cycle comprises of a condition of receiving a certain number of update matrices from all local participant, which is analogous to the update criteria of a threshold quantity of the corresponding updates being received from the one or more of the computing devices within the claim.)
Claims 6, 7 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et.al (US 20210342749 A1) in view of Yao et.al (NPL: LOCAL-GLOBAL KNOWLEDGE DISTILLATION IN HETEROGENEOUS FEDERATED LEARNING WITH NON-IID DATA), further in view of Jandial et.al (US 20240062057 A1)
Regarding claim 6 depends on claim 1, thus the rejection of claim 1 is incorporated.
Wang teaches the limitation “process, using the primary version of the global ML model, corresponding data obtained by the given computing device to generate one or more predicted outputs” (paragraph 3 “Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events.” Wang discloses machine learning models are built to perform prediction of future events, wherein the machine learning models may be the updated global machine learning model as disclosed by Wang, which can be utilized to perform prediction of future events as understood by one of ordinary skilled in the art.)
Wang teaches the update aspect of the limitation “generate, based on at least the one or more predicted outputs and based on the distillation regularization term, a given corresponding update for the primary version of the global ML model”. (paragraph 22 “Whenever a gradient (e.g., any of local gradients 32, 33, and 34) is received by the server 30, the asynchronous federated learning system 100 causes the server 30 to update the current version of the global parameter to a new version of the global parameter by aggregating the received local gradient (e.g., any of local gradients 32, 33, and 34) with an appropriate version of the global parameter.” Wang discloses the update to the global model of the global server upon receive the local gradient from one of the clients, wherein the gradient is computed by one of the clients. One of ordinary skilled in art would recognize that the client computes the gradient by calculating a gradient of the loss function with respect to the local model's parameters. This necessarily requires the client to obtain predicted outputs from the local model and to compute a loss term. Accordingly, Wang teaches the update mechanism, while the specific component of the loss function – such as the distillation regularization term and the predicted outputs as recited in the claim are addressed by additional teachings by Jandial below. The motivation to combine the teachings is below.)
Wang teaches the limitation “transmit, to the remote system, the given corresponding update for the primary version of the global ML model” (paragraph 22 “the asynchronous federated learning system 100 causes the server 30 to update the current version of the global parameter to a new version of the global parameter by aggregating the received local gradient (e.g., any of local gradients 32, 33, and 34) with an appropriate version of the global parameter”, and paragraph 61 “Computer system 600 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network.” Wang discloses the transmitting of gradient from the clients to the global server, which can be represented as remote processing devices that are linked through a communications network, to update the global parameter at the global machine learning model at the server.)
Wang/Yao does not teach the limitation “process, using the corresponding historical version of the global ML model, the corresponding data obtained by the given computing device to determine a distillation regularization term”. However, Jandial teaches this limitation (paragraph 14 “the retrospective distillation learning system combines the past-state outputs of the student machine learning model with the outputs of the teacher machine learning model to determine a combined student-regularized teacher outputs that regularizes training targets by making the training targets similar to the student outputs while preserving semantics from the teacher training targets. Indeed, in some instances, the retrospective distillation learning system determines a retrospective knowledge distillation loss using a comparison of the outputs of the student machine learning model (from the present training step) and the combined student-regularized teacher output.” Jandial discloses a distillation learning system combines the past-state outputs of the student machine learning model with the outputs of the teacher machine learning model to determine a combined student-regularized teacher outputs that regularizes training targets by making the training targets similar to the student outputs while preserving semantics from the teacher training targets. This regularization is analogous to the claimed distillation regularization term. The teacher model corresponds to the historical version of the global ML model, and the outputs of the student model corresponds to the corresponding data obtained by the given computing device, since the student model may be configured as the local model at each client device according to the teaching by Wang/Yao, and the motivation to combine the teachings is below.)
Before the effective filing date, it would have been obvious to one of ordinary skilled in the art to combine the teaching of the method and system of adaptive asynchronous federated learning by Wang, the teaching of using the past global model as the teacher model at the local model via adaptive knowledge distillation techniques by Yao with the teaching of the retrospective distillation learning system determines a retrospective knowledge distillation loss as the regularization term by Jandial. The motivation to do so is referred to in Jandial’s disclosure (paragraph 3 “In this manner, the disclosed systems improve the accuracy of student machine learning models during knowledge distillation through already existing data from the student machine learning models while utilizing less computational resources (e.g., without utilizing additional external information and/or without utilizing intermediate machine learning models to train the student machine learning models)”, paragraph 19 “Accordingly, in one or more cases, the retrospective knowledge distillation learning system improves the accuracy of knowledge distillation while reducing the utilization of intermediate training models or excessively training and/or modifying the teacher network during knowledge distillation”, and paragraph 21 “in some embodiments, the retrospective knowledge distillation learning system also increases flexibility during teacher-to-student network knowledge distillation. For instance, unlike conventional systems that require intermediate models and/or access into a teacher network to perform knowledge distillation, the retrospective distillation learning system utilizes internally available student network states and already-converged teacher network outputs.” Jandial discloses the benefit of the retrospective knowledge distillation learning system and method, which by leveraging past state outputs of a student machine learning model and teacher model, a regularization term with regard to the distillation loss can be obtained, which help improve the accuracy of student machine learning models during knowledge distillation while utilizing less computational resources, reducing the utilization of intermediate training models or excessively training and/or modifying the teacher network, and increases flexibility during teacher-to-student network knowledge distillation. Given that the teaching by Wang/Yao discloses a framework of federated learning with the global server transmit the data to the client, wherein the historical version of the global model of the global server is employed as the teacher model to help configure the local (student) model at each client, one of ordinary skilled in the art may further incorporate the teaching by Jandial into the teaching by Wang/Yao to obtain the distillation loss as the regularization term in training at the local (student) model at each device, thereby improve the training at each client device, which inherently improve the training of the global model as well.)
Regarding claim 7 depends on claim 6, thus the rejection of claim 6 is incorporated.
Jandial teaches the limitation “The method of claim 6, wherein the distillation regularization term is determined based on one or more labels generated from processing the corresponding data obtained by the given computing device and using the corresponding historical version of the global ML model” (paragraph 14 “the retrospective distillation learning system combines the past-state outputs of the student machine learning model with the outputs of the teacher machine learning model to determine a combined student-regularized teacher outputs that regularizes training targets by making the training targets similar to the student outputs while preserving semantics from the teacher training targets. Indeed, in some instances, the retrospective distillation learning system determines a retrospective knowledge distillation loss using a comparison of the outputs of the student machine learning model (from the present training step) and the combined student-regularized teacher output.”, and paragraph 37 “In some embodiments, an input task includes corresponding ground truth data that indicates a known or desired prediction, label, or outcome for the input task” Jandial discloses the retrospective knowledge distillation learning system that generate the distillation loss as the regularization term. The distillation loss is generated based on the output of the student (local) model, which can be a label output that corresponds to the data obtained by the given computing device, and the distillation loss is also generated based on the output of the teacher model, wherein the teacher model is analogous to the corresponding historical version of the global ML model according to the teaching by Wang/Yao above.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUY TU DIEP whose telephone number is (703)756-1738. The examiner can normally be reached M-F 8-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY T DIEP/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights