DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This communication is in response to the Applicant’s submission filed 31 August 2023, where:
Claims 1-28 are pending.
Claims 1-28 are rejected.
Information Disclosure Statement
3. An information disclosure statement was submitted on 22 February 2024. The submission complies with the provisions of 37 CFR 1.97. Accordingly, the Examiner considered the information disclosure statement.
Claim Rejections - 35 U.S.C. § 101
4. 35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1-28 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a computer-implemented method, which is a process, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “while training the first DP model, generating a plurality of intermediate checkpoints,” [(c)] determining an aggregate of the first DP model and the plurality of intermediate checkpoints,” and “[(d)] determining, using the aggregate, a second DP model, the second DP model satisfying the same differential privacy budget.” These activities of “generating” and “determining” contain limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
The claim also recites more details or specifics of the abstract idea of “generating a plurality of intermediate checkpoints,” where “[(b.1)] each intermediate checkpoint of the plurality of intermediate checkpoints representing a different intermediate state of the first DP model, and “[(b.2)] each of the intermediate checkpoints satisfying the same differential privacy budget,” and accordingly, are merely more specific to the abstract idea. Thus, the claim recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “data processing hardware,” which is recited at a high-level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites a “first differentially private (DP) model” and a “second DP model,” which are also recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application.
The claim also recites “[(a)] training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples,” which is the activity of using a generic computer component (data processing hardware, first DP model) to implement the abstract idea, (MPEP § 2106.05(f)),and accordingly, does not serve to integrate the abstract idea into a practical application. The claim recites more details or specifics to the additional element of “training,” where “[(a.1)] the first DP model satisfying a differential privacy budget,” and “[(a.2)] the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model,” and accordingly, are merely more specific to the additional element. Therefore, the claim is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements recited in the claim beyond the identified judicial exception include “data processing hardware,” which is recited at a high-level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites a “first differentially private (DP) model” and a “second DP model,” which are also recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea.
The claim also recites “[(a)] training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples,” which is the activity of using a generic computer component (data processing hardware, first DP model) to implement the abstract idea, (MPEP § 2106.05(f)),and accordingly, does not serve to integrate the abstract idea into a practical application. The claim recites more details or specifics to the additional element of “training,” where “[(a.1)] the first DP model satisfying a differential privacy budget,” and “[(a.2)] the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model,” and accordingly, are merely more specific to the additional element. Therefore, claim 1 is subject-matter ineligible.
Claim 15 recites a system, which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “while training the first DP model, generating a plurality of intermediate checkpoints,” [(c)] determining an aggregate of the first DP model and the plurality of intermediate checkpoints,” and “[(d)] determining, using the aggregate, a second DP model, the second DP model satisfying the same differential privacy budget.” These activities of “generating” and “determining” contain limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
The claim also recites more details or specifics of the abstract idea of “generating a plurality of intermediate checkpoints,” where “[(b.1)] each intermediate checkpoint of the plurality of intermediate checkpoints representing a different intermediate state of the first DP model, and “[(b.2)] each of the intermediate checkpoints satisfying the same differential privacy budget,” and accordingly, are merely more specific to the abstract idea. Thus, the claim recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “data processing hardware” and “memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations,” which are recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites a “first differentially private (DP) model” and a “second DP model,” which are also recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application.
The claim also recites “[(a)] training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples,” which is the activity of using a generic computer component (data processing hardware, memory hardware, first DP model) to implement the abstract idea, (MPEP § 2106.05(f)),and accordingly, do not serve to integrate the abstract idea into a practical application. The claim recites more details or specifics to the additional element of “training,” where “[(a.1)] the first DP model satisfying a differential privacy budget,” and “[(a.2)] the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model,” and accordingly, are merely more specific to the additional element. Therefore, the claim is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements recited in the claim beyond the identified judicial exception include “data processing hardware” and “memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations,” which are recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea. The claim also recites a “first differentially private (DP) model” and a “second DP model,” which are also recited at a high-level of generality, and accordingly, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea.
The claim also recites “[(a)] training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples,” which is the activity of using a generic computer component (data processing hardware, first DP model) to implement the abstract idea, (MPEP § 2106.05(f)),and accordingly, does not serve to integrate the abstract idea into a practical application. The claim recites more details or specifics to the additional element of “training,” where “[(a.1)] the first DP model satisfying a differential privacy budget,” and “[(a.2)] the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model,” and accordingly, are merely more specific to the additional element. Therefore, claim 15 is subject-matter ineligible.
Claims 2, 7, and 8 depend directly or indirectly from claim 1. Claims 16, 21, and 22 depend directly or indirectly from claim 15. The claims recite more details or specifics to the abstract idea of “[(c)] determining the aggregate of the first DP model and the plurality of intermediate checkpoints,” where (claims 2 and 16: [(c.1)] determining aggregate parameter values based on parameter values of the first DP model and parameter values of the plurality of intermediate checkpoints,” and “[(c.2)] determining, using the aggregate, the second DP model comprises using the aggregate parameter values as parameter values of the second DP model”; claims 7 and 21: “[(c.1)] determining a combination of the first DP model and the plurality of intermediate checkpoints,” and “[(c.2)] the second DP model comprises the determined combination”; claims 8 and 22: “[(c.3)] selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints” and “[(c.4)] determining the combination to include the first DP model and the selected subset of the intermediate checkpoints”), and accordingly, are merely more specific to the abstract idea. Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea. Therefore, claims 2, 7, 8, 16, 21, and 22 are subject-matter ineligible.
Claims 3 and 4 depend directly or indirectly from claim 1. Claims 17 and 18 depend directly or indirectly from claim 15. The claims recite more details or specifics of “determining the aggregate parameter values,” where (claims 3 and 17: [(c.1.1)] determining a weighted sum of the parameter values of the first DP model and the parameter values of the plurality of intermediate checkpoints”; claims 4 and 18: “[(c.1.1)] selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints,” and “[(c.1.2)] averaging the parameter values of the first DP model and the parameter values of the subset of intermediate checkpoints”), and accordingly, are merely more specific to the abstract idea. Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea. Therefore, claims 3, 4, 17, and 18 are subject-matter ineligible.
Claims 5, 6, 9 and 10 depend directly or indirectly from claim 1. Claims 19, 20, 23 and 24 depend directly or indirectly from claim 15. The claims recite more details or specifics of the abstract idea of “[(c)] determining the aggregate,” where (claims 5 and 19: “[(c.1.1.1)] a threshold number of latest intermediate checkpoints.”; claims 6 and 20: “[(c.1.1.1)] determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor”, and “[(c.1.1.2)] selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor”; claims 9 and 23: “[(c.3.1)] a threshold number of latest intermediate checkpoints”; claims 10 and 24: “[(c.3.2)] determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor,” and “[(c.3.3)] selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor,”) and accordingly, are merely more specific to the abstract idea. Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea. Therefore, claims 5, 6, 9, 10, 19, 20, 23, and 24 are subject-matter ineligible.
Claim 11 depends directly or indirectly from claim 1. Claim 25 depends directly or indirectly from claim 15. The claims recite further limitations of “[(e)] determining outputs of the first DP model,” “[(f)] determining a plurality of outputs for respective ones of the plurality of intermediate checkpoints,” and “[(g)] determining outputs of the second DP model comprising an aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints.” The activities of “determining” contain limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)). Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea. Therefore, claims 11 and 25 are subject-matter ineligible.
Claims 12 and 13 depend directly or indirectly from claim 1. Claims 26 and 27 depend directly or indirectly from claim 15. The claims recite more details or specifics of “determining outputs of the second DP model comprising an aggregate,” (claims 12 and 26: “[(g.1)] a majority vote based on the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints”; claims 13 and 27: “[(g.1)] an average of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints.”), and accordingly, are merely more specific to the abstract idea. Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea.
Claim 14 depends directly or indirectly from claim 1. Claim 28 depends directly or indirectly from claim 15. The claim recites further limitations of “[(e)] predicting, using the second DP model, an output;” and “[(f)] determining, using at least one of the plurality of intermediate checkpoints, an uncertainty of the predicted output.” The activities of “predicting” and “determining” contain limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)). Also, the additional elements of the claim does not serve to integrate the abstract idea into integrated into a practical application, (see MPEP § 2106.04(d)), nor do the additional elements amount to significantly more than the abstract idea, (MPEP § 2106.05 sub I; see also MPEP § 2106.05(a) – (h)), and thus, the claim recites no more than the abstract idea. Therefore, claims 14 and 28 are subject-matter ineligible.
Claim Rejections - 35 U.S.C. § 102
6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
7. Claims 1, 2, 4, 5, 7-9, 11, 13, 15, 16, 18, 19, 21-23, 25, 27, and 28 are rejected under 35 U.S.C. § 102(a)(2) as being anticipated by US Published Application 20230394374 to Marathe et al. [hereinafter Marathe].
Regarding claims 1 and 15, Marathe teaches [a] computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations (Marathe ¶ 0045 teaches “various exemplary flowcharts illustrating methods and techniques, which may be implemented by these machine learning systems or other systems or applications are discussed. Finally, an example computing system [(that is, data processing hardware)] is discussed upon which various embodiments may be implemented is discussed [(that is, a computer-implemented method)]”) of claim 1, and [a] system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (Marathe ¶ 0069 teaches “one or more processors 1070 [(that is, data processing hardware)], the storage device(s) 1050, and the system memory 1010 may be coupled to the system interconnect 1040 [(that is, memory hardware in communication with the data processing hardware)]. One or more of the system memories 1010 may contain program instructions 1020. Program instructions 1020 may be executable to implement various features described above, including a machine learning model training system 1022”) comprising:
training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples (Marathe, Fig. 2, teaches a “user-entity differential privacy system generating a natural language model that provides user-entity differential privacy [Examiner annotations in dashed-line text boxes]:”
PNG
media_image1.png
624
814
media_image1.png
Greyscale
Marathe ¶ 0047 teaches “individual ones of the federated model user systems 220, 230 and 240 [(that is, a first differentially private (DP) model)], may independently generate locally updated versions of the machine learning models 222, 232, and 242 by training the model using local, training data sets 224, 234, and 244 [(that is, training a first differentially private (DP) model using a private training set, the private training set comprising a plurality of training samples)]”; Marathe ¶ 0047 teaches that “[i]ndividual ones of the federated model user systems 220, 230, and 240 may independently alter, by clipping and applying noise, to their local model parameter updates to generate modified model parameter updates, where the altering provides or ensures privacy of their local training data sets 224, 234, and 244 [(that is, the private training set comprising a plurality of training samples)]”; Marathe ¶ 0026 teaches “[d]ifferential privacy may be described differently in other scenarios, such as federated learning. Let U be the set of n users participating in a federation, and Di, be the data set of user ui ⋲ U. Let Du =Ui=1n Di. Let ℳ be the domain of models resulting from the federated learning training process. Given a federated learning training a F:D→ ℳ, F is a user level (ε, δ) differentially private if for any two adjacent user sets U . . . .”),
the first DP model satisfying a differential privacy budget (Marathe ¶¶ 0035-36 teaches “[o]ne consideration for enforcing subject level differential privacy is that to guarantee subject level differential privacy [(that is, to “guarantee” is satisfying)], a training algorithm may have to obfuscate the entire contribution made by any subject in the model's parameter updates. . . . [T]he noise added to the averaged gradients may be Gaussian noise. The Gaussian noise scale σ is calculated independently at each user ui using standard parameters, the privacy budget ε, the failure probability δ, total number of mini-batches T. R. and the sampling fraction per mini-batch
PNG
media_image2.png
77
64
media_image2.png
Greyscale
The calculation may use the moments accountant method to compute σ [(that is, the first DP model satisfying a differential privacy budget)]”),
the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model (Marathe ¶ 0043 teaches “sequential composition of privacy loss across federation users [(that is, the “users” being the first DP model)] may be referred to as ‘horizontal composition.’ Horizontal composition may have a significant effect on the number of federated training rounds permitted under a given privacy loss budget [(that is, “training rounds” pertain to the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model)]”);
while training the first DP model, generating a plurality of intermediate checkpoints, each intermediate checkpoint of the plurality of intermediate checkpoints representing a different intermediate state of the first DP model, each of the intermediate checkpoints satisfying the same differential privacy budget (Marathe ¶¶ 0043-44 teaches a “sequential composition of privacy loss across federation users may be referred to as ‘horizontal composition’ [(that is, such “sequential composition of privacy loss” is while training the first DP model, generating a plurality of intermediate checkpoints)]. Horizontal composition may have a significant effect on the number of federated training rounds permitted under a given privacy loss budget. Consider a federated learning training algorithm F = (Fl, Fg) that samples s users per training round, and trains the model ℳ for R rounds [(that is, “R rounds” is each intermediate checkpoint of the plurality of intermediate checkpoints representing a different intermediate state of the first DP model)]. Let Fl at each participating user, over the aggregate of R training rounds, locally enforce subject-level (ε, δ) differential privacy [(that is, “locally enforce” is each of the intermediate checkpoints satisfying the same differential privacy budget)]. Then F globally enforces the same subject-level (ε, δ) differential privacy guarantee by
PNG
media_image3.png
75
74
media_image3.png
Greyscale
training for rounds. The s-way horizontal composition via Fg results in an increase in training mini-batches by a factor of s. As a result, the privacy loss calculated by the moments accountant method [(that is, “the privacy loss calculated by the moments accountant method” is a plurality of intermediate checkpoints)] amplifies by a factor of
s
, thereby forcing a reduction in number of training rounds by a factor of
s
to counteract the inflation of privacy loss. This reduction in training rounds can have a significant impact on the resulting model's performance”);
determining an aggregate of the first DP model and the plurality of intermediate checkpoints (Marathe ¶¶ 0043-44 teaches “[l]et Fl at each participating user, over the aggregate of R training rounds (that is, determining an aggregate of . . . the plurality of intermediate checkpoints)], locally enforce subject-level (ε, δ) differential privacy. Then F globally enforces the same subject-level (ε, δ) differential privacy guarantee by
PNG
media_image3.png
75
74
media_image3.png
Greyscale
training for rounds. The s-way horizontal composition via Fg results in an increase in training mini-batches by a factor of s. As a result, the privacy loss calculated by the moments accountant method amplifies by a factor of
s
, thereby forcing a reduction in number of training rounds by a factor of
s
to counteract the inflation of privacy loss. This reduction in training rounds can have a significant impact on the resulting model's performance”; Marathe, Fig. 2, teaches aggregated model parameter updates [Examiner annotations in dashed-line text boxes]:
PNG
media_image4.png
932
1197
media_image4.png
Greyscale
Marathe ¶ 0049 teaches “[u]pon receipt of the collective modified model parameter updates, the federation server 210 may then aggregate the respective modified model parameter updates to generate aggregated model parameter updates 214 [(that is, determining an aggregate of the first DP model and the plurality of intermediate checkpoints)]”); and
determining, using the aggregate, a second DP model, the second DP model satisfying the same differential privacy budget (see above, Marathe, Fig. 2, that teaches a federated machine learning model 212; Marathe ¶ 0049 teaches “[t]he federation server 210 may then apply the aggregated model parameter updates 214 to the current version of the federated machine learning model 212 to generate a new version of the model 212. This process may be repeated a number of times until the model 212 converges or until a predetermined threshold number of iterations is met [(that is, determining, using the aggregate, a second DP model)]”; Marathe ¶ 0044 teaches “[l]et Fl at each participating user, over the aggregate of R training rounds, locally enforce subject-level (ε, δ) differential privacy. Then F globally enforces the same subject-level (ε, δ) differential privacy guarantee by
PNG
media_image3.png
75
74
media_image3.png
Greyscale
training for rounds [(that is, “F-globally enforces” is the second DP model satisfying the same differential privacy budget)]”).
Regarding claims 2 and 16, Marathe teaches all of the limitations of claims 1 and 15, respectively, as described above in detail.
Marathe teaches -
wherein:
determining the aggregate of the first DP model and the plurality of intermediate checkpoints comprises determining aggregate parameter values based on parameter values of the first DP model and parameter values of the plurality of intermediate checkpoints (Marathe ¶ 0030 teaches “subject level differential privacy may be enforced locally at each user. But to prove the privacy guarantee for any subject across the entire federation [(that is, the first DP model)], the federation server may ensure that the local subject level differential privacy guarantee composes correctly through global aggregation of parameter updates [(that is, “parameter updates” is parameter values of the plurality of intermediate checkpoints)] received from the users”); and
determining, using the aggregate, the second DP model comprises using the aggregate parameter values as parameter values of the second DP model (Marathe ¶ 0031 Fig. 2 teaches “federation server techniques may include the federation server [(that is, “federated machine learning model 212” is the second DP model)] sampling a random set of users for each training round and sending them a request to perform local training. . . . The gradients may then be summed over the full mini-batch [(that is, “summed” is using the aggregate)], and noise scaled to C is added to the sum. This sum may then be averaged over the mini-batch size, and applied to the parameters [(that is, determining, using the aggregate, the second DP model comprises using the aggregate parameter values as parameter values of the second DP model)]”).
Regarding claims 4 and 18, Marathe teaches all of the limitations of claims 2 and 16, respectively, as described above in detail.
Marathe teaches -
wherein determining the aggregate parameter values comprises:
selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints (Marathe ¶ 0044 teaches a “s-way horizontal composition via Fg results in an increase in training mini-batches by a factor of s. As a result, the privacy loss calculated by the moments accountant method [(that is, intermediate checkpoints from the plurality of intermediate checkpoints)] amplifies by a factor of
s
”; Marathe ¶ 0058 teaches “a sample of data items from the data set may be identified, in some embodiments. For example, various different random sampling techniques (e.g., using random number generation) may be implemented to select the sample of data items [(that is, selecting a subset)]. The sample of data items may be less than the entire number of data items from the data set, in some embodiments [(that is, selecting a subset)]. In this way, different samples taken for different iterations of the technique performed in a training round (e.g., for different mini-batches) may likely have at least some data items that are different from a prior sample [(that is, selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints)]”;
[Examiner notes that the broadest reasonable interpretation of “selecting a subset of intermediate checkpoints” covers the selection of the underlying mini-batches also data items that may be less than the entire number of data items as taught by Marathe, which is not inconsistent with the Applicant’s disclosure. (MPEP § 2111))]; and
averaging the parameter values of the first DP model and the parameter values of the subset of intermediate checkpoints (Marathe ¶ 0035 teaches “[c]lipping and then averaging gradients may ensure that the entire subject's gradient contribution is bounded by [gradient norm bound] C. Subsequently, the technique may then sum all the per-subject averaged gradients [(that is, parameter values)] along with the noise scaled to clipping threshold C, which are then averaged over the mini-batch size B [(that is, averaging the parameter values of the first DP model and the parameter values of the subset of intermediate checkpoints)]”).
Regarding claims 5 and 19, Marathe teaches all of the limitations of claims 4 and 18, respectively, as described above in detail.
Marathe teaches -
wherein the subset of intermediate checkpoints comprises
a threshold number of latest intermediate checkpoints (Marathe ¶ 0037 teaches “For every sampled mini-batch S in a samples user ui's training round, the subject sensitivity 𝕊s for S is bounded by [gradient norm bound] C (e.g., 𝕊s <IC|) [(that is, “gradient norm bound C” is a threshold number of latest intermediate checkpoints)]”).
Regarding claims 7 and 21, Marathe teaches all of the limitations of claims 1 and 15, respectively, as described above in detail.
Marathe teaches -
wherein:
determining the aggregate of the first DP model and the plurality of intermediate checkpoints comprises
determining a combination of the first DP model and the plurality of intermediate checkpoints (Marathe ¶ 0031 teaches “federation server techniques may include the federation server sampling a random set of users for each training round [(that is, “sampling a random set of users” is determining the combination)] and sending them a request to perform local training. Each federated user may train for several mini-batches, even multiple epochs, and introduce noise (e.g., Gaussian noise in parameter gradients computed for each mini-batch). For each mini-batch, gradients are computed for each data item separately, and clipped to the threshold C to bound the gradients' sensitivity (e.g., maximum influence of any data item on the computed gradients). The gradients may then be summed over the full mini-batch, and noise scaled to C is added to the sum. This sum may then be averaged over the mini-batch size, and applied to the parameters [(that is, determining the combination to include the first DP model and the plurality of intermediate checkpoints)]”); and
the second DP model comprises the determined combination (Marathe ¶ 0049 teaches “The federation server 210 may then apply the aggregated model parameter updates 214 to the current version of the federated machine learning model 212 to generate a new version of the model 212 [(that is, the second DP model comprises the determined combination)]”).
Regarding claims 8 and 22, Marathe teaches all of the limitations of claims 7 and 21, respectively, as described above in detail.
Marathe teaches -
wherein determining the aggregate of the first DP model and the plurality of intermediate checkpoints comprises:
selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints (Marathe ¶ 0044 teaches a “s-way horizontal composition via Fg results in an increase in training mini-batches by a factor of s. As a result, the privacy loss calculated by the moments accountant method [(that is, intermediate checkpoints from the plurality of intermediate checkpoints)] amplifies by a factor of
s
”; Marathe ¶ 0058 teaches “a sample of data items from the data set may be identified, in some embodiments. For example, various different random sampling techniques (e.g., using random number generation) may be implemented to select the sample of data items [(that is, selecting a subset)]. The sample of data items may be less than the entire number of data items from the data set, in some embodiments [(that is, selecting a subset)]. In this way, different samples taken for different iterations of the technique performed in a training round (e.g., for different mini-batches) may likely have at least some data items that are different from a prior sample [(that is, selecting a subset of intermediate checkpoints from the plurality of intermediate checkpoints)]”;
[Examiner notes that the broadest reasonable interpretation of “selecting a subset of intermediate checkpoints” covers the selection of the underlying mini-batches also data items that may be less than the entire number of data items as taught by Marathe, which is not inconsistent with the Applicant’s disclosure. (MPEP § 2111))]; and
determining the combination to include the first DP model and the selected subset of the intermediate checkpoints (Marathe ¶ 0031 teaches “federation server techniques may include the federation server sampling a random set of users for each training round [(that is, “sampling a random set of users” is determining the combination)] and sending them a request to perform local training. Each federated user may train for several mini-batches, even multiple epochs, and introduce noise (e.g., Gaussian noise in parameter gradients computed for each mini-batch). For each mini-batch, gradients are computed for each data item separately, and clipped to the threshold C to bound the gradients' sensitivity (e.g., maximum influence of any data item on the computed gradients). The gradients may then be summed over the full mini-batch, and noise scaled to C is added to the sum. This sum may then be averaged over the mini-batch size, and applied to the parameters [(that is, determining the combination to include the first DP model and the selected subset of the intermediate checkpoints)]”).
Regarding claims 9 and 23, Marathe teaches all of the limitations of claims 8 and 22, respectively, as described above in detail.
Marathe teaches -
wherein the subset of intermediate checkpoints comprises
a threshold number of latest intermediate checkpoints (Marathe ¶ 0037 teaches “[f]or every sampled mini-batch S in a samples user ui's training round, the subject sensitivity 𝕊s for S is bounded by [gradient norm bound] C (e.g., 𝕊s <IC|) [(that is, “gradient norm bound C” is a threshold number of latest intermediate checkpoints)]”).
Regarding claims 11 and 25, Marathe teaches all of the limitations of claims 7 and 15, respectively, as set out above in detail.
wherein the operations further comprise:
determining outputs of the first DP model (Marathe ¶ 0048 teaches “This independently performed training may then generate model parameter updates that provide respective model contributions 223, 233, and 243 [(that is, the cumulative contributions is determining outputs of the first DP model)] to federation server 210”);
determining a plurality of outputs for respective ones of the plurality of intermediate checkpoints (Marathe ¶ 0031 teaches “[e]ach federated user may train for several mini-batches, even multiple epochs, and introduce noise (e.g., Gaussian noise in parameter gradients computed for each mini-batch) [(that is, determining a plurality of outputs for respective ones of the plurality of intermediate checkpoints)]”); and
determining outputs of the second DP model comprising an aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints (Marathe ¶ 0046 teaches “federation server 210 may maintain a federated machine learning model 212 [(that is, the second DP model)] and, to perform training, may distribute a current version of the machine learning model 212 [(that is, the plurality of outputs for respective ones of the plurality of intermediate checkpoints)] to the federated model user systems 220, 230, and 240 (as indicated by respective updated models 221, 233, and 243)”; Marathe ¶ 0049 teaches “[u]pon receipt of the collective modified model parameter updates, the federation server 210 may then aggregate the respective modified model parameter updates to generate aggregated model parameter updates 214 [(that is, determining outputs of the second DP model comprising an aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints)]”).
Regarding claims 13 and 27, Marathe teaches all of the limitations of claims 11 and 25, respectively, as set out above in detail.
Marathe teaches -
wherein the aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints comprises
an average of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints (Marathe, abstract, teaches a “sample of data items from a training data set is identified and respective gradients [(that is, the plurality of outputs for respective ones of the plurality of intermediate checkpoints)] for the data items are determined. The gradients are then clipped. Each subject's clipped gradients in the sample are averaged [(that is, an average of the outputs of . . . the plurality of outputs for respective ones of the plurality of intermediate checkpoints)]. A noise value is added to a sum of the averaged gradients of each of the subjects in the sample. An average gradient for the entire sample [(that is, outputs of the first DP model)] is determined from the averaged gradients of the individual subjects with the added noise value [(that is, an average of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints)]”).
Regarding claims 14 and 28, Marathe teaches all of the limitations of claims 1 and 15, respectively, as described above in detail.
Marathe teaches -
wherein the operations further comprise:
predicting, using the second DP model, an output (Marathe ¶ 0012 teaches “different training techniques may be performed to generate a machine learning model that can generate an inference (sometimes referred to as a prediction)”); and
determining, using at least one of the plurality of intermediate checkpoints, an uncertainty of the predicted output (Marathe ¶ 0056 teaches “Gradient descent training techniques may be implemented to minimize a cost function (e.g., a difference between a predicted value or inference of the machine learning model given an input from a training data set and an actual value for the input) according to a gradient and a learning rate (e.g., a “step size” or α) [(that is, “cost function” is determining, using at least one of the plurality of intermediate checkpoints, an uncertainty of the predicted output)]”; as noted in relation to Fig. 2, Marathe ¶ 0047 teaches “federated model user systems 220, 230 and 240, may independently generate locally updated versions of the machine learning models 222, 232, and 242 by training the model using local, training data sets 224, 234, and 244 [(that is, “training” includes “minimizing a cost function,” which is an uncertainty of the predicted output)]”).
Claim Rejections - 35 USC § 103
8. The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
10. This application currently names joint inventors. In considering patentability of the claims the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the Examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
11. Claims 3, 6, 10, 17, 20, and 24 are rejected under 35 U.S.C. § 103 as being unpatentable over Published Application 20230394374 to Marathe et al. [hereinafter Marathe] and McMahan et al., “Learning Differentially Private Recurrent Language Models,” arXiv (2018) [hereinafter McMahan].
Regarding claims 3 and 17, Marathe teaches all of the limitations of claims 2 and 16, respectively, as described above in detail.
Though Marathe teaches determining model parameters aggregated from a federated collection of user systems for deployment in a federated model, Marathe, however, does not explicitly teach -
wherein determining the aggregate parameter values comprises
determining a weighted sum of the parameter values of the first DP model and the parameter values of the plurality of intermediate checkpoints.
But McMahan teaches –
wherein determining the aggregate parameter values comprises
determining a weighted sum of the parameter values of the first DP model and the parameter values of the plurality of intermediate checkpoints (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training, Bounded-Sensitivity Estimators for Weighted Average Queries,” last partial paragraph, teaches “we consider weighted databases d where each row k ϵ d is associated with a particular user [(that is, “each user” provides the plurality of intermediate checkpoints in relation to the first DP model)], and has an associated weight wk ϵ [0; 1]. This weight captures the desired influence of the row on the final outcome. For example, we might think of row k containing nk different training examples all generated by user k, with weight wk proportional to nk We are then interested in a bounded-sensitivity estimate . . . for per-user vectors Δk, for example to estimate the weighted-average user update in FedeAvg [(that is, determining a weighted sum of the parameter values of the first DP model and the parameter values of the plurality of intermediate checkpoints)]”).
Marathe and McMahan are from the same or similar field of endeavor. Marathe teaches performing gradient averaging as part of training a machine learning model to enforce subject level privacy in a federated learning environment. McMahan teaches the use of weighted averages relating to training data batches in a federated learning environment.
Thus, it would have been obvious to a person having ordinary skill in the art to modify Marathe pertaining to training a machine learning model in a federated learning environment with the weighted averages affecting DP model training of McMahan.
The motivation to do so is because “[i]t is worthwhile to perform extra computation on each user’s data to minimize the number of communication rounds required to train a model, due to the significantly limited bandwidth when training data remains decentralized on mobile devices. We observe, however, that FedAvg is of interest even in the datacenter when DP is applied: larger updates are more resistant to noise, and fewer rounds of training can imply less privacy cost. Most importantly, the algorithm naturally forms per-user updates based on a single user’s data, and these updates are then averaged to compute the final update applied to the shared model on each round. As we will see, this structure makes it possible to extend the algorithm to provide a user-level differential privacy guarantee.” (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training,” second paragraph).
Regarding claims 6 and 20, Marathe teaches all of the limitations of claims 4 and 18, respectively, as described above in detail.
Though Marathe teaches minimizing a cost function based on the difference between a predicted value and an actual value, Marathe, however, does not explicitly teach –
wherein selecting the subset of intermediate checkpoints from the plurality of intermediate checkpoints comprises:
determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor; and
selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor.
But McMahan teaches -
wherein selecting the subset of intermediate checkpoints from the plurality of intermediate checkpoints comprises:
determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training,” third paragraph, teaches “[w]e also evaluate the FederatedSGD algorithm, essentially large-batch SGD where each minibatch is composed of ‘microbatches’ that include data from a single distinct user [(that is, each respective intermediate checkpoint of the plurality of intermediate checkpoints)]. In some datacenter applications FedSGD might be preferable to FedAvg, since fast networks make it more practical to run more iterations”; McMahan at p. 12, “B. Experiment Details-Accuracy Metrics,” first paragraph, teaches “We evaluate using AccuracyTop1,theprobability that the word to which the model assigns highest probability is correct (after some minimal normalization) [(that is, “AccuracyTop1” tool produces a respective quality factor)]”); and
selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor (McMahan at p. 8, “3. Experimental Results, Estimating the Accuracy of Private Models for Large Datasets,” first partial paragraph, teaches ”[b]ecause running such experiments is so computationally expensive, for experimental purposes it is useful to ask: does using an expected 1250 users per round produce a model with different accuracy than a model trained with only 100 expected users per round [(that is, “users per round” is each intermediate checkpoint of the subset of intermediate checkpoints)]? If the answer is no, we can train a model with
C
~
= 100 and a particular noise level σ, and use that model to estimate the utility of a model trained with a much larger q (and hence a much better privacy guarantee) [(that is, an equivalent accuracy between users is selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor)]”).
Marathe and McMahan are from the same or similar field of endeavor. Marathe teaches performing gradient averaging as part of training a machine learning model to enforce subject level privacy in a federated learning environment. McMahan teaches the use of a respective quality factor, as in accuracy, for training data batches in a federated learning environment.
Thus, it would have been obvious to a person having ordinary skill in the art to modify Marathe pertaining to training a machine learning model in a federated learning environment with the accuracy evaluation of McMahan.
The motivation to do so is because “[i]t is worthwhile to perform extra computation on each user’s data to minimize the number of communication rounds required to train a model, due to the significantly limited bandwidth when training data remains decentralized on mobile devices. We observe, however, that FedAvg is of interest even in the datacenter when DP is applied: larger updates are more resistant to noise, and fewer rounds of training can imply less privacy cost. Most importantly, the algorithm naturally forms per-user updates based on a single user’s data, and these updates are then averaged to compute the final update applied to the shared model on each round. As we will see, this structure makes it possible to extend the algorithm to provide a user-level differential privacy guarantee.” (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training,” second paragraph).
Regarding claims 10 and 24, Marathe teaches all of the limitations of claims 8 and 22, respectively, as shown above in detail.
Though Marathe teaches repeating the process of aggregating modified model parameter updates repeatedly until the federated machine learning model converges, Marathe, however, does not explicitly teach -
wherein selecting the subset of intermediate checkpoints from the plurality of intermediate checkpoints comprises:
determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor
But McMahan teaches -
wherein selecting the subset of intermediate checkpoints from the plurality of intermediate checkpoints comprises:
determining, for each respective intermediate checkpoint of the plurality of intermediate checkpoints, a respective quality factor (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training,” third paragraph, teaches “[w]e also evaluate the FederatedSGD algorithm, essentially large-batch SGD where each minibatch is composed of ‘microbatches’ that include data from a single distinct user [(that is, each respective intermediate checkpoint of the plurality of intermediate checkpoints)]. In some datacenter applications FedSGD might be preferable to FedAvg, since fast networks make it more practical to run more iterations”; McMahan at p. 12, “B. Experiment Details-Accuracy Metrics,” first paragraph, teaches “We evaluate using AccuracyTop1,theprobability that the word to which the model assigns highest probability is correct (after some minimal normalization) [(that is, “AccuracyTop1” is a respective quality factor)]”); and
selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor (McMahan at p. 8, “3. Experimental Results, Estimating the Accuracy of Private Models for Large Datasets,” first partial paragraph, teaches ”[b]ecause running such experiments is so computationally expensive, for experimental purposes it is useful to ask: does using an expected 1250 users per round produce a model with different accuracy than a model trained with only 100 expected users per round [(that is, “users per round” is each intermediate checkpoint of the subset of intermediate checkpoints)]? If the answer is no, we can train a model with
C
~
= 100 and a particular noise level σ, and use that model to estimate the utility of a model trained with a much larger q (and hence a much better privacy guarantee) [(that is, an equivalent accuracy between users is selecting each intermediate checkpoint of the subset of intermediate checkpoints based on the respective quality factor)]”).
Marathe and McMahan are from the same or similar field of endeavor. Marathe teaches performing gradient averaging as part of training a machine learning model to enforce subject level privacy in a federated learning environment. McMahan teaches the use of weighted averages relating to training data batches in a federated learning environment.
Thus, it would have been obvious to a person having ordinary skill in the art to modify Marathe pertaining to training a machine learning model in a federated learning environment with the weighted averages affecting DP model training of McMahan.
The motivation to do so is because “[i]t is worthwhile to perform extra computation on each user’s data to minimize the number of communication rounds required to train a model, due to the significantly limited bandwidth when training data remains decentralized on mobile devices. We observe, however, that FedAvg is of interest even in the datacenter when DP is applied: larger updates are more resistant to noise, and fewer rounds of training can imply less privacy cost. Most importantly, the algorithm naturally forms per-user updates based on a single user’s data, and these updates are then averaged to compute the final update applied to the shared model on each round. As we will see, this structure makes it possible to extend the algorithm to provide a user-level differential privacy guarantee.” (McMahan at p. 3, “2. Algorithms for User-Level Differentially Private Training,” second paragraph).
12. Claims 12 and 26 are rejected under 35 U.S.C. § 103 as being unpatentable over Published Application 20230394374 to Marathe et al. [hereinafter Marathe] in view of US Published Application 20220121999 to Wang et al. [hereinafter Wang].
Regarding claims 12 and 26, Marathe teaches all of the limitations of claims 11 and 25, respectively, as shown above in detail.
Though Marathe teaches federated users may train for several mini-batches, each having a training instance, Marathe, however, does not explicitly teach –
[(g)] wherein the aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints comprises
[(g.1)] a majority vote based on the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints.
But Wang teaches -
[(g)] wherein the aggregate of the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints comprises
[(g.1)] a majority vote based on the outputs of the first DP model and the plurality of outputs for respective ones of the plurality of intermediate checkpoints (Wang, Fig. 6, teaches data processing and optimization of system weights [Examiner annotations in dashed-line text boxes]:
PNG
media_image5.png
510
689
media_image5.png
Greyscale
Wang ¶ 0045 teaches “post-processing 134 can include, as one example, a ‘one-hot’ quantization, such that in the output vector, the most likely label corresponds to a value of 1 and all other labels correspond to a value of zero. This can provide a majority voting-based ensemble”; Wang ¶ 0047 teaches an “optimal set of wi's can be directly recomputed with the goal is modified to only fit a subset of clients' data [(that is, “subset” provides respective ones of the plurality of intermediate checkpoints)], without sharing models or recomputing the aggregated information again”).
Marathe and Wang are from the same or similar field of endeavor. Marathe teaches performing gradient averaging as part of training a machine learning model to enforce subject level privacy in a federated learning environment. Wang teaches evaluating prediction models on at least a portion of a local dataset resident on each of the plurality of clients to output a quantification indicating how each of the prediction models fit at least the portion of the local dataset of each of the plurality of clients.
Thus, it would have been obvious to a person having ordinary skill in the art to modify Marathe pertaining to training a machine learning model in a federated learning environment with the majority voting post-processing of Wang.
The motivation to do so is because “systems and computerized methods provide a technical improvement in the efficiency, scalability and privacy of data in model training computer systems by using only the models to be shared between clients and by permitting the incremental/decremental updates of the ensemble model without the need to re-evaluate all the models” (Wang ¶ 0034).
Conclusion
13. The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
(Vu et al, "dpUGC: Learn Differentially Private Representation for User Generated Contents," arXiv (2019)) teaches a user-level approach to learn personalized differentially private word embedding model on user generated contents (UGC). To our best knowledge, this is the first work of learning user-level differentially private word embedding model from text for sharing. The proposed approaches protect the privacy of the individual from re-identification, especially provide better trade-off of privacy and data utility on UGC data for sharing. The experimental results show that the trained embedding models are applicable for the classic text analysis tasks (e.g., regression).
(Abadi et al., "Deep Learning with Differential Privacy," arXiv (2016)) teaches models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.
(US Published Application 20210360010 to Zaccak et al.) teaches preventing exfiltration of training data by feature reconstruction attacks on model instances trained on the training data during a training job. Procedural calls modify corresponding training parameters in the plurality of training parameters for respective training cycles in the training job. The system comprises a trainer configured to execute the training cycles in dependence on the modified training parameters. The trainer can determine a performance accuracy of the model instances for each of the executed training cycles. The system comprises a differential privacy estimator configured to estimate a privacy guarantee for each of the executed training cycles in dependence on the modified training parameters.
14. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEVIN L. SMITH whose telephone number is (571) 272-5964. Normally, the Examiner is available on Monday-Thursday 0730-1730.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, KAKALI CHAKI can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.L.S./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights