DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are presented for examination.
Response to Amendment
Applicant’s amendment has obviated most, but not all, of the objections to the specification, drawings, and claims identified in the last Office action. To the extent that an objection or rejection appears in the previous Office Action(s) but not this Office Action, that objection or rejection is withdrawn. To the extent that it appears both in a previous Office Action(s) and this Office Action, the objection or rejection is maintained.
Drawings
The drawings are objected to because “determination” in reference character 203 is written on two different lines. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
Claim 8 is objected to because of the following informalities: “that that a private dataset” should be “that stores a private dataset”. Claims 9-10 are objected to for dependency on claim 8. Appropriate correction is required.
Claim Interpretation
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “private device”, “public device”, and “weighted knowledge transfer device” in claim 8.
Regarding the “private device”, paragraph 53 of the specification indicates that the private device may include hard disk drives, solid state drives, or cloud-based storage repositories. Any one of these will be deemed to read on the claim. For an analysis of the other claimed limitations, see rejection under 35 USC § 112(b) infra.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
Claims 8-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The claim limitations identified above invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, with the exception of the “private device,” the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions. Therefore, it is unclear whether Applicant had possession of the claimed invention as of the effective filing date. See analysis in rejection under 35 USC § 112(b) infra for further analysis.
Claims 8-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claim limitations identified above invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, with the exception of the “private device,” the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions.
Regarding the “public device”, paragraphs 56-57 disclose the functions of the public device, but no algorithm for providing machine learning model services to users is disclosed.
Regarding the “weighted knowledge transfer device”, the claims indicate that the “weighted knowledge transfer device” comprises CPUs that perform functions for which the disclosure fails to provide the structure, material, or acts for performing, as shown in the last Office action. Therefore, a fortiori, there is insufficient structure, material, or acts for performing all the claimed functions of the “weighted knowledge transfer device.”
Therefore, the claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. For purposes of examination, any software that performs the claimed functions will be deemed to read on the claims.
Applicant may:
(a) Amend the claims so that the claim limitations will no longer be interpreted as limitations under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed functions, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the functions recited in the claims, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the functions so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed functions, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed functions and clearly links or associates the structure, material, or acts to the claimed functions, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed functions. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
All claims dependent on a claim rejected hereunder are also rejected for being dependent on a rejected base claim.
Claim Rejections - 35 USC § 101
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites a device comprising one or more CPUs; therefore, the claim is directed to the statutory category of machines.
Step 2A Prong 1: The claim recites, inter alia:
[G]enerat[ing] a public knowledge transfer dataset and a private knowledge transfer dataset by determining a set of target features shared between a public dataset and a private dataset, the private knowledge transfer dataset and the public knowledge transfer dataset both including the set of target features: This limitation could encompass mentally determining the shared target features and mentally generating the datasets based thereon.
[G]enerat[ing], based on a similarity calculation of the public knowledge transfer dataset and the private knowledge transfer dataset: a public training dataset which is a subset of the public knowledge transfer dataset that achieves a similarity threshold with respect to the private knowledge transfer dataset, and a similarity weight vector that indicates weights of a set of public features included in the public training dataset: This limitation could encompass mentally generating the public training dataset and the similarity weight vector by mentally calculating the similarity between the two datasets.
[G]enerat[ing] … a public label vector that indicates labels for the set of public features: This limitation could encompass mentally generating the public label vector.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the relevant functions are performed by “one or more central processing units (CPUs)” and that the generation of the public label vector is performed “by processing the public training dataset with a set of trained private machine learning models trained based on the private knowledge transfer dataset”. However, these amount to mere instructions to apply the judicial exception using a generic computer programmed with a generic class of computer algorithms. MPEP § 2106.05(f). The claim further recites “generat[ing] a public machine learning model based on the weight vector, the public training dataset, and the public label vector”, which amounts to a mere restriction of the judicial exception to the field of use of machine learning model generation. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable process of generating datasets with features and labels. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “divid[ing] the private knowledge transfer dataset into a plurality of partitions; … wherein each partition includes a mutually exclusive set of private features with respect to other partitions.” This limitation could encompass mentally dividing the dataset into partitions.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the division is performed by “the one or more CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the division is performed by “the one or more CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 3
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the same judicial exceptions as in claim 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that “the one or more CPUs are configured to generate the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions.” However, this amounts to a mere restriction of the judicial exception to the field of use of machine learning model training. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that “the one or more CPUs are configured to generate the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions.” However, this amounts to a mere restriction of the judicial exception to the field of use of machine learning model training. MPEP § 2106.05(h).
Claim 4
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “determining … a probability of the first set of generated features belonging to the private knowledge transfer dataset; and selecting, as the public training dataset, a subset of the set of generated features associated with a probability of belonging to the private knowledge transfer dataset that exceeds a first probability threshold.” These limitations could encompass mentally determining the probability of the set of generated features and mentally selecting the subset of generated features.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “generating a trained generator network by training a generator network with the private knowledge transfer dataset to generate a first set of generated features; generating a trained discriminator network by training a discriminator network to distinguish between the first set of generated features and a set of private features included in the private knowledge transfer dataset” and then determining the probability “using the trained discriminator network”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “generating a trained generator network by training a generator network with the private knowledge transfer dataset to generate a first set of generated features; generating a trained discriminator network by training a discriminator network to distinguish between the first set of generated features and a set of private features included in the private knowledge transfer dataset” and then determining the probability “using the trained discriminator network”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 5
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
[M]erging the private knowledge transfer dataset and the public knowledge transfer dataset into a merged dataset: This limitation could encompass mentally merging the datasets.
[D]etermining, by … process[ing] the public knowledge transfer dataset, a probability indicating a likelihood that the set of public features belongs to the private knowledge transfer dataset: This limitation could encompass mentally determining the probability that the features belong to the dataset.
[S]electing, as the public training dataset, a subset of the set of public features associated with a probability of belonging to the private knowledge transfer dataset that exceeds a first probability threshold: This limitation could encompass mentally selecting the subset of features exceeding the probability threshold as the public training set.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that determining the probability is performed “by using the trained discriminator network” that is “generat[ed] … by training a discriminator network to classify a first set of features of the merged dataset as belonging to the private knowledge transfer dataset or the public knowledge transfer dataset”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that determining the probability is performed “by using the trained discriminator network” that is “generat[ed] … by training a discriminator network to classify a first set of features of the merged dataset as belonging to the private knowledge transfer dataset or the public knowledge transfer dataset”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 6
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites that “the similarity calculation is selected from the group consisting of a Euclidean distance calculation method, a Manhattan distance calculation method, a Chebyshev distance calculation method, and a Mahalanobis distance calculation method.” The generation of the training datasets using this similarity calculation remains mentally performable under these additional limitations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 7
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A Prong 1: The claim recites, inter alia:
[G]enerating a public knowledge transfer dataset and a private knowledge transfer dataset by determining a set of target features shared between the public dataset and the private dataset, the private knowledge transfer dataset and the public knowledge transfer dataset both including the set of target features: This limitation could encompass mentally generating the datasets by mentally determining the shared target features.
[G]enerating, based on a similarity calculation of the public knowledge transfer dataset and the private knowledge transfer dataset: a public training dataset which is a subset of the public knowledge transfer dataset that achieves a similarity threshold with respect to the private knowledge transfer dataset, and a similarity weight vector that indicates weights of a set of public features included in the public training dataset: This limitation could encompass mentally generating the public training set and similarity weight vector by mentally calculating a similarity between the public and private knowledge transfer datasets.
[G]enerating, by processing the public training dataset …, a public label vector that indicates labels for the set of public features: This limitation could encompass mentally generating a public label vector.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the method is “implemented in a weighted knowledge transfer device” and that the generation of the public label vector is performed with a “set of trained private machine learning models” that are generated “by training a set of machine learning models with the private knowledge transfer dataset”. However, these amount to mere instructions to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f). The claim further recites “generating a public machine learning model based on the weight vector, the public training dataset, and the public label vector.” However, this merely limits the judicial exception to the field of use of generating machine learning models. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable method of generating datasets and label vectors. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 8
Step 1: The claim recites a weighted knowledge transfer system comprising one or more CPUs; therefore, the claim is directed to the statutory category of machines.
Step 2A Prong 1: The claim recites, inter alia:
[G]enerat[ing] a public knowledge transfer dataset and a private knowledge transfer dataset by determining a set of target features shared between a public dataset and a private dataset, the private knowledge transfer dataset and the public knowledge transfer dataset both including the set of target features: This limitation could encompass mentally generating the datasets by mentally determining the shared target features.
[D]ivid[ing] the private dataset into a first plurality of partitions: This limitation could encompass mentally dividing the dataset.
[G]enerating, based on a similarity calculation of the public knowledge transfer dataset and the private knowledge transfer dataset: a public training dataset which is a subset of the public knowledge transfer dataset that achieves a similarity threshold with respect to the private knowledge transfer dataset, and a similarity weight vector that indicates weights of a set of public features included in the public training dataset: This limitation could encompass mentally generating the public training set and similarity weight vector by mentally calculating a similarity between the public and private knowledge transfer datasets.
[G]enerat[ing], as a processed public dataset, a group of labels and weights created by processing the public dataset …, divide the processed public dataset into a second plurality of partitions based on a set of thresholds determined based on the group of labels and weights: This limitation could encompass mentally generating the labels and weights and mentally partitioning the dataset based on the thresholds.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the method is performed using a “weighted knowledge transfer system comprising: a private device including one or more first CPUs that [stores] a private dataset; a public device including one or more third CPUs configured to include a public dataset and provide machine learning model based services to users; and a weighted knowledge transfer device including one or more second CPUs configured to generate a set of trained public machine learning models using the private dataset and the public dataset, wherein the weighted knowledge transfer device is communicably connected to the private device via a first network connection, wherein the weighted knowledge transfer device is communicably connected to the public device via a second network connection different from the first network connection, [and] wherein the private device is inaccessible from the public device”. However, these limitations amount to mere instructions to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
The claim further recites “generat[ing] a set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the first plurality of partitions” and “generat[ing] a set of trained public machine learning models by training each of a set of machine learning models based on a separate partition of the second plurality of partitions”. These limitations merely restrict the field of use of the judicial exception to generating trained machine learning models. MPEP § 2106.05(h).
Finally, the claims recite “deploy[ing] the set of trained public machine learning models to the public device for provision to users.” This limitation recites the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2, with the exception that the deploying limitation, in addition to reciting insignificant extra-solution activity, also recites the well-understood, routine, and conventional activity of receiving or transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network). As an ordered whole, the claim is directed to a mentally performable process of dividing datasets into partitions and assigning labels and weights to them. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 9
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “evaluat[ing] a performance of the set of trained private machine learning models with respect to a set of external test data selected from the private dataset; and determin[ing], based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a first set of model configuration parameters for the set of trained private machine learning models that achieves a predetermined performance criterion.” These limitations could encompass mentally evaluating the performance of the models by visually observing their outputs and mentally determining what the model configuration parameters should be based thereon.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the above functions are performed by “the one or more second CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the above functions are performed by “the one or more second CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 10
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “evaluat[ing] a performance of the set of trained public machine learning models with respect to a set of external test data selected from the private dataset; and determin[ing], based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a second set of model configuration parameters for the set of trained public machine learning models that achieves a predetermined performance criterion.” These limitations could encompass mentally evaluating the performance of the models by observing their outputs when fed the external test data and mentally determining the second set of model configuration parameters based thereon.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the above functions are performed by “the one or more second CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the above functions are performed by “the one or more second CPUs”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim Rejections - 35 USC § 103
Claims 1-3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot et al., “Semi-Supervised Knowledge Transfer for Deep Learning from Private Training Data,” in arXiv preprint arXiv:1610.05755 (2017) (“Papernot”) in view of Sidorov et al., “Soft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model,” in 18.3 Computación y Sistemas 491-504 (2014) (“Sidorov”) and further in view of Antonatos et al. (US 20200265069) (“Antonatos”).
Regarding claim 1, Papernot discloses “[a] weighted knowledge transfer device comprising:
one or more central processing units (CPUs) (Papernot sec. 1, first bullet point, discloses a machine learning strategy that provides differential privacy for training data in a black-box manner [the recitation of machine learning implies the existence of a processor such as a CPU to execute the machine learning]) programmed to:
generate a public knowledge transfer dataset and a private knowledge transfer dataset (Papernot Figure 1 discloses a set of sensitive data [private knowledge transfer dataset] and a set of public data [public knowledge transfer dataset]) …, …;
generate …:
a public training dataset which is a subset of the public knowledge transfer dataset (Papernot Fig. 1 discloses a set of incomplete public data [subset of the public knowledge transfer dataset]) …, …;
generate, by processing the public training dataset with a set of trained private machine learning models trained based on the private knowledge transfer dataset, a public label [set] that indicates labels for the set of public features (Papernot Fig. 1 shows that the dataset used to train the student model is labeled [to create a public label set indicating labels for the set of public features] with an aggregate teacher model aggregated from a set of models [private machine learning models] trained based on the sensitive data [private knowledge transfer dataset] and that are not accessible by an adversary), and
generate a public machine learning model based on … the public training dataset[] and the public label [set] (Papernot Fig. 1 discloses that a student model accessible to an adversary [i.e., a public machine learning model] is generated based on the incomplete public data [public training dataset] and the labels produced by the aggregate teacher model [public label set]).”
Papernot appears not to disclose explicitly the further limitations of the claim. However, Sidorov discloses “determining a set of target features shared between a [first] dataset and a [second] dataset (Hamming distance counts how many features two vectors do not share [i.e., it indirectly counts how many they do share] – Sidorov, paragraph spanning pp. 492-93) …; …
generat[ing], based on a similarity calculation of the … [first] dataset and the … [second] dataset:
a … dataset that achieves a similarity threshold with respect to the [second] dataset (there is almost always some similarity between words which can be identified independently from the vector space model; a natural assumption can be made that this similarity can be modeled as a cosine between these objects – Sidorov, p. 494, paragraphs immediately preceding and containing Equation (8); a small number of highly similar pairs of features [feature sets = datasets] is identified whose similarity measure is greater than a threshold – id. at sec. 2.3, second paragraph), and
a similarity weight vector that indicates weights of a set of … features included in the … dataset (if the similarity between features is given by some list, automatically only a small number of elements in the matrix sij of similarities will not equal zero; the matrix sij can be operated upon as a sparse matrix – Sidorov, sec. 2.3, third-fourth paragraphs [i.e., sij is a series of weight vectors showing the similarity weights between feature i and each feature j = 1, …, n]); … [and]
generat[ing] a … model based on the weight vector (sij is used to construct data vectors a and b, and a and b are used to form a soft cosine value [model] between pairs of features that represents a weight or importance that is a similarity of the two features – Sidorov, sec. 2.2, paragraph containing Equations (15)-(16)) ….”
Sidorov further discloses that the data are organized as vectors (traditional cosine measure is commonly used to determine the similarity between two objects represented as vectors – Sidorov, p. 49, first full paragraph) ….”
Sidorov and the instant application both relate to similarity metrics in artificial intelligence and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot to generate a similarity vector indicating the similarities between features, as disclosed by Sidorov, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to account for similarity between data points, thereby creating a more robust model. See Sidorov, abstract.
Neither Papernot nor Sidorov appears to disclose explicitly the further limitations of the claim. However, Antonatos discloses that “the private knowledge transfer dataset and the public knowledge transfer dataset both includ[e] the set of target features (knowledge base may include public and/or private datasets related to target data such as population/census data, voter lists, client lists, etc. [i.e., the two datasets have the same target features] – Antonatos, paragraph 77) ….”
Antonatos and the instant application both relate to data privacy and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Papernot and Sidorov to have both public and private datasets share target features, as disclosed by Antonatos, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would ensure that both can be used for similar purposes while preserving data privacy. See Antonatos, paragraph 77.
Regarding claim 2, Papernot, as modified by Sidorov/Antonatos, discloses that “the one or more CPUs are configured to divide the private knowledge transfer dataset into a plurality of partitions (see mapping of limitation below), …
wherein each partition includes a mutually exclusive set of private features with respect to other partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint [mutually exclusive] sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”; see also Fig. 1 [showing that the training data are sensitive and not accessible by an adversary, i.e., private]).”
Regarding claim 3, Papernot, as modified by Sidorov/Antonatos, discloses that “the one or more CPUs are configured to generate the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint [mutually exclusive] sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”).”
Claim 7 is a method claim corresponding to device claim 1 and is rejected for the same reasons as given in the rejection of that claim, with the exception that claim 7 additionally recites the following limitations, taught by Papernot: “receiving a public dataset and a private dataset (Papernot Fig. 1 shows the receipt of sensitive data [public dataset] and incomplete public data [private dataset]); … [and]
generating, by training a set of machine learning models with the private knowledge transfer dataset, a set of trained private machine learning models (Papernot Fig. 1 shows that the sensitive data are partitioned into n disjoint sets [private knowledge transfer dataset], each of which is used to trained a teacher model that is not accessible by an adversary [i.e., private]) ….”
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Papernot in view of Sidorov and Antonatos and further in view of Afshar et al. (US 20210224582) (“Afshar”).
Regarding claim 6, neither Papernot, Antonatos, nor Sidorov appears to disclose explicitly the further limitations of the claim. However, Afshar discloses that “the similarity calculation is selected from the group consisting of a Euclidean distance calculation method, a Manhattan distance calculation method, a Chebyshev distance calculation method, and a Mahalanobis distance calculation method (similarity value may be a distance metric between a first vector space distance and a second vector space distance in vector space (e.g., Euclidean distance or Chebyshev distance) – Afshar, paragraph 68).”
Afshar and the instant application both relate to distance metrics in machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Papernot, Antonatos, and Sidorov to calculate a Chebyshev distance as the similarity metric, as disclosed by Afshar, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would give the system an objective measure of similarity between two data points, thereby increasing the information with which the system may work. See Afshar, paragraph 68.
Claims 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot in view of Tetko et al., “Efficient Partition of Learning Data Sets for Neural Network Training,” in 10.8 Neural Networks 1361-74 (1997) (“Tetko”) and further in view of Sidorov and Antonatos.
Regarding claim 8, Papernot discloses “[a] weighted knowledge transfer system comprising:
a private device including one or more … CPUs that [stores] a private dataset (Papernot Fig. 1 shows sensitive data [stored on a private device]; sec. 1, first bullet point, discloses a machine learning strategy that provides differential privacy for training data in a black-box manner [the recitation of machine learning implies the existence of a processor such as a CPU to execute the machine learning]);
a public device including one or more … CPUs configured to include a public dataset and provide machine learning model[-]based services to users (Papernot Fig. 1 shows a public device that comprises a student model [machine learning model-based services] accessible by an adversary [user] and incomplete public data [public dataset]); and
a weighted knowledge transfer device including one or more … CPUs configured to generate a set of trained public machine learning models using the private dataset and the public dataset (Papernot Fig. 1 shows a student model accessible by an adversary [set of trained public machine learning models] that is trained using incomplete public data [public dataset] and predictions given by the aggregated teacher models [themselves trained on sensitive data, i.e., a private dataset; note that the incomplete public data and their private labels may be considered to be the weighted knowledge transfer device]),
wherein the weighted knowledge transfer device is communicably connected to the private device via a first network connection (Papernot Fig. 1 shows that the aggregate teacher model outputs labels for the incomplete data, i.e., is connected to the system that provides the data for the student model),
wherein the weighted knowledge transfer device is communicably connected to the public device via a second network connection different from the first network connection (Papernot Fig. 1 shows that the public data [i.e., the system that provides the data for the student model, or the knowledge transfer device] are fed to the student model via a different connection from that used to transfer the outputs of the aggregate teacher);
wherein the private device is inaccessible from the public device (Papernot Fig. 1 shows that the student and the incomplete public data [public device] are accessible by an adversary and that the remainder of the system [private device] is not),
wherein the one or more … CPUs of the weighted knowledge transfer device are configured to:
generate a public knowledge transfer dataset and a private knowledge transfer dataset (Papernot Figure 1 discloses a set of sensitive data [private knowledge transfer dataset] and a set of public data [public knowledge transfer dataset]) …, …;
divide the private dataset into a first plurality of partitions (see mapping of limitation below), and
generate a set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the first plurality of partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”; see also Fig. 1 [showing that the ensemble of teachers is not accessible to the adversary, i.e., they are private]);
generate …:
a public training dataset which is a subset of the public knowledge transfer dataset (Papernot Fig. 1 discloses a set of incomplete public data [subset of the public knowledge transfer dataset]) …, …;
generate, as a processed public dataset, a group of labels … created by processing the public dataset using the set of trained private machine learning models (Papernot Fig. 1 and associated caption show that the incomplete public data are processed using the aggregated teacher model [trained private machine learning models] to predict labels and complete the public data), …; and
execute machine learning to generate a set of trained public machine learning models …, and deploy the set of trained public machine learning models to the public device for provision to users (Papernot Fig. 1 shows that the student model [set of trained public machine learning models] is generated using the completed public data and is accessible to an adversary [i.e., deployed to a public device] for querying [i.e., provision to users]).”
Papernot appears not to disclose explicitly the further limitations of the claim. However, Tetko discloses “generat[ing] … a group of … weights (unsupervised learning algorithm uses arrays of predicted values to subdivide the initial training set into a number of clusters Q; for each cluster q1, a normalized mean squared error between the target values and ensemble predictions was calculated; a weight proportional to the mean squared error [weight] is attributed to each cluster – Tetko, p. 1366, second full paragraph) …;
divid[ing] the processed … dataset into a second plurality of partitions based on a set of thresholds determined based on the group of labels and weights (unsupervised learning algorithm uses arrays of predicted values to subdivide [partition] the initial training set into a number of clusters Q; for each cluster q1, a normalized mean squared error between the target values [labels] and ensemble predictions was calculated; a weight proportional to the mean square error [i.e., based on the labels] is attributed to each cluster, and a case is randomly selected given a probability [threshold] of selecting a case xi belonging to q proportional to the weight – Tetko, p. 1366, second full paragraph); and …
training each of a set of machine learning models based on a … partition of the second plurality of partitions (for each ANN [set of machine learning models] computation begins with two cases in the learning data set selected at random; at the end of the first iteration, four additional cases are selected from the input data set and added to the corresponding data sets; thus, the second iteration of the training procedure for each ANN is based upon four cases in the learning data set – Tetko, p. 1366, third full paragraph) ….”
Tetko and the instant application both relate to dataset partitioning for machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot to partition the dataset based on thresholds, as disclosed by Tetko, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would speed up the training of each network and reduce overfitting by ensuring that each network is only trained on a portion of the data. See Tetko, sec. 3, first paragraph.
Neither Tetko nor Papernot appears to disclose explicitly the further limitations of the claim. However, Sidorov discloses “determining a set of target features shared between a [first] dataset and a [second] dataset (Hamming distance counts how many features two vectors do not share [i.e., it indirectly counts how many they do share] – Sidorov, paragraph spanning pp. 492-93) …; … [and]
generat[ing], based on a similarity calculation of the … [first] dataset and the … [second] dataset:
a … dataset that achieves a similarity threshold with respect to the [second] dataset (there is almost always some similarity between words which can be identified independently from the vector space model; a natural assumption can be made that this similarity can be modeled as a cosine between these objects – Sidorov, p. 494, paragraphs immediately preceding and containing Equation (8); a small number of highly similar pairs of features [feature sets = datasets] is identified whose similarity measure is greater than a threshold – id. at sec. 2.3, second paragraph), and
a similarity weight vector that indicates weights of a set of … features included in the … dataset (if the similarity between features is given by some list, automatically only a small number of elements in the matrix sij of similarities will not equal zero; the matrix sij can be operated upon as a sparse matrix – Sidorov, sec. 2.3, third-fourth paragraphs [i.e., sij is a series of weight vectors showing the similarity weights between feature i and each feature j = 1, …, n]) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot/Tetko to generate a similarity vector indicating the similarities between features, as disclosed by Sidorov, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to account for similarity between data points, thereby creating a more robust model. See Sidorov, abstract.
Neither Papernot, Tetko, nor Sidorov appears to disclose explicitly the further limitations of the claim. However, Antonatos discloses “one or more first CPUs …; … one or more third CPUs …; and … one or more second CPUs (components of computer system/server may include one or more processors or processing units [e.g., first, second, and third CPUs] – Antonatos, paragraph 43), … [wherein]
the private knowledge transfer dataset and the public knowledge transfer dataset both includ[e] the set of target features (knowledge base may include public and/or private datasets related to target data such as population/census data, voter lists, client lists, etc. [i.e., the two datasets have the same target features] – Antonatos, paragraph 77) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Papernot, Tetko, and Sidorov to have both public and private datasets share target features, as disclosed by Antonatos, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would ensure that both can be used for similar purposes while preserving data privacy. See Antonatos, paragraph 77.
Regarding claim 9, Papernot, as modified by Tetko, Antonatos, and Sidorov, discloses that “the one or more second CPUs are configured to:
evaluate a performance of the set of trained private machine learning models with respect to a set of external test data selected from the private dataset (number n of partitions of private data for training the teacher models [private models] is tested by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”; see also sec. 4.1, first paragraph (indicating that the ensemble of teachers was evaluated on respective test sets, i.e., sets culled from the private data)); and
determine, based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a first set of model configuration parameters for the set of trained private machine learning models that achieves a predetermined performance criterion (number n of teachers [model configuration parameter] is limited by a tradeoff between the classification task’s complexity and the available data; larger values of n lead to larger absolute gaps, but imply a smaller training dataset for each teacher; potentially reducing the accuracy [accuracy/complexity = predetermined performance criteria]; number n of partitions of private data for training the teacher models [private models] is tested [determined] by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”).”
Regarding claim 10, the rejection of claim 8 is incorporated. Papernot further discloses that “the one or more second CPUs are configured to:
evaluate a performance of the set of trained public machine learning models with respect to a set of external test data selected from the private dataset (number n of partitions of private data for training the teacher models [private models] is tested by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”; see also sec. 4.1, first paragraph (indicating that the ensemble of teachers was evaluated on respective test sets, i.e., sets culled from the private data)); and
determine, based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a … set of model configuration parameters for the set of trained public machine learning models that achieves a predetermined performance criterion (number n of teachers [model configuration parameter] is limited by a tradeoff between the classification task’s complexity and the available data; larger values of n lead to larger absolute gaps, but imply a smaller training dataset for each teacher; potentially reducing the accuracy [accuracy/complexity = predetermined performance criteria]; number n of partitions of private data for training the teacher models [private models] is tested [determined] by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”).”
Papernot/Sidorov/Antonatos appears not to disclose explicitly the further limitations of the claim. However, Tetko discloses “determin[ing] … a second set of model configuration parameters (methods aimed to decrease the number of adjustable parameters in an ANN include pruning algorithms, penalized regularization algorithms, and specialized architexture and weight sharing – Tetko, p. 1362, second full paragraph) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot/Sidorov/Antonatos to determine a set of model configuration parameters, as disclosed by Tetko, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would speed up the training of each network and reduce overfitting by ensuring that each network is only trained on a portion of the data. See Tetko, sec. 3, first paragraph.
Response to Arguments
Applicant's arguments filed May 14, 2026 (“Remarks”) have been fully considered but they are, except insofar as rendered moot by the entry of a new ground of rejection, not persuasive.
Applicant first argues that the amended claims are eligible under 35 USC § 101 because the specification discloses an improvement to the technical field of creating a privacy preserving public machine learning model that achieves high performance while maintaining data privacy and that this improvement is reflected in the limitations reciting generation of the machine learning models. Remarks at 14-17. However, even assuming arguendo that the specification recites an improvement to a technical field, which Examiner does not concede, this improvement is not reflected in the claims themselves. As noted in the rejection, the generation of the models is recited at a high level of generality and amounts to little more than a recitation of a field of use of the judicial exception of generating datasets and vectors. As currently recited, the claims do not recite an improvement to machine learning model generation, but merely the use of abstractly generated public and private datasets and weight and label vectors to generate the model.
Applicant’s arguments with respect to the art rejection, Remarks at 17-20, are moot in light of the use of newly cited reference Antonatos to teach the disputed limitations.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125
Prosecution Insights